Getting this error: mismatched input 'from' expecting <EOF> while Spark SQL

Question

While running a Spark SQL, I am getting mismatched input 'from' expecting <EOF> error.

I checked the common syntax errors which can occur but didn't find any.

val prevPartitionDate = spark.sql(
"select to_date(concat(year_p,'-',month_p,'-',day_p)) 
from ips.command_room_table 
where 
to_date(concat(year_p,'-',month_p,'-',day_p)) < to_date(concat($yearAsInt,'-',$monthAsInt,'-',$dayAsInt)) 
group by year_p, month_p, day_p 
order by to_date(concat(year_p,'-',month_p,'-',day_p)) desc limit 1"
).first.getDate(0)

What can I do to resolve this?

did u check "$" sign?

Sathiyan S
– Sathiyan S

2020-12-21 10:49:53 +00:00
Commented Dec 21, 2020 at 10:49 — Sathiyan S
– Sathiyan S, Commented Dec 21, 2020 at 10:49

Tech Fukrey · Accepted Answer · 2020-12-21 10:58:06Z

1

No worries, able to figure out the issue. As I was using the variables in the query, I just have to add 's' at the beginning of the query like this:

val prevPartitionDate = spark.sql(s
"select to_date(concat(year_p,'-',month_p,'-',day_p)) 
from ips.command_room_table 
where 
to_date(concat(year_p,'-',month_p,'-',day_p)) < to_date(concat($yearAsInt,'-',$monthAsInt,'-',$dayAsInt)) 
group by year_p, month_p, day_p 
order by to_date(concat(year_p,'-',month_p,'-',day_p)) desc limit 1"
).first.getDate(0)

answered Dec 21, 2020 at 10:58

Tech Fukrey

392 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Getting this error: mismatched input 'from' expecting <EOF> while Spark SQL

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related