Need help understanding PySpark explain output

Question

My query is timing out and I want to understand the output of the explain command to better understand whats the problem.

First my query looks like:

WITH f (
    SELECT  
        /*+ BROADCAST(h) */
        /*+ COALESCE(36) */
        CONCAT(f.outboundlegid, '-', f.inboundlegid, '-', f.agent) AS key,
        f.querydatetime,

        f.outboundlegid,
        f.inboundlegid,
        f.agent,
        f.queryoutbounddate,
        f.queryinbounddate,
        f.price,
        f.outdeparture,
        f.outarrival,
        f.indeparture,
        f.inarrival,
        f.querydestinationplace,
        CASE WHEN type = 'HOLIDAY' AND (out_date BETWEEN start AND end)
            THEN true
            ELSE false
            END out_is_holiday,
        CASE WHEN type = 'LONG_WEEKENDS' AND (out_date BETWEEN start AND end)
            THEN true
            ELSE false
            END out_is_longweekends,
        CASE WHEN type = 'HOLIDAY' AND (in_date BETWEEN start AND end)
            THEN true
            ELSE false
            END in_is_holiday,
        CASE WHEN type = 'LONG_WEEKENDS' AND (in_date BETWEEN start AND end)
            THEN true
            ELSE false
            END in_is_longweekends
    FROM flights f
    CROSS JOIN holidays h
    LIMIT 10
)
 SELECT
    /*+ BROADCAST(a) */
    /*+ BROADCAST(p) */
    key,
    querydatetime,
    first(outboundlegid) as outboundlegid,
    first(inboundlegid) as inboundlegid,
    first(agent) as agent,
    first(p.countryName) as countryName,
    first(p.airportName) as airportName,
    first(a.name) as agentName,
    first(queryoutbounddate) as queryoutbounddate,
    first(queryinbounddate) as queryinbounddate,
    first(price) as price,
    first(outdeparture) as outdeparture,
    first(outarrival) as outarrival,
    first(indeparture) as indeparture,
    first(inarrival) as inarrival,
    first(querydestinationplace) as querydestinationplace,
    CASE WHEN array_contains(collect_set(out_is_holiday), true)
        THEN true
        ELSE false
        END out_is_holiday,
    CASE WHEN array_contains(collect_set(out_is_longweekends), true)
        THEN true
        ELSE false
        END out_is_longweekends,
    CASE WHEN array_contains(collect_set(in_is_holiday), true)
        THEN true
        ELSE false
        END in_is_holiday,
    CASE WHEN array_contains(collect_set(in_is_longweekends), true)
        THEN true
        ELSE false
        END in_is_longweekends
FROM f
INNER JOIN agents a
ON f.agent = a.id
INNER JOIN airports p
ON f.querydestinationplace = p.airportId
GROUP BY 
    querydatetime, 
    key

Then my explain output:

Parsed Logical Plan

== Parsed Logical Plan ==
CTE [f]
: +- 'SubqueryAlias f
: +- 'GlobalLimit 10
: +- 'LocalLimit 10
: +- 'UnresolvedHint COALESCE, [36]
: +- 'Project ['CONCAT('f.outboundlegid, -, 'f.inboundlegid, -, 'f.agent) AS key#351, 'f.querydatetime, 'f.outboundlegid, 'f.inboundlegid, 'f.agent, 'f.queryoutbounddate, 'f.queryinbounddate, 'f.price, 'f.outdeparture, 'f.outarrival, 'f.indeparture, 'f.inarrival, 'f.querydestinationplace, CASE WHEN (('type = HOLIDAY) && (('out_date >= 'start) && ('out_date <= 'end))) THEN true ELSE false END AS out_is_holiday#352, CASE WHEN (('type = LONG_WEEKENDS) && (('out_date >= 'start) && ('out_date <= 'end))) THEN true ELSE false END AS out_is_longweekends#353, CASE WHEN (('type = HOLIDAY) && (('in_date >= 'start) && ('in_date <= 'end))) THEN true ELSE false END AS in_is_holiday#354, CASE WHEN (('type = LONG_WEEKENDS) && (('in_date >= 'start) && ('in_date <= 'end))) THEN true ELSE false END AS in_is_longweekends#355]
: +- 'Join Cross
: :- 'SubqueryAlias f
: : +- 'UnresolvedRelation `flights`
: +- 'SubqueryAlias h
: +- 'UnresolvedRelation `holidays`
+- 'GlobalLimit 10
+- 'LocalLimit 10
+- 'UnresolvedHint BROADCAST, ['a]
+- 'UnresolvedHint BROADCAST, ['p]
+- 'Aggregate ['querydatetime, 'key], ['key, 'querydatetime, first('outboundlegid, false) AS outboundlegid#320, first('inboundlegid, false) AS inboundlegid#322, first('agent, false) AS agent#324, first('p.countryName, false) AS countryName#326, first('p.airportName, false) AS airportName#328, first('a.name, false) AS agentName#330, first('queryoutbounddate, false) AS queryoutbounddate#332, first('queryinbounddate, false) AS queryinbounddate#334, first('price, false) AS price#336, first('outdeparture, false) AS outdeparture#338, first('outarrival, false) AS outarrival#340, first('indeparture, false) AS indeparture#342, first('inarrival, false) AS inarrival#344, first('querydestinationplace, false) AS querydestinationplace#346, CASE WHEN 'array_contains('collect_set('out_is_holiday), true) THEN true ELSE false END AS out_is_holiday#347, CASE WHEN 'array_contains('collect_set('out_is_longweekends), true) THEN true ELSE false END AS out_is_longweekends#348, CASE WHEN 'array_contains('collect_set('in_is_holiday), true) THEN true ELSE false END AS in_is_holiday#349, CASE WHEN 'array_contains('collect_set('in_is_longweekends), true) THEN true ELSE false END AS in_is_longweekends#350]
+- 'Join Inner, ('f.querydestinationplace = 'p.airportId)
:- 'Join Inner, ('f.agent = 'a.id)
: :- 'UnresolvedRelation `f`
: +- 'SubqueryAlias a
: +- 'UnresolvedRelation `agents`
+- 'SubqueryAlias p
+- 'UnresolvedRelation `airports`

Analyzed Logical Plan

== Analyzed Logical Plan ==
key: string, querydatetime: date, outboundlegid: string, inboundlegid: string, agent: string, countryName: string, airportName: string, agentName: string, queryoutbounddate: string, queryinbounddate: string, price: string, outdeparture: string, outarrival: string, indeparture: string, inarrival: string, querydestinationplace: int, out_is_holiday: boolean, out_is_longweekends: boolean, in_is_holiday: boolean, in_is_longweekends: boolean
GlobalLimit 10
+- LocalLimit 10
+- Aggregate [querydatetime#207, key#351], [key#351, querydatetime#207, first(outboundlegid#184, false) AS outboundlegid#320, first(inboundlegid#185, false) AS inboundlegid#322, first(agent#181, false) AS agent#324, first(countryName#24, false) AS countryName#326, first(airportName#22, false) AS airportName#328, first(name#74, false) AS agentName#330, first(queryoutbounddate#177, false) AS queryoutbounddate#332, first(queryinbounddate#178, false) AS queryinbounddate#334, first(price#183, false) AS price#336, first(outdeparture#186, false) AS outdeparture#338, first(outarrival#187, false) AS outarrival#340, first(indeparture#196, false) AS indeparture#342, first(inarrival#197, false) AS inarrival#344, first(querydestinationplace#206, false) AS querydestinationplace#346, CASE WHEN array_contains(collect_set(out_is_holiday#352, 0, 0), true) THEN true ELSE false END AS out_is_holiday#347, CASE WHEN array_contains(collect_set(out_is_longweekends#353, 0, 0), true) THEN true ELSE false END AS out_is_longweekends#348, CASE WHEN array_contains(collect_set(in_is_holiday#354, 0, 0), true) THEN true ELSE false END AS in_is_holiday#349, CASE WHEN array_contains(collect_set(in_is_longweekends#355, 0, 0), true) THEN true ELSE false END AS in_is_longweekends#350]
+- Join Inner, (querydestinationplace#206 = cast(airportId#38 as int))
:- Join Inner, (agent#181 = id#83)
: :- SubqueryAlias f
: : +- GlobalLimit 10
: : +- LocalLimit 10
: : +- Project [concat(outboundlegid#184, -, inboundlegid#185, -, agent#181) AS key#351, querydatetime#207, outboundlegid#184, inboundlegid#185, agent#181, queryoutbounddate#177, queryinbounddate#178, price#183, outdeparture#186, outarrival#187, indeparture#196, inarrival#197, querydestinationplace#206, CASE WHEN ((type#57 = HOLIDAY) && ((out_date#243 >= start#55) && (out_date#243 <= end#56))) THEN true ELSE false END AS out_is_holiday#352, CASE WHEN ((type#57 = LONG_WEEKENDS) && ((out_date#243 >= start#55) && (out_date#243 <= end#56))) THEN true ELSE false END AS out_is_longweekends#353, CASE WHEN ((type#57 = HOLIDAY) && ((in_date#280 >= start#55) && (in_date#280 <= end#56))) THEN true ELSE false END AS in_is_holiday#354, CASE WHEN ((type#57 = LONG_WEEKENDS) && ((in_date#280 >= start#55) && (in_date#280 <= end#56))) THEN true ELSE false END AS in_is_longweekends#355]
: : +- Join Cross
: : :- SubqueryAlias f
: : : +- SubqueryAlias flights
: : : +- Project [Id#174, QueryTaskId#175, QueryOriginPlace#176, QueryOutboundDate#177, QueryInboundDate#178, QueryCabinClass#179, QueryCurrency#180, Agent#181, QuoteAgeInMinutes#182, Price#183, OutboundLegId#184, InboundLegId#185, OutDeparture#186, OutArrival#187, OutDuration#188, OutJourneyMode#189, OutStops#190, OutCarriers#191, OutOperatingCarriers#192, NumberOutStops#193, NumberOutCarriers#194, NumberOutOperatingCarriers#195, InDeparture#196, InArrival#197, ... 12 more fields]
: : : +- Project [Id#174, QueryTaskId#175, QueryOriginPlace#176, QueryOutboundDate#177, QueryInboundDate#178, QueryCabinClass#179, QueryCurrency#180, Agent#181, QuoteAgeInMinutes#182, Price#183, OutboundLegId#184, InboundLegId#185, OutDeparture#186, OutArrival#187, OutDuration#188, OutJourneyMode#189, OutStops#190, OutCarriers#191, OutOperatingCarriers#192, NumberOutStops#193, NumberOutCarriers#194, NumberOutOperatingCarriers#195, InDeparture#196, InArrival#197, ... 11 more fields]
: : : +- LogicalRDD [Id#174, QueryTaskId#175, QueryOriginPlace#176, QueryOutboundDate#177, QueryInboundDate#178, QueryCabinClass#179, QueryCurrency#180, Agent#181, QuoteAgeInMinutes#182, Price#183, OutboundLegId#184, InboundLegId#185, OutDeparture#186, OutArrival#187, OutDuration#188, OutJourneyMode#189, OutStops#190, OutCarriers#191, OutOperatingCarriers#192, NumberOutStops#193, NumberOutCarriers#194, NumberOutOperatingCarriers#195, InDeparture#196, InArrival#197, ... 10 more fields]
: : +- SubqueryAlias h
: : +- SubqueryAlias holidays
: : +- LogicalRDD [start#55, end#56, type#57]
: +- ResolvedHint isBroadcastable=true
: +- SubqueryAlias a
: +- SubqueryAlias agents
: +- Project [cast(id#73L as string) AS id#83, name#74]
: +- Project [id#73L, name#74]
: +- LogicalRDD [id#73L, name#74, type#75]
+- ResolvedHint isBroadcastable=true
+- SubqueryAlias p
+- SubqueryAlias airports
+- Project [cast(airportId#18L as string) AS airportId#38, countryName#24, cityName#23, airportName#22]
+- Project [airportId#18L, countryName#24, cityName#23, airportName#22]
+- LogicalRDD [airportId#18L, cityId#19L, countryId#20L, airportCode#21, airportName#22, cityName#23, countryName#24]

== Optimized Logical Plan ==
GlobalLimit 10
+- LocalLimit 10
+- Aggregate [querydatetime#207, key#351], [key#351, querydatetime#207, first(outboundlegid#184, false) AS outboundlegid#320, first(inboundlegid#185, false) AS inboundlegid#322, first(agent#181, false) AS agent#324, first(countryName#24, false) AS countryName#326, first(airportName#22, false) AS airportName#328, first(name#74, false) AS agentName#330, first(queryoutbounddate#177, false) AS queryoutbounddate#332, first(queryinbounddate#178, false) AS queryinbounddate#334, first(price#183, false) AS price#336, first(outdeparture#186, false) AS outdeparture#338, first(outarrival#187, false) AS outarrival#340, first(indeparture#196, false) AS indeparture#342, first(inarrival#197, false) AS inarrival#344, first(querydestinationplace#206, false) AS querydestinationplace#346, CASE WHEN array_contains(collect_set(out_is_holiday#352, 0, 0), true) THEN true ELSE false END AS out_is_holiday#347, CASE WHEN array_contains(collect_set(out_is_longweekends#353, 0, 0), true) THEN true ELSE false END AS out_is_longweekends#348, CASE WHEN array_contains(collect_set(in_is_holiday#354, 0, 0), true) THEN true ELSE false END AS in_is_holiday#349, CASE WHEN array_contains(collect_set(in_is_longweekends#355, 0, 0), true) THEN true ELSE false END AS in_is_longweekends#350]
+- Project [key#351, querydatetime#207, outboundlegid#184, inboundlegid#185, agent#181, queryoutbounddate#177, queryinbounddate#178, price#183, outdeparture#186, outarrival#187, indeparture#196, inarrival#197, querydestinationplace#206, out_is_holiday#352, out_is_longweekends#353, in_is_holiday#354, in_is_longweekends#355, name#74, countryName#24, airportName#22]
+- Join Inner, (querydestinationplace#206 = cast(airportId#38 as int))
:- Project [key#351, querydatetime#207, outboundlegid#184, inboundlegid#185, agent#181, queryoutbounddate#177, queryinbounddate#178, price#183, outdeparture#186, outarrival#187, indeparture#196, inarrival#197, querydestinationplace#206, out_is_holiday#352, out_is_longweekends#353, in_is_holiday#354, in_is_longweekends#355, name#74]
: +- Join Inner, (agent#181 = id#83)
: :- Filter (isnotnull(agent#181) && isnotnull(querydestinationplace#206))
: : +- GlobalLimit 10
: : +- LocalLimit 10
: : +- Project [concat(outboundlegid#184, -, inboundlegid#185, -, agent#181) AS key#351, querydatetime#207, outboundlegid#184, inboundlegid#185, agent#181, queryoutbounddate#177, queryinbounddate#178, price#183, outdeparture#186, outarrival#187, indeparture#196, inarrival#197, querydestinationplace#206, CASE WHEN ((type#57 = HOLIDAY) && ((out_date#243 >= start#55) && (out_date#243 <= end#56))) THEN true ELSE false END AS out_is_holiday#352, CASE WHEN ((type#57 = LONG_WEEKENDS) && ((out_date#243 >= start#55) && (out_date#243 <= end#56))) THEN true ELSE false END AS out_is_longweekends#353, CASE WHEN ((type#57 = HOLIDAY) && ((in_date#280 >= start#55) && (in_date#280 <= end#56))) THEN true ELSE false END AS in_is_holiday#354, CASE WHEN ((type#57 = LONG_WEEKENDS) && ((in_date#280 >= start#55) && (in_date#280 <= end#56))) THEN true ELSE false END AS in_is_longweekends#355]
: : +- Join Cross
: : :- Project [QueryOutboundDate#177, QueryInboundDate#178, Agent#181, Price#183, OutboundLegId#184, InboundLegId#185, OutDeparture#186, OutArrival#187, InDeparture#196, InArrival#197, querydestinationplace#206, querydatetime#207, to_date(cast(outdeparture#186 as date)) AS out_date#243, to_date(cast(indeparture#196 as date)) AS in_date#280]
: : : +- LogicalRDD [Id#174, QueryTaskId#175, QueryOriginPlace#176, QueryOutboundDate#177, QueryInboundDate#178, QueryCabinClass#179, QueryCurrency#180, Agent#181, QuoteAgeInMinutes#182, Price#183, OutboundLegId#184, InboundLegId#185, OutDeparture#186, OutArrival#187, OutDuration#188, OutJourneyMode#189, OutStops#190, OutCarriers#191, OutOperatingCarriers#192, NumberOutStops#193, NumberOutCarriers#194, NumberOutOperatingCarriers#195, InDeparture#196, InArrival#197, ... 10 more fields]
: : +- LogicalRDD [start#55, end#56, type#57]
: +- ResolvedHint isBroadcastable=true
: +- Project [cast(id#73L as string) AS id#83, name#74]
: +- Filter (isnotnull(id#73L) && isnotnull(cast(id#73L as string)))
: +- LogicalRDD [id#73L, name#74, type#75]
+- ResolvedHint isBroadcastable=true
+- Project [cast(airportId#18L as string) AS airportId#38, countryName#24, airportName#22]
+- Filter (isnotnull(airportId#18L) && isnotnull(cast(airportId#18L as string)))
+- LogicalRDD [airportId#18L, cityId#19L, countryId#20L, airportCode#21, airportName#22, cityName#23, countryName#24]

Physical Plan

== Physical Plan ==
CollectLimit 10
+- ObjectHashAggregate(keys=[querydatetime#207, key#351], functions=[first(outboundlegid#184, false), first(inboundlegid#185, false), first(agent#181, false), first(countryName#24, false), first(airportName#22, false), first(name#74, false), first(queryoutbounddate#177, false), first(queryinbounddate#178, false), first(price#183, false), first(outdeparture#186, false), first(outarrival#187, false), first(indeparture#196, false), first(inarrival#197, false), first(querydestinationplace#206, false), collect_set(out_is_holiday#352, 0, 0), collect_set(out_is_longweekends#353, 0, 0), collect_set(in_is_holiday#354, 0, 0), collect_set(in_is_longweekends#355, 0, 0)], output=[key#351, querydatetime#207, outboundlegid#320, inboundlegid#322, agent#324, countryName#326, airportName#328, agentName#330, queryoutbounddate#332, queryinbounddate#334, price#336, outdeparture#338, outarrival#340, indeparture#342, inarrival#344, querydestinationplace#346, out_is_holiday#347, out_is_longweekends#348, in_is_holiday#349, in_is_longweekends#350])
+- ObjectHashAggregate(keys=[querydatetime#207, key#351], functions=[partial_first(outboundlegid#184, false), partial_first(inboundlegid#185, false), partial_first(agent#181, false), partial_first(countryName#24, false), partial_first(airportName#22, false), partial_first(name#74, false), partial_first(queryoutbounddate#177, false), partial_first(queryinbounddate#178, false), partial_first(price#183, false), partial_first(outdeparture#186, false), partial_first(outarrival#187, false), partial_first(indeparture#196, false), partial_first(inarrival#197, false), partial_first(querydestinationplace#206, false), partial_collect_set(out_is_holiday#352, 0, 0), partial_collect_set(out_is_longweekends#353, 0, 0), partial_collect_set(in_is_holiday#354, 0, 0), partial_collect_set(in_is_longweekends#355, 0, 0)], output=[querydatetime#207, key#351, first#413, valueSet#414, first#415, valueSet#416, first#417, valueSet#418, first#419, valueSet#420, first#421, valueSet#422, first#423, valueSet#424, first#425, valueSet#426, first#427, valueSet#428, first#429, valueSet#430, first#431, valueSet#432, first#433, valueSet#434, ... 10 more fields])
+- *Project [key#351, querydatetime#207, outboundlegid#184, inboundlegid#185, agent#181, queryoutbounddate#177, queryinbounddate#178, price#183, outdeparture#186, outarrival#187, indeparture#196, inarrival#197, querydestinationplace#206, out_is_holiday#352, out_is_longweekends#353, in_is_holiday#354, in_is_longweekends#355, name#74, countryName#24, airportName#22]
+- *BroadcastHashJoin [querydestinationplace#206], [cast(airportId#38 as int)], Inner, BuildRight
:- *Project [key#351, querydatetime#207, outboundlegid#184, inboundlegid#185, agent#181, queryoutbounddate#177, queryinbounddate#178, price#183, outdeparture#186, outarrival#187, indeparture#196, inarrival#197, querydestinationplace#206, out_is_holiday#352, out_is_longweekends#353, in_is_holiday#354, in_is_longweekends#355, name#74]
: +- *BroadcastHashJoin [agent#181], [id#83], Inner, BuildRight
: :- *Filter (isnotnull(agent#181) && isnotnull(querydestinationplace#206))
: : +- *GlobalLimit 10
: : +- Exchange SinglePartition
: : +- *LocalLimit 10
: : +- *Project [concat(outboundlegid#184, -, inboundlegid#185, -, agent#181) AS key#351, querydatetime#207, outboundlegid#184, inboundlegid#185, agent#181, queryoutbounddate#177, queryinbounddate#178, price#183, outdeparture#186, outarrival#187, indeparture#196, inarrival#197, querydestinationplace#206, CASE WHEN ((type#57 = HOLIDAY) && ((out_date#243 >= start#55) && (out_date#243 <= end#56))) THEN true ELSE false END AS out_is_holiday#352, CASE WHEN ((type#57 = LONG_WEEKENDS) && ((out_date#243 >= start#55) && (out_date#243 <= end#56))) THEN true ELSE false END AS out_is_longweekends#353, CASE WHEN ((type#57 = HOLIDAY) && ((in_date#280 >= start#55) && (in_date#280 <= end#56))) THEN true ELSE false END AS in_is_holiday#354, CASE WHEN ((type#57 = LONG_WEEKENDS) && ((in_date#280 >= start#55) && (in_date#280 <= end#56))) THEN true ELSE false END AS in_is_longweekends#355]
: : +- CartesianProduct
: : :- *Project [QueryOutboundDate#177, QueryInboundDate#178, Agent#181, Price#183, OutboundLegId#184, InboundLegId#185, OutDeparture#186, OutArrival#187, InDeparture#196, InArrival#197, querydestinationplace#206, querydatetime#207, to_date(cast(outdeparture#186 as date)) AS out_date#243, to_date(cast(indeparture#196 as date)) AS in_date#280]
: : : +- Scan ExistingRDD[Id#174,QueryTaskId#175,QueryOriginPlace#176,QueryOutboundDate#177,QueryInboundDate#178,QueryCabinClass#179,QueryCurrency#180,Agent#181,QuoteAgeInMinutes#182,Price#183,OutboundLegId#184,InboundLegId#185,OutDeparture#186,OutArrival#187,OutDuration#188,OutJourneyMode#189,OutStops#190,OutCarriers#191,OutOperatingCarriers#192,NumberOutStops#193,NumberOutCarriers#194,NumberOutOperatingCarriers#195,InDeparture#196,InArrival#197,... 10 more fields]
: : +- Scan ExistingRDD[start#55,end#56,type#57]
: +- BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]))
: +- *Project [cast(id#73L as string) AS id#83, name#74]
: +- *Filter (isnotnull(id#73L) && isnotnull(cast(id#73L as string)))
: +- Scan ExistingRDD[id#73L,name#74,type#75]
+- BroadcastExchange HashedRelationBroadcastMode(List(cast(cast(input[0, string, true] as int) as bigint)))
+- *Project [cast(airportId#18L as string) AS airportId#38, countryName#24, airportName#22]
+- *Filter (isnotnull(airportId#18L) && isnotnull(cast(airportId#18L as string)))
+- Scan ExistingRDD[airportId#18L,cityId#19L,countryId#20L,airportCode#21,airportName#22,cityName#23,countryName#24]

Can I understand what is each type of the plan for? Like whats the difference? And when should I look at which?

And what does some of the steps mean? Eg. Project, Scan, BroadcastExchange, Local limit vs Global limit. What are some common things I should look out for? Eg. in MySQL explain, full table scan may indicate I should use some sort of index.

How should I read the output? Isit top down?

sandesh dahake · Accepted Answer · 2019-02-14 22:01:29Z

A logical plan is a tree that represents both schema and data. These trees are manipulated and optimized by catalyst framework.

There are three types of logical plans: 1 Parsed logical plan. 2 Analyzed logical plan. 3 Optimized logical plan.

Analyzed logical plans go through a series of rules to resolve. Then, the optimized logical plan is produced. The optimized logical plan normally allows Spark to plug in a set of optimization rules. You can plug in your own rules for the optimized logical plan. This optimized logical plan is converted to a physical plan for further execution. These plans lie inside the DataFrame API.

Eg if you are using two filter operations one after another logical plan will have two filter transformations but before actual physical plan is created for final action spark performs some optimization and but the and condition between two filters and create a single filter transformation.

For more details read spark catalyst framework

As far as your question is concern just check physical plan for any debugging as that is the actual plan. Spark executes.

Collectives™ on Stack Overflow

Need help understanding PySpark explain output

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related