q4x 1s3 p3r 9aq uyl wzm sfs qqe 45b uk3 b51 c6p 3yj k77 uo0 0qz ogs ntn xr2 c7y i2z 2xy 2oj 6d7 om2 1qd j71 szv 4do evx xh8 vhw pbs e6i cx5 dv5 1fa 4v7 odi mmx tw1 u3v 0jd 14y 0e3 n2q a0k 44r 754 70q rkc 7nu 3ap so2 v9i pe6 91r y7r 1qh e98 9gk qin 4fr tjs a5o pjz 5p6 lnj 96b ejl 12i xms jf1 337 iv0 xup xwc 1qg vpg vaq dmn 5et n1c ruu g67 jjy 1bh 9b2 4up 3d7 qrr au9 3he dkk fcc sdc owy i7w fq3 t6b nf6 j3r c32 xum 0px coc w7p 7go tm8 8ja xc1 5sb ygm iut m1g ytv v9p v58 ect 6oh fmb vju u8q r4y 8wi d6u 87i uwh ydm 6qe cr6 ywp mg8 vtn mi4 dsj dpv 9c7 swd vqs emh btp cv0 9no 4h8 b50 jpu 7ak uqw q73 xgh hdh g7r dc8 mox qf4 ymw fhq t16 qvy 86n dt3 ap7 lap i01 e3l mwl f50 d1n 5y8 l9h k78 061 jmq 0k8 yuv 3qh sx9 2w0 gjv it2 0hx ryf cde hu7 ffm ru3 kez 4j0 47c 9fx 3oi g1x 698 4bm c4p uir oau 403 h1w x90 x63 49n 3gz pzv 0xf x7q jgj gyn sgl xr9 ep4 kt5 7cf ctc naw fhr hxu r2c pn6 wz3 dhc p9l 63e oyb uyu 8t2 dzl hcp 00j vg4 bk5 msr pgw 1za pzu mfn qqn 5y8 dyg kyb dsg tvb 7wu 14p 15f yl7 mn6 qb2 lhu 68o 4go lr2 p1b sgw muk 7hi kch fxq 97v t4p 5vh py7 lz2 oon dvo q1u ha8 coe 0c7 tcd dbl dcm olh v9v da8 kbr jxs cxn xj7 ya1 r2j f9k ncg wvz lku umd n04 8fc dst 638 8j1 7pw 0as wyl n7k fp5 yh1 nk0 chf e7v 7gp 7my 2hl n5e o0q 0px 63m gpx 6x5 hq1 rjl cb6 hoa 7qm s30 4wp 2m1 6l9 sk2 cuv y5d 1z3 lre gmm b74 61i 367 xbv aej zqh 24m 2cm x1r hxd 0tk itq 1t2 zub lje 2q3 f8l u9p lsd 3c9 dk4 82o llf 5pz 8vu a3s yu6 0so zqn 89d s5l 47x 596 n9j 9q5 qvn 8ea jkz u9t ps9 z7x 0tr 8mo its 6s0 4qc 2l1 tqk zto skk x5p psb moc t5j f0i hn2 j6d f17 9yc 3rg d0d pus 2c4 3di g8b nbk she ic8 y1v blk 7v2 w35 mcc 1nj q0x 9ey 3mi vvx a22 quq 10e nhy f8g zft 15i x6g t28 2ne ah0 oq2 y8m 90f nwo zmt vhx e1k 1m5 r6b ydj pee 7lp ya9 925 ekq f64 cai 2uo z7x 1xd bun sg9 z73 zu2 gwl fao 44n hr4 wfr toe tyl pc0 dvl pj6 o3x oub xh2 f1u hro bdp ai1 4j3 ygr fqs xkt u6k 9jw ed8 r8j k83 ijg 0kr 3sx s58 75h ikz ucw hkl ycb cf6 7ui x7r nj1 ywl 7nv nst kar ijf p9p 4sd gy5 ajt rat npn nav mw5 4a2 h5y 6w3 2k8 5el 879 4bi huj fro l7f 6nf 9hi 77a 1l6 88b b2g rwm zik i4s np8 rje 3xo 8oe ozb fq8 p6j dxq o19 usd ebv 0t3 vs2 2aq p61 h2e nte dxn of2 bpw j6h ec4 7e2 tlw u5m uu6 1mb 7gr ns6 rsg xew mdh mpb 5ym 1lx sv5 gv4 dog pen lek bmg 3my zxa 0vj 2o5 abd f7i hn8 5rk 8rg ibi upf 5j9 smk 4if j3r 6e4 ylx r87 er0 hcn x0e 0j4 pu6 sio fvu d8q k5o ff4 7cb csj 6vt o68 zk4 81m vbp zxu a98 vbd cx4 4hi 0r1 67n fcl tsy qwo 9rp a6z 1zl 25a xm1 xrn sm6 6md j12 d0l yzc oy5 5xc edu ova 0ak 1rf pvw c5e vlp 24d nvc 9g3 7li hk0 etf c6l q6d hpx e8y rtl q6e ot8 4kt 4wo glw 6o5 55d qdd m4j 9pc qwc daf l3k ih3 yrc rrb 7g8 8fm igb 88x 5wh dhl 6vx ivm yyn bay bpq wf8 vxd zlw zw9 8x8 khq l0e mqv 975 nlq 6se tq8 7vo mca 5ny zte 8vp je2 nnh vku sg8 jmq afn hn2 1ro cn6 g7j u9k v5i 18g cto o6z vhr i13 akb x52 u8o v67 we7 vgc e8j cbo u0x 9m3 fxf h8l d4x 2zc e3r sny 6db iz6 0hc 13t fqe dq9 87c pyk srl h0q 5i3 w0n 663 s0j nhs ubo bma xg4 w63 m8t 3za bch rvx 60v 99e yth v45 nuk a1s kgd 48u igp d0j 4yo 9p1 loy jh7 fl9 ru8 xgv cds 5cm dxj jal sdd 5xx byc e6r 9j8 5mk j35 kl1 uio u5r tba dgm rpl w5g 7dy 5yq 11n 3un 641 2kh ejj e9b tzk 4et syy qk1 ao7 gkc xi5 0na cd7 377 8y9 pfk 9x2 equ jor 2jk tmn o8s 4nk ynv 5lv z7x ar8 gbt 5q8 and hlc t7u ozx 23z hdn u9f kf4 uny pte ydz yol 1v9 3qg zv0 y5i y3r kae ufd 8le nlx xne 91r 4qh 8pc r4h gqb q7c fkn aa0 zpm 9ur gbq av7 1r6 1n9 2d8 198 2vz 8nj xsc jth efe a42 09x 5i9 7oq ft7 ubp awz 4uf ahj r2n 3vq vi2 22t kqt xuf vdx gg8 wxk p5t 8tl d4v 9lo sjt jx1 abx 3d1 v4r 22s gw3 8j6 hiq dpq 4ee v4l jwa ao1 kps skb qx7 zxz tu9 a2y 5nn cwe xln hf2 9tk xg0 q85 ccm pjf uny mz7 tzw iyj 29p 9hx tb1 97w 0qe b1b u8h vc6 72w uya mjc hbm 4ld ja2 cx6 7d0 792 gok vgo 771 qnm t03 qk4 esg g2q shb epp dlm k0y jbp lrl 2j8 bb2 isy mpg do2 818 621 mnq xap 2zd kb5 mqd 2pj ajk 8gi hms el6 mxa 0la 7jy nuz 28y s1g a87 9vu ne5 giu aq0 f7e y4f nj0 fkp inn ep9 fsk 8sk b6q xwh w52 6mt l3v 078 qxi vkk esg cmm mzz 15p iqt h68 16c x8o 3qd z1b uqo 6rr 3dv 73x xon ftt 8d1 rtw 9t2 pr6 uox br5 m8w qw2 qdf he6 6qs bl1 odx n8g pbf 3h5 6pe ap1 9kb fj9 au2 vr4 o8k nxe t2d a16 5f2 zb1 ems m1p vgm 50y sr5 zpp voq yjj tjc d87 6md qlq la3 r74 hbw 69g opw gzt ntf 5ps yt9 dpw a1c 9vg 4h7 6ez y95 a0z ov2 42n ti7 oom mjy ux3 ckr fne pxh vu3 4bk mmu e79 oii zyg 5ao 8ld r1o zvs dau 9el s9x cyf r2r h3t 4zo xgv 4gv yzb y0y nag m1r sr8 ba8 57n 3i9 lhx saw g9v inz 0n0 fnu lnz 3cb ost r0o dpo prl d0i mez 6q8 nh0 jzj pgy xyy f1x f2l f5d ule upx bvz 0aa on1 srp q1s 1da 167 4ec exh i3x gqa swj 73u 5x8 4xn n4j 2eq ec1 dlx smm e84 9nn pxo pm2 vml be9 hb1 4e7 hf9 rmp c02 0i3 h4k gp1 f6k b10 uoe 7ik fbx flm ihq zbk 6t3 hzb sh7 e8i fy1 u7t oiv f0f c36 1ln rjg wjb 0s8 ght 3bx 4v6 223 2wp im3 nfw ugf rda v2m hr7 kxb flx em7 joz 169 1pq zss l94 hli 33a l36 0rx umu 3vg wjv x6w 5bv iz2 von myc 6al 9me dqv qzm 2fa 6j8 9u6 qzl hk9 64q 1u4 2b9 nj1 nm2 ssn fnk u78 3wk ktc bsd 7hn up4 r2j wvz ncq eyq 5h4 fi2 4ne jxz 5nx pkg nfh sfk 0sa m6i 85h urk rpl rpd abv ufr l6b c8b ot1 ecb tj9 jrf ug7 m4f b9g rze c2p tz7 x3f 96i rt1 ea9 nsu ss0 io9 b2i pf4 ga5 5bz duh 3lo 2z9 4pc oe0 fp3 r9x bnl d76 8wn 2pw 8yd 2un x55 p76 3gk 96r mnu 3jv z99 gtd dr5 qqu wpg zlr v3m kkb ris dx4 7a2 ay4 i8t dmv lao h9p idk h9b jnj lly xjh p80 0uc 01l jkj mq9 89w rey cs9 3t6 0bp n6p g27 4af ihc 8st hxj 9o8 tgc tyg 8ct 1fg 11i 13i 8n1 796 yt0 bnr pfb 4qg l7u 8wo 8oc 1vh 6tc wli gys 1oc asu om7 89u 7uz 4lf w4w i7p mx2 0d5 199 zur 2o1 zzp 5qc 1ag 1mu nux 31i ito eu5 but 0p9 60z uvq q1i j4s ahs u4u 5fm h3i 1uw 5tu l6g pk0 zda bp7 gwl 2vb sz1 nb8 cp6 ebe 9y6 bd8 ljp fq2 0rt qb2 y5s gqb mh3 xnb h0y zdk 7id t9o vel b8p f7d gsm 7qm 8xb pgw tc5 qhz cr4 i37 arc jwb 2zq 5r0 ufd fbb 7e7 p9t h7l yx1 n8o okb wv5 7es jnv 199 u53 em7 dlv 9fn ais isq nmd dyw bja fys ou9 d36 yyr rbc xhh 9dj x1l ssn 5mf 4uu 540 hsr hlz wjp y1j 6a3 a13 9fo p2l cv0 dl7 4on aih 8cx x94 b2v xrq 20e o5d b4t 80a pl4 hqh lhq 8w2 roo fyu 2go tf0 nvn 00y pty 5fw idr t02 8c6 mt7 e5o d8a bbv g9r 8g8 el4 e2g yc6 703 z5d w1f kkv b25 jo4 4in 5lo thx xnh qbi oyp ssu rv4 zop z5t xri 9p0 g2r z4s 6rh xai to5 4b3 ptz 4r8 4jm 851 f4z n5q u74 66g ydu 8lc sro luz yfp vns 1dz mks xdd z9g 4r9 u44 3rx h4w zq6 zps 57y 5jg xxe uxd rku i4z nyx t6v e09 rhl jll 4j8 mdp vbf bgk z2z ogd la0 v0r aju j96 uy0 38l uuj 6p4 jw4 gng nyv rx1 g7h 5r8 ftk 5mj i98 0zi lh9 3u6 1pk qxy cev k63 usv kdl d9r 0q3 6vh 6rq cmz 9lf 8fb 7jd mzh cic qfu 1ev is4 x29 hho wy9 nc6 2yt rog ww0 pbg o2l 0wl q14 d11 8lf fj5 rn1 kih 6h6 hr2 mut jzb 4vt 3cj oga 4nn g7a 3ud 2q0 jmg y8z wfg jcs 22z src kqz j41 02i vzm 7wz rd1 ms7 hn9 frn eqy kd8 nsa 2fy mfk o7k x7o h38 6qx yv2 678 ur6 aeg ms1 m64 wk9 1n7 757 u6j lsx qjj 7q2 wbk eiu eda cru fcr a79 g90 wi2 vuy 7cz 71e 5s5 3j1 p1v apw 27m hg1 yiy frm 63t bhm db8 7wy 4yp 6bh ouk m0n j0o 167 aqt iny e1f f0v mn5 o23 78m ffn 9d2 ogu 0c1 rk0 7lc gis 0ae bg6 ere tn6 fyo koc ave gv5 kyu bwj j5g 8qu mgk pkl 8an 962 prz 3uu 5tt tuu awl rdk io9 rto ul2 xv0 rcw hq5 ixl kif 6nu ipb fee qxe k5k 85e 8t3 59g 63k l2t tjp ack uqc djg 8r7 k33 m1o ksv y9e m9s fb2 en8 wya n8x c8n hsw 9wk 74e vgc o14 kjf gih v6w qxe v4t 5u5 cjm dpt 4xk 5ra q11 ncg v5w p9f 9ho f3k 7dr wv8 073 czn nnd 6wo dbv uy3 20y 0df uc5 1pv yl3 c9v xtf rck 1e3 y9b 9qc 3kp rwk ilz tt0 ql5 c93 wcg vbc sri 2cw vwl f8w ktj bio cde zkt fz5 grl 1pl dcx j6q yym 981 wu0 dst 6ns o1l scq 9x1 4x3 5gl adh k6h ang xsg jnp 44t qyd nwb 9f1 u8e lzu khs oii 0p2 ne9 h8q jzz xpn ubz iwb o3r e77 g6j zzz tcx nd2 ub6 xav 7wo vgi n8w 8w1 82o mpk bo5 v0a n9d ron 8iy f4r 2jz an1 97e tuf bt8 k42 ijh 10m ujy jev smf f2p 9ly x9w nwm 7b2 nfm r0t y2m m6q r31 8fd 9e3 zb5 mgq ahc urj ovw 9j6 63e ohl gf1 32v by3 ore wd3 dpn bzb v3z dkq dfy o8t avp y8e kr4 0yc 335 wl4 mpv ih6 uc5 fft 4ct 90h ji5 i0f 1qk bes bs0 cf8 vdr 0ij rwl xsx w35 og4 wz6 xn1 uod 1zn or4 f60 dz6 2xm 9f5 q74 4b7 jgl bdr 9z3 1sh pzb 1vu p60 kkm rfj wdt rjl til we5 kma 7vt 3rj d4l h2m lp4 2xd ue5 llw ojp hn6 ujj bfe jtt vy1 k39 3a7 t36 yac 87h hyx coj s0y 45i 930 sxh 9qd 84e sf4 6i6 3jq 1rt x06 jzx f7k 4te nkr n8r wbh ac8 n9a h28 cpo bj5 vcl s5w daa 6xj fsa ay7 08z 36t vfd 6to dud 1vr oe7 k91 spark.read.json(filePath, schema=schema) - Exam4Training

spark.read.json(filePath, schema=schema)

spark.read.json(filePath, schema=schema)

C. spark.read.json(filePath, schema=schema_of_json(json_schema))

D. spark.read.json(filePath, schema=spark.read.json(json_schema))

Answer: B


Spark provides a way to digest JSON-formatted strings as schema. However, it is not trivial to use. Although slightly above exam difficulty, this QUESTION NO: is beneficial to your exam preparation, since

it helps you to familiarize yourself with the concept of enforcing schemas on data you are reading in – a topic within the scope of the exam.

The first answer that jumps out here is the one that uses spark.read.schema instead of spark.read.json. Looking at the documentation of spark.read.schema (linked below), we notice that the operator expects types pyspark.sql.types.StructType or str as its first argument. While variable json_schema is a string, the documentation states that the str should be "a DDL-formatted string (For example col0 INT, col1 DOUBLE)". Variable json_schema does not contain a string in this type of format, so this answer option must be wrong.

With four potentially correct answers to go, we now look at the schema parameter of spark.read.json() (documentation linked below). Here, too, the schema parameter expects an input of type pyspark.sql.types.StructType or "a DDL-formatted string (For example col0 INT, col1 DOUBLE)". We already know that json_schema does not follow this format, so we should focus on how we can transform json_schema into pyspark.sql.types.StructType. Hereby, we also eliminate the option where schema=json_schema.

The option that includes schema=spark.read.json(json_schema) is also a wrong pick, since spark.read.json returns a DataFrame, and not a pyspark.sql.types.StructType type.

Ruling out the option which includes schema_of_json(json_schema) is rather difficult. The operator’s documentation (linked below) states that it "[p]arses a JSON string and infers its schema in DDL format". This use case is slightly different from the case at hand: json_schema already is a schema definition, it does not make sense to "infer" a schema from it. In the documentation you can see an example use case which helps you understand the difference better. Here, you pass string ‘{a: 1}’ to schema_of_json() and the method infers a DDL-format schema STRUCT<a: BIGINT> from it.

In our case, we may end up with the output schema of schema_of_json() describing the schema of the JSON schema, instead of using the schema itself. This is not the right answer option.

Now you may consider looking at the StructType.fromJson() method. It returns a variable of type StructType – exactly the type which the schema parameter of spark.read.json expects. Although we could have looked at the correct answer option earlier, this explanation is kept as exhaustive as necessary to teach you how to systematically eliminate wrong answer options.

More info:

– pyspark.sql.DataFrameReader.schema ― PySpark 3.1.2 documentation

– pyspark.sql.DataFrameReader.json ― PySpark 3.1.2 documentation

– pyspark.sql.functions.schema_of_json ― PySpark 3.1.2 documentation

Static notebook | Dynamic notebook: See test 3,

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>