From 0f3e7761da5904baf6a2e04a8f1dcd83d686aa71 Mon Sep 17 00:00:00 2001 From: Shivani Bhardwaj Date: Mon, 27 Feb 2023 18:47:16 +0530 Subject: [PATCH] doc: add dataset examples --- .../dataset-examples/detect-unique-tlds.png | Bin 0 -> 28362 bytes doc/userguide/rules/datasets.rst | 25 ++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100644 doc/userguide/rules/dataset-examples/detect-unique-tlds.png diff --git a/doc/userguide/rules/dataset-examples/detect-unique-tlds.png b/doc/userguide/rules/dataset-examples/detect-unique-tlds.png new file mode 100644 index 0000000000000000000000000000000000000000..78b862f36e2deb40d7813ef36548d6b38edc4db7 GIT binary patch literal 28362 zcmeFYbyQW|*EW0%KoE>u6hsB2l4tZ1?q5CQd;fXH_?~Zke>}(G7>Bd>UVFt{b6)eB*LtBKCrN-uf`>w(2+)rnDxpy4 z{ZXhhXD?vGFaCGDuAxx4ZXPO+os04^>+AnbcP=L#14&kJEIRKeAk@xp|s$q+ztQVQ)u|`!GbEDgCLE z)<7DG`>r+Iew|SL@T!wJ=46uTPFY>|n%&VSoX+O+uEVCUuIpW& z64z7MJKQEUYJ)Pg->9~#)T%uJ=V zNF8_jjOK@T4qhRWr8}tC&vWTR&}Yx4#$?dQzMmOUdK+=ak4CyfY5nXI z6-%xgaUswB34*`&j&yfBVv6E=o@_k7`A9_(qt9-9R*m-haas3dXMvJ*p{iwmZ5W?w znYn6T7E^F3nm(;BtFU}bB0{IpgW3(Wr~F+U`ws z`GJ}Fg>~QKZ|(msS?V8k88?z|p$YVPqgiUxFpI{aOLAE3>X8$kYQ5N3^i<2H)?;|t zb!6|w(5?RQF<(DTkMtO_jKeS6FG$R^4u~nL5>*=UKRMjT{hA<#N-H(G;Yml$?+~VO zyU2mF^vB{G4TrA}-X*?Xk|q23>`=7#)q&)PXKHm5F(-P{{$e{pKaRqN<#XbOJAJNr zI;RnF;*BV5rCrN{F0DFxZDuuW_)gi1n}?>L4$;S-FE_+2(?Zv) z9#5I1YOm0`E3NQb&+;tM+NjU@d5T=6B;Oj$PmUwO3YT=MiXyw!G;~Kba?Gw(?iEKY zKdG@WDoiD8I>A-Auz~xjW8=Vg87Z-~fr^iY%g@hDym;15I=#soD#fUl$izgDV0D*V zFxk|Bej*bqE4yH9u;+@bR_{}EhhZ1*<4oi0B;1AWCXAQ{9gE8f*Yr2fO4zvG;WCgp z@5CoXXH>{aG(jAzFKH@fBe(PJSH8mJSk}zSiqZRN6_ZtKV#1j2Rn{Dbj=`z0`Ewbx zu6>x#z6H^@o>eDMlt|yCiV)8(*i8NtH28Qcc25*Txc4@qJ1i-`Bye^RlX~BQx;>h% zQ01`S{bt8+(ipq^OQmJB7iBu=KUH;YP#vPfU+VW_`DgUoy?nG}k{3JA%7tOH;q$y? zR_>tKVQG+CVu5?kB9>P^+b1ZwTf2?zPQ(z8a}0xYLe+Ff9RXj6g6Xb})w3S!vA$KI zwBqs%h9&Es2zwq?=)yIuu%-+Pj+=$N12SA@H;r>Ljl$b)%KLpt2rtw$)t%kaAmS@e zk}g;FN@wQUy1mqjtvXpt_(SV5Py1cVp1sP#T9QqRNrg9YiUe0m%ki-u+Qa+Wyq2&}c<^-peT9{`(jr*2TeO zG7NFfv!{#_rI8PDM&%7C=?9(2C|XB#63njcJI}Cp4zo+YH`e-?B*>7|c0FHqBc;hk zhrhM@;`*bYnIfqGjN^jAf*Sd~JHip%$(K7Z;)GI6?@Gt<;(j)NDh-Kzecz=;;gYLF zk=3~;j7GTLHaRR2Ev<7akCN=)d6PJ&%c_|M>=B>GPQ5)Og!4U7N9I7L=>?5$laF&_ zRflNLQL{_?6XHtK>Xl^8?4W(YsvcY1X~B(&pbEOUOeHrOX&dS{KbGlmdMmN5H6P4T)6=n1`?(iR?N&n@V z+HTYTnL&DVP6?ItT>lHJLdL?D&o%l!3cqIWV9HP43pe@|mFPzjaY6$fo|B*Z;WYSAIYo3e zF$oKSpP5$QI^fnw?b_&1eJE3GVU?>;JrrTUxF?C`ZWYU0gU zbW6z7J6HQwF^+dDZkp&nKQFit_>i;MKcvN?_>=Oaw{+pa7!ct z$lqgyeg4`L5w&hl&>#@d^ic9l=_6(Lg0O=5_YL)XM*T}6X1@20=Wo$WavKs56!`F6 z#7z||40%iO^fh}R?#bHHQirga8`b{1<48@7)r=~C}=T~xIwa9_Ei-Grd3bG~xx8?YP4Tm?mzf(O;L}hFO@9TM6YzW_O+7q4iXpQ6)aqyqGRCyPhMPm!Y zvpOReMK7%=MsCVQR2GMBk7f8Je_tgbh8pcq*kNbg`{1vit_@9-OjDRAuS?E2M0Z78 z8g_r8u_&sr&V&ymyiRCex7sK~S%9=F6P%0C?2}0rLwbIZ%=a@I`|SXUUkY2 z_3Jh|N5zHzK#=__whNB19~+Z)@03lp{MuvX7cBy3WvF*QC(hej4)>Cp>zy}UVC{CO zdbi`&+aU6+#f`32fc*KdOU;y40Z(F`uTYuKy}Wv9SUw7?+QzqRIE56$KTRCKV1gwj zY5Ks_{kOk?>uwb7cS#4?VSYSO!^x*q-N%-^x;GrSL$JpJ_R2>%+*G)Vr84|gp5c!w z%AGX~Yq!%}N1IPAOy#L}C^+23weOKNnaII;{Ux?s_t)DKy-r`0aOA<8ASe2bX?f_G zc$W8;k{IEc9Us8?|Yn9CG`N;P9t!&dSQ$el2%WSDPYa2c!Kto@rll6Hd2nM%7k67>FOxG9t^q z{2;hmY6OK7^yyJ@$$AxrIzw`~cayp#<>>CJGLtSsL{TO((rd+3LP6v+Ge zjBSlD#(bw=xs6SXcvwtK7+Ja4xEVQkcv%?@IZTZiIoP?lSlGBtxeVEj|D!3ijiZx+ zjgbk`6g19+fp$zdj99o?c-a_@p)p1d7Gonu0|Rz$MnfKU9%C+EBX$!uqyK0^-T?zv zX<+rAy+WEYhNiep4OrP&4S5+^4B6QkISg107Jk8H|my+rKZUV607) zoeYrKWaZ-K=44^xVBz3mW#?w$`u9z0CJv5ZiAbBQEKKZww~H(b9}EU2)&Q|nXyEh- z7z>}6gNcEYt%HiKt(71J(j{`_$y4(t7x;To9$_5e4tK=j|6%jWCiZ{-_4gLA!kk_r zCqFe?J_Dn_U*c%sV)FNjz;l0JGBP)?F*AYv{SSls=kJ*R%VM!|o0xKPny@o+@^Hd> z^P0dqals~prD8EPW@TkF=JX0%#7^ne~wsy`TsE|0;f0phZ}(B z{{9Wz3wSH$f4QrF%o+6HfAaH>wfLWW1v&ZuzU03(zyCX3|2tj(tr_@l-Tc4S^}o~g z-tUvTXnX*i-# z7oo0j@!A_vNOF{4VBQ@qJ3YHmdbG{iN2RbgG%Ag@KRzVp3fSNXjju<6p==e@IcXaS)K0 z(nEFr?mga)M&~N3Ij@z2kvZ>iUf#Lrj(3k_WMl~8-S8<9ke2%Un?C6UsD2=y*7G7) zPQUoYkzY7_`ic$y_;q=JeYeTz~SvFS3hVIe+?1pXUE3jai{vOc( z{Aib2o!-5-f5GR*q{J`xbg8x6eDqSda%pq>qSAOYJ1&a3>5#Ee!N5a%~!`en4U+y+QfBJQ6Y8EC$nslcl?QEpQj?E4s8s-09Y6BBir6!nMA zId?U$zc1TjLry)vq0a8*VJ5_=03ZDyR_y?q2t`MqG&-faXlkE$Z96%BTq3Uhg(^%#H3V5aph&D$OOnCU)*YmRhNb z%d^ptfoHd5ZWBN=Z}6fn`nw!mON-Aebu>khN0q*dAbIWDokcnMcN16PHAkM*QYM1S zyzHn5EET5NoGFnVT4aalQ^+t~sJ=sbuQHK=8y}@1a;(i&?Q${}QdQR$51Op>4D=OmpV-3ywLU&*N{gnR?EaYj_|3@2>ExO8a zE>ExChY(!E#7$%`{dsLOTAit!EqAjn{N>BuG_jyEOb-n&AD_jaiO(HJ#P8p~-`|@{ zDYqRLN*|`ApeVB*=4z;a{``6N*iLt*^vwKxj&1$Xc3c=E6O$aj%Wsz34Q&bBDN9wP zf*vXtiI`N}+}z@an6)ZoEC-6bf3Gc%)~Fj87z7R?Kkr@p^m-9uR z;pOSna1pVPFJ!0qC_&daU4B3Qcb_}!eSM!jeLC-Za*#rW87jjJRXJv3UnXUcl$0#D z8dP#!8R?4Wv`k*IuRB=b2U2U{XlIsPL%&bB4o8|I>*5KT@_rc=p@ z81r1MHQt$PiIpd~#igRJpX#zQve(s~i>@$$VutmI4b43mF(0dp@>>ypD(bYZchu7hu^-!7O z_U}PuE9G(vRalmmrY71!?~6AWIt#Tbjcsj8;8s*wI!ChWqe$C@Cd%B_vNXUn|D!ERafVEA8dO}g>pO1ci!Dv9T(Op z&Vdz|jvRBJyMykIV>hEhgPn2_u_h0{xOR^Uy|X$#kSY?8%#*PO_RzO8?#q1f;zep2 zn#G^*jXJh=SL+Un=v4EJFYuf}xoIMiG^6XsD@tm#PPNl3D%p4r^v=pCyB9MW8EjBc z(DCln<rzDzQon(~s+O_m>;p4}+xVS?f?9(sZ<{=jr7PcNN z5&!)8b3Ch2VDJq_3v=J8?*wfr!g>TG9100s*04&cxYuZM6w*XVxU7hYSi5CIZZHyF zy_)tdgh;qz^yN+V*KgkBs1<5vwoH)7hu)~Xs@-q_!9PoeyS*J*2GqZaO zpQ#$deVEZPF)<#iwL65+pG@gkp`NM3?P1%JiVC%({jC&_;d0A?#ovRaFl*`6P74(3 z>gurl<iSncuN9NB2K zvo!Qjy&;ewN-#*b)+5(zZ#{4r*(MrVT9)mJye046bv8%SZ>`p!SiqmT+eK%S#Vc(_ zsnDfn9dhMuwVpd=c9Wvl=~NyG2?@!M%t3S4iHXfQy7o$|J$H)0I-QsL3pvU!;aO4ZcZC<*te&NN4d4heR=%LgN)XJJu5ItQ;!@c>VA zq8r(pE_r!ya4;8K8s6p0@@owi9{5JmA@lR|_qO?vx%n1+{rZg?rW{V+hlVsGoes8_ zRpu^x^yVsOa~OTcM|v4q=6m83dYe}XmTuy#M777!)_`h}Ht!uFug3>FtMc#L5T5{E zaAmaS)@;FkG5FFPa77OG_OG(Dv+;( z)p~N@BSw|5AmFgu{IfrnW~EJ%!}O24Xc7UpcNI3Hd2A-l#O0pL9)YR1C`J63E$iEE_YhmFLb_NpMO7_Iv z-Hg~wynLHS|3jflu2OS~aJ_#(Kp*(wQHW9R4;}+B?m9nwjYftAf8kuW)U%twb-d5vnq*(0mO3~_i~fQ*1JSv~zWjX0 zx%R+;6xVTY^(21Repo@<`DK^mLwD6;U7cH8Rth~?vYZyZ_qV_^HHMRMx5jPd==mP0 zf*r?(goFfMx|0X3EP;`GEEQG~-oD+6i|xk{jD#to2@TI-D%pnz8EGoCacm~Re_gvq z(2^yslqULYb2h|3`!-9(9oCCHoYNGW#T zn7!$7*=Ow=QMKb-7&17pAui|+fv%tjD+mrb3dH#;NCvVZ4qc~LKT3mO$Z@CO_ z9~&D(9zlPkL7{Z<5ew1v+g=`Sfj9zAM#<37Fuw94F77KT*xSN=Q==cS$pRA*+wFz%wkH7Ui1QM0ir<*OA&*4EZy6Ob4?>D77darhkW8V^I%5P=ZS z?J<%u@N}v%+|mCGc}PhXz_$!xD8q1?cRm11>4eUz*0?NlSjTc~(q@`WlHC&!&SN=MYIoiM5ej}>t)-R^9Px~H0_k~`-ycK7aG z39(Yrt#MLZa0d!-C&BnV0(UfKR~-zg%$n!N zus+Iw!;DHvNipq6E=(5ml!`3Ut1m)q768%f%1YPx?Q1u0=E541c-){SXp?FgdE6Z^ zLBqwR298a7t) z%8r47fkXe3UnA)p07SSW|GiP8(14?hy?s}aPR&c9k1k97vX!>u{SaeKon(sZ_VcUv-pxSuefu_3l`P-gG2XLjd>&=Ygimk| zHG$=Ruv|GaJ)Lo#UQNKhuYd~e=$+BH-i7I_^Y%pINY`6&sYj38R*w2|6ve=t1APZe z&0<4CL%Xb<7kgi2X08oKWRPB{a+vY>W%L*xfSuF)iZbZSmoHA>N11qxFMwn9-kl7J zBxRr`BO=NM9NIKCrklj)?D4ze#ful~YinHyZUo?h0gqK}H|fu${UPF4qENVzkrCyB zNCyQ9MS37amoNM|MhBZtz+dk<>JJrf;_Y&WvGaJZ!fxNW(^q054SPcYEL91NE%`%e zettgH0_+6lWuA@MwtR_D5)2r?P=$2@IHuUUTT&kF;6vm#aK>hwADr8DUbNC*6O$|&3iBT(WjCXQj-VA_!UMC|v z`I+pDLT$v%1f>3X{%G%*Hy&+);zeS8%)~w_4M@8$?t>x*Pk5M;1S-} z=Lq^?SzK$n%d%4v?g;%`-`e8D^A{BronBon{)<{_K9qaREv*r}KG_|%3WEmhZ{NQ! z7FLc)_4@!Qtae_C5fBtqEipzjaT&ik+}p^9puE1b!!6T(oPmCuc|z8{HO}k(Ha*>W z_s=9@rM;b91eH#$N8daDH&oYYl^#NT>xchkR)cJ{J7b=K2fu#(;_%*g+FBltjlx3d zhany-!((f4AzEFx3oPAfV^+2SrxHBk0s`5=*CKGZfh8C0OltqcWLht#a@i=!_a* z>eu(=NyWv*F&)W*Lv`LYB_=IdNcv(=8Wa0s><6u5^Gd4ToNt0jiXPw@WUhYvcmy$7 z1eVSF7gsc$YGgn_00947Se$$ac6fkbBJPR-emEwqq}8e@K!Wa7N{M-7cft1Tt@U(8 z;MLD}28%=G`PHtgMSz46EM4uk_LfFAJ_8)XVwAiPYi`PQ)PwH`Uo?SLQdZUr*+w?N zNAspg>ix|gdFrDRFH#p57YVY~k+218S2CuoyWT#N2)mt& zcb)D{FK;%tLvx8#aYIJRzzH^;Yywv__zdtXFswp!eRv0b~Vwk((JXWZytBcIID;8)z^1*(n z(l(_zhOvdt;Kvoq{sKi~LT??5E z1g%$R&z?o>@j9cX3cwQucvjbQnX~qIe@SAwkVZ*NOD z64%~G3*LfQA7zp7{P_h94UJr-OevYCA^1@o;p$Q!KYpB^nTh`89Tpard(42OY=E>S zDtUQ$GKPe}KQ#>rH@CIPS`SyqZXH7snpthDr8OM8&VCv7gNK3aGU}6bXPOuey8Eqo zNb=7L!*9WPFb*UOg$y+F=PY2s&iSr%=h@b{O;6=NcEnoR+KGpUhf3D22iw*Sol(2H zyK_Dx&^1P#Y850>hKI}NE!&KJq=gVK3f{qQ?m;->B>Vw$xAgUKwly#cog6wMsWy_N z{+VnbsEnmmmdIBx?!PR2NRFmcFG>Vw5o6Oh2)TbWgU0&%d^??bA00jKt?0Yw;qr*1 z;4FZ^>!hSbXU<^@xLk??TujHr6c&OD49qE;`R6YlUS2RwIgHJC8Ach&(^pnjBr3yi zvP&3!f9Y<}xG+1r{dI~#IZLJ*n+>){A1qH?mTF8vq5INVvCJuX>c#C5%%8qujn}2WSDv*Te^O5iq2CZ$dVu=VEh_|^ zn0<=nA;$F#lp8fF^ z&+~)%VWZGKrmJ}0XhRO~EGiBkVWZtJ{$iqAeHpw@MIO+ovlN%sE%c~_s2B9)Icw2g zKp9gYFQ3RH8Opt6a3un3x+6*Knh@r#-hfsA`Z%}371R$ZBt!coy``z>`h}c4&Sn7cjbNa0T_|K7w*1 z(oS3d+%93{!P#A&AIs^}>_tsQr9g;cVeNkn(`Q^J*^oMUt~%dp!H^JZA+IkeWyxk& z5O3|a0}pRTZjug{rT1X^le%QG)MA>pu1+5v6YI9dODLA>G+V)m5#$=(lDMc6LTPE; z@mE|=MciJk1djLY)9TQEm5yv1lA;tU$hQ|>?aaX~{m@t-X0yOlN_g^H`-c4=wEXooXUNmvs9Y&qO1V&8y!=H&BW3xor}C-tehHeB*Q7E?|iB z`)=IFkfbB+@-)XfR6B0pou&RqizM-R<2yHQ+}O`Cx!$%VoYNuUU5ra9q}qOakQN_h zjEdTucAC{JV;T+aTGG9gc-=Uej9glJrYTBNx$Z>w;ACIgsPSc&-6iLUzqpl4JevXR8(Y;wfv>KvI4m+Z4)Ew&oK6`97W3I!CEcEbyhGS{0OJlu3f_p z&<^SPEYe*4Hx^YoRM`h`l<@_w!A7!N==@MXqPC_~e|3C8!tMgr8Nb+8s=&a)5l6|p zllB&ByxD_Eet@_U>I1;^>)2TJVCy37s<+<0N0|_4a@LoMvwtG_#F^m=TF9!lmimCs zQqCH6wYYR!AZKH)Js0SNJm8^%tr311D9V=pb~%Du91)qBnQz~|g@h}uF*ykG*L+BY zNwx6x#We)>t&G&^2{)2x~c(^PE{7x|-YaHZj^Uu$uwEA8OWW6Y-?P4O&m zi`VBI)9<77nM69rd{>*e$se+N61|Sj+}pz5KMY|;@{+2Wnx)Rq0kjo2S@iuXR)5}= zmX^LF3_bNC6(E4wmg|%^mY?Ddy?4x}!oX(8Q!UNvIs$6a zqh1W>4zTv{ThvanU`cs-Uts7hpa+T@lW~A-N;;fNaT2b7wX=Gw&c)t+f3LV*??Wmn zl7Eg_j_U%{l`3C;!)i2LY5by~pkTA(#Wf^*nygEiZSR9=xPBw%VGM`@@(4%!JY~q02IIBm$d=YI+ z;OyZZ_exLXJrOQA{vc15#Oo*#P`4`z1lGW(C+EuTduYLku&fzWZkYQi(85Xeg8s0x0s>fTPD4_0cmy>pg{Sc(cwYCYd`^I+|_sgJzA5PRU#%6>Qt3 z<6W^xm0C^C^eg7Sf4(OX^7;TtE<(Wr#>)kWL8tikYrs*FPCl#AcY!UO%jjzFg9^^k`t-L?C!cDL8L8?{oXfk0KH;uT4kL><(4L}cOl!DF9q+1P7y*p zpP`Wvr{k;)WMl*HLd(m`rD7Oz5!|{(>baTd;qFfBxt*QhI@W*K&CJ;TXSTKr&5suU5+IQ5h_2{)HlE%+~W_s9Euv3V3*0 z{oxA2VOVgnw6x1Yk1M^5-uhv`kG$eCDU>*91iGse%BjpwYy_%E59y?vv z<&ixVkoaYZWja7Sl-W06hq@UnEoJu=>F^(Mn1yNU=nz!au6@5{F)y@s(&E9%aZ})V&Bu?;zdpYjIH?2tFY|A+KaD{map0j$JL0Lixur=y~S1i>vS4;EL2?(#4k!2qAdsa(y>(xuY9*gn^GYQ}MANtjSt z9)_^9Pgv$zP21u#sikA3BCB1OvZflt7K*(OkeCxM;9deAaC1P<7xU}ui>3;l-Hi@} z`j7}G$z;!#r@zml;JZAW4KD;vAFVibw-oAkEG&QS-sJOHybX}3_4fJl z6ekhJ#uVipvH%X}Wm)%`(!O8E&$e7t%}Num-5=D`r!X72yS%eJw(V2#1$_8O^%yqjI-*5v@EcajD z47_VS^*z*cG3_CS)%=gXxTNs>GK*U$dHSTLd_zx956j-#_t0CR$IbL;0JNI*KD8M* z`ERJ7U4?2UH-1w-gvPvU-24D<9y~eQ;baAQWc==OWfM(SOa@=M<*m9pV&lVs+KdhG z+nCiAdJxwQ)YaA< zu1hzp-hhGE4i_bP2q#`!HPah67W)cu8!4b~&385Y@hpZOBgS|84q5AP`ILdY4ml;% zROD?v>+|fDZz$lS+@9#}8h1%!6RO!gPP7SXTri*TUgo=*wo_k|+P7;~cK?j6pvFa;Cf@`i~ zdLXPwI?x%GP$&%aLrReQ@1$ez+4w@O$Qx+C$PA!7Zh2T!y$B7BuCMnkA(cUUd3hxh zFSaIecd#VpLg5mMHew}Qz61nWu-Got$Kx`^rYbHw&tO*y-{{IJ8ZD8${-iS$&eRN4GLk zm7bGxe{e8!m8BceZ3GsodVMFE@oFuf*nT&lEyc;=x`z7#1T|S$S~mHqT3p`irliq~ z#?0#XEURY9vL3i)vE0v>C$BY{lWSNyYP=jIbuxfd*5NSeb!Igg2tpDKMR%ZhyZ>tZ z8XBs9X(05lJUKqX!!@9_HBPLT)9qUw4#P~FF_fTCaeNmdsVWmMkNu4G`X*;>IomyG z*%kj@=yCq`GmFHV@`Onla?3*{m6ZdqFUo-$6-zC#9AE;RlLJc*EZTDrZ5)svUbxWt z&Y%G*M0PvCHC^@(Ae>EYDmd_od_^f?4zxEOF0P`R+XsMJ%FA)qiX(#_d8E@zA2C`B@1A8jZe+UFhY`K%iw zS#Nj1CksFuP?qb1MDZq;NIqG+*z~VoZ>ym?Fl;;a5wpwJ;T!n07N0Bp{6n-OD>d$= zOEFZ>0t55<8gz~1q23p@BGQ@8*}QYs!g#~(m;f#1##6p|CLp%kmW2|#iy-d+)aEy?7S&iiQSq~GE zkulTL(=WU6e9YT!n4|`b*#r3ols9OBD_4c+p#G`CT1rOd^|NQvQ9c&yTw4NlEiya~P=RydHa$Jqz*(Sg-_uax*Yb z;&pXU;e+gObaa&LfJaw6grrUm&mG)q2GmcN#%etsoSamgoXWUtM$9jSka`E4%%5-T zzdUEbb{#D?3!Hm*@3m~O{SJY$75iVLxv!!*YZY_@eVN2D&sVQ|D-5VI=JP6F$++cs zAt3V0l7RcVK+ksglJ~(|wT|=M6ce3TZ5rMWrh>LqUx|_eB}w5?{Jr^V1w! zfY9Gr;EBy?G4dDzedgvlmq-P2fCnX}u_}OhqslF+L{iG@5JDmQ$T4aR3!Fsi|Liga z398#iJm8RfAO|9#R~DC)w4C_#1l+E7o>+9z_=hy8jcP`!l$y$x%bVE5gTq*eg%Z{N z8>Ei94P^SCKXxGV!K}R%r`vz@WUof4lc)wetl;Uy`v@>#>g-bIjVEz*8Cnx(C6Mlt-s^O?Lx2 z0O}KYA3pGittK=Q6gL+f$hAEQz$w?C6h(J~V94sR((zZ}6XKXN@=X(;1DpZ9gGS&^7V z#C69z!uc*HW&W4%6Fz3`N1_UEAGKJ~=Yy7^&R zX(8dcL|*C_suy0F6pT4bNpq3*d{sQU@B}qM&Gtf=0JMeDOmaXi+Sd8L>VVwbKy(~d zoH$0$cdJhwjM>m!9hMf|4Zw4AZe=tunKb;CZnVZuQsmE{kG;1C&7gP~0osdwpY<0) z3q6w0@UBNv$tH53W0`cMDn?rn#ZWYL}Rf(kRdyS@APBr`%lCQQr1T_gVB5axdV83ho-i`I~ntDYIL${`Ni2wm9{>kWges6fo0+>obwZzHX|K84qS)Pm=(7mVr|%5;=C;blW?H z$6l0xgsb;fUmm8ZJF|)OG*3ysrRTkuX=QC~VrgmlW|J3KqJSWL)}dKoZAkX*ezk&< zBuBj?dBLS!R7?y3m{?d?O_1LsMQ%+l!CGp9li(LGy3EeukbHTeMtfca(glknJBrzf z34QPAFJFINy?T{ocW-WS&#r2cDujqxt$eYsFID7Rl7KtgXo@6=67Z$3hu`E^1F{ia zzPxpdTS@!d_pMi{Yk$Ulv#XE58P4=%yg(8_U?@1NI{<`Vd5e_Yr@n{FR~!Ml->UaH z>fPy)-wR!PX*l&)In2z~`edk(F23W1CsetBr9ptF2qLc-My(jb1Sl8LLzI8=Q7``>+hUzBi5?qu3U&#)xwm|D%Cz+Vt;?BIQq>S z^T)b1E=hb6)>5%_)KHZ`Is+7d`#T*Hk3QoV?_!Ob{&$;)Pyu;4yT^QBMDMV*j8*2qPu&_8a~xOj~( z@8lyoIS-bmO(-0d0a@2uykYolb4`*=aD!7R-2)0d3F=_XEUl#~AZt;%_wG1Do~-ck zdr{!y;<;4E0W44%GLY`xyGX>bIGGm>{E_k){$tj^+#;*;FV9a5>U;0NlD57|@<;jc;)p;DaB^AR zzxTx|AQjdIL=)056+rIXSBP$ojw%NVE%^qsuGOv(h`QYuX|a(b3VGACD>^Ei(DAY%e`nQ(fI`yJqO!nHvQ)u3XT35v$2cNAkPc zc<-l-F@?)L+@JqzejyhkUN9(s-@G?Mi4g&v`{j0Dvd|iHY=N)pT2eI!pO#)yyF3QE`c4+?piHw@uxw^hUayd zRY?eHs8>=31>f>em9p@`z6B7+e6A~rkoeIQQ;4e+>k5`*dL$s0Nep{&>HFOPMgxk$ z%J}i|92i758GT=oqOWlaP+`sT5G@}cU*FtlshNE3R=?J|R>?!0%cO_`_^-eIdd+So zhX@{^mitP6>O1XrFQSe-bl>2x+_thFtyYCMD<=!MLluT{#&s% z+aU~}+5%GLphFl5-OI-t@=5h}A|M-Cr&D@}2uUIRF#)=_Og2A;(RqJ!p;4C`_OvqS zQ9vI2s%S&WgzH52g;HrV?_TId(Nw1hK~I;X<6&D8VV{pT7%n;=dV#}a&_?Idi~gstPND*8?}(rA!#92f?v5>LeFTn9RANN-t>39vFP9o5gwGr_ zu(Wfa1VkF^^elUV$$Gr&S6}jHr|i2m_cie;4`jnVkqc&uK%emxVc zc%iv3-@)0r&$76V4*Fl}89Hh9vkVfA#o=%K*GZ_odFLqy2Tj?VrtlXBm8(2AC!U^h zd#hDy*0S;HDoNZcbS-_yjCmob_=^}&KSV_-e!Xt#iWeoIp1@Y`}> z>w8X2&)kwpOkF$IS(kfyxoDVl^*V<0tipIX^YpmW_i)ajZic8E!#RWX3s3%hRBKOo za5Y)Efr~6VS4TxAQR1jfgQ;p`F{Dn%xJmCE>Qlz)Nth^!3+IhVUnv5GYnhIBV^gI* zoDW1({%_%A$X(DT~95A+NJ}`q2E(=9a}!PQ_5|vKW-DuAQDrq6t8*7L8Tse-C%G@pq=gXG?M;b7oPaFPW=j1AV{jyy)#ssn?I|s zu_RrZQ=!01F{D*py1)Nu=wv~m{&c%dw-g@N@?Qao3CcKvf$aRukCoIg@A_GpbDyGg znUikQp4I(;r7WookNV-i26a{RM*R%KIcOGzdd>@DL_V$9j!&xOs+=RU?4}#u*{&uRD zKrX{6vZpe4p|Swj2gPoxLsGcdq>o@)HQOb&@+k@=pk- zK0xaX!gR>RyEhYIj3CIq=uiIUBhWBVh0{SKRAV3AIiXSn`!7_&#DX|(z zBXa-)Neu7r?KMGx2tjPaNIe0xbvL(TAx4Ih3J#gqAVMQZ`4+p{&Nv>mEfO87LK|`*UpCcA@$ztqd)t80_kg(@KK&OS^*w;wL zYWNhCcR99odlK35DQa*4p$`Nj-@bjjxpgTxBt#j8S7IdPp^b)Wk{aZ#fWzKG`}aXW zJMO)cBoR(p2pQ;2YXmZ3oj*Sh>b4%}XE6{2@R0@~8;8eBn0b1rPs0g?3B%I`5x5#9 z-jHzwf-umCy$03k$jAs%D*&~+2AGFz|28n5r?MYJX&W!(U5y-T@BN)@-p55M#8Kh@`M4bOjh(t5# zXh!xzAY|05)83ft;2tsqI-$QK>+&V<;&{u{^X;xm1P_x`{gdnnvh1wgVbFS zWOg8x{iG5U_$(+$Ix;aav3sU!q}Ed%&SZe1ITBhVI01cGKAg1@u=kXgmq!lObaco^ zA|gm+Z2goLkc4ji4Fde!y3GYuL7F@O`0&<$j08q7QYx5k*) z1(kBa!jJm$yB|xo{{_b57$P;2~YmeCPz(!8m`iw(14{1IkZy)nmVr;NKbrXN6 zHZS<2^R=r=Kr5XMC3F&-jh&r=&jC0o9(r9}U4hrXKuSokajaeErO_Qg`T*o|ffJ2A zJ<4F2U@-akwSkCk6Y#t7!8TwakPYLH2f-1N#ok;A>yNdykR|0H0$)c#cs1?4d-sq- z6u^?I!C96M$dO1C3L_Ct{-zA_R7=o;A)&57$G44*!US60?fJ0AfI}fT+Nb#2(ZI%0La3o3@bv1xPcz13LTS zC+BeTz?4Dk;16oE>BU8h-YhteVDa-^n#cC;rZ=U5uVwc1^xWIq!@x*j z(9eLrR;h9@NpD;judfF_eyfGK-W;0P{}sYKy}Vo~C3Tg+}JkAABoX)lzct&k1WIg;iN*~EsXN}-1S^lXX z;w?IFhE1~sJR;mtfF#a$uW>SKmLZ4LdZ8AFDU%}6&vVnkLA|L?O%{yiCkSyxeJUI* zo#C}q%ksI@diA_98?rrM(Quz2?W5$|ru{*8voPsZ=Z6N}0;EFGaQaAh-$)Mx|qHObA8R9cWGAF~9A+!7_Q;1~9EMv%6N(y18 zWQff3ka;-w?)y6Dzu#GBowLsShqdI|!IJ)WXy|_p=q%q*oZ$-fh5naM0NgiF=OM>q@viLJ3Nf33cWbsJ^)_V zC#=Z3cH5HUMtQy^73I}W@rHD4gy?%!|1`r-1rJxGLB;a`@1gDKl=G&iWy`aD%;FD)*C_I=k^`WI_<7$Xl^BdT))Aq_;Mft)e1F3U4E~ydsGC)x0)Y z(K{H+cB*OZRlqP^X9@vUbL@*;7XI^FuMHhYtpjv}p%laA`nf(QA9S!?zJF@ro@!u5 zq|R-2sS}8#Hs>zOSr>sL0leCShJAhGwcyc zb9~%cQW3f^IcFnO^91)q74dlwqwOIht96-h+3CxBk{cAUy3@NQAW`CLYk?afbdUM< z=>JJa-8Skh2%hGunoy}d&stYFgqJdXHN7!c@5YN7JDt)rSq(o0L4+9)Fu%;584TCl zEIk3b0X%(YFzj%vNN7Wi^m3zaYrHry!jyB>--!H z#wi3jc?@3{;P-$0Z!G#&7*LDC3>3dgOgx+_m!^!U_sy%?+Cd1F{I1~dzP;Q1x1i9n z!ag}M*CIoA`v6U5HU4BlsLDk}jO_QGgO_V+UV$$#2k(NFlWVneEq0nVm*h*P){h9c z3;Tqw2s~Hk;cjw{`75kcF!GJEixJ5OOj;V*;t6ZKW=zVe_{pI@&l>HbysY1-rB@_O9}191?TiNnM5 zkB#?>LWKVNI^Cj34eTENivO1-$L|L4M_ogdTIHWBhli2mPjfw53iRKtX=wOKiSGYt zpNJ+@`qTc({UDA%?f>)k@w8_J1ywXOG@=&nIXLj==+qtR$tV>vMYB&Vgdb$7=xp!iw;bRp5;fK}`` zyZo*AgVN$+!|jpl0|NtO)7m+j{nBWaQZ0<9B<(oY>W8m_9ukWL-Xa$228I_ze|Py~ z8Q2*O3=BA=r5S+>V4==V!g`}=8kLz}n?;Uw`|AkQ$Mq>>U>6SMsL1dSuPaW%S>`f) z@z&RhW8lP@!csT#G;oK{oVZwP#}IZQwym@03>DGe!*nnUgXbF>8RhcKI`0$9?}6Oa zH#9Q32|R)HopE#sHP~%iyHdn6Umbyz$I8($l3zb}41nG?5Nz=&DWnl91Rx)%t*uq8 zil`(8Eze+6paPh9LMkfG**iJ4jf}8?&>OQfnh1b5bYHICdpRsVo(5?KTjK48RkfGj z;;%dpkBA5|5D0zs>LOrMN-$9>CEWMDuK875Kt z9uD>APzOwnOF@Ah;*PMWD01*+ck!X#i&3ixeXbK{k`C};pV|>r8vkVSMAnzuxpJu0 zVO*kNSbH_jS}B8`wEV}r4~+!e6S3Qt3n~pen=%MCWRRq?7pK(yrMS%VlBk4)MA^?m zo%|OCKa@Z_eHFdSG};G|_&DaeV?6MWqdYfgtWPYryqvULes zdR)rAvx4f!^^Z+=d4v>GDvK1#Gv1m;5b1PZ0*$?+U?*cYOK8~Arq*gZv$rNc+F*_i z?=f6i+asv2xy&uT$iw@}|9(()TiN=2anC?s$It;0*ZKX?S}#dr+yQZm=iBQrHJq+V zYn^6Snid`wU8ECX^p==Bb_r%$PCIB~ML{kKp5L>HfXRt~e8k`Ymr#Kz0S!_@6BCnW z$m0MzzqS}3A0Hka-4630tDvBOnukf!ksiQnc8|He{TTpf<*+pDHn#wJC4(DMIi<9; zbborP`XykQu2f0#$;rvJ3U~oGAunq`RJ}+Z^H)HCJiH<>LqF4DqoutMlSMN_)-+3& zg98D3O(I`jNey6G@W2s44%o>f8hAm?OnFw2_Bn6(U{cNXj`{-^fN z1Q7&sXb28M-A5`N=m4A3FfzX5BSe7%$T3Se{fLyboMy!%4f_sLm{r_&Qo$~_W%w>y zFM+*r#>PTADuRk|wQVb+M@m%X^@hOrw_hhZC!-Lo^@B}LPGMSc-iA0Q7`cGsMP=TdPJ|?CpWbmc1QOn3@&p+-I$9(b6S4Y+04kw$C^*%(MG2C-6VI;+BSuu@HK+wdZ>% z$@75{bOCoiwDQcZBrDC_MJGVv{<7; zYSk+ToFB-^G=FgY{uyBin(-qW4 zuoXT9Pshi(mn9{_fF@~9{5sR9?CL5CS$Y~)));WxDZGqq3?V|9-;UeN{5&@& zCki%;GeSad07P=ctB=jjM&#u1gkFT;_-zO&oh!I!icHE0V6LcodP-8U%TvI94f^z( zU~s~0I)kFY)!X|rsO!q?y4hG{yfoR8LAuroYN(wMjvaQQ?51d?fTZ>m*{|YF`-ES} zknu(YpH ziLJ=pLn(G`Z09NO=?}|=R%>00PmJdlxyDeZk5F5!?!nzt!uL1J9;)-S`XAbj&!k+> zD$Emkl2>=Ok~?0V{@ci-L}4D9~Y z6A6bM;A~aVmQmXEX?{V0?%>H& zrvz?)>FZms+qJQ=;X%T$CS@ERxoT)c^x$wO1q1}f!6F}1T?)x1gc^hmWRpNk6JSjs zn0a<_c6Ju}1nnRoTU&#TVrQrJ)vL%CF9dd~Q&Ug%)_dPZDQ@rXu8}G!lW_W6rmnJ& z3dUW!q)(3Ys9htjZc`f{uM^Kd6Ns(QV>;KTCwg-OUD+E@*>5w7Ixi}^z;V=gwPq=K zuNw*VieuAM2~5kfzilHZ!fYc$0SjTeVvcLpe}oBJPN$&UHRZXol}buUiSgO-U+q6+ zyRgCcmHv5*sE7+ zH;#5G5gjSXSUg}!?1Nnsg}|X%R#|BF!+F`y z>X~=Zs0gM0EVpr2<4xDOb}i~(^>5CI1Y)7MfrrHWZvT0YUmMJg(%&0&!&~>AW~_7j z?2o>sp4#bNKS=t~>^;h{_)`~M^gW;2)$xkeh@|~|W$5dmEakhG`-Bm!OwDb9M1|KD zjU5RKRlPVD2{tn`&zccjzrngPCqOnmJ)mNUb&@RW_(5`i&z2cP+qVsHS(k>13*q_5 zCczxu3~U2Ho1lgUS(pK>BO|YM55S6SG*S}=BpVqsbDDz3Cpf`C3^$4(z}z^yvCGLd zs^N8T(jGH9&(^?fd=u1yYwJK)KsgU$$QZ_e%dNSy^M!7DSC=M&1(X6F_fyWL`S}k6 zMFa%ygT18zUKbUJfYJhv*7x=vhwY@El7CDYthlzN-bB307FeWy7Z66^*~#Ob2x3-j z_Ti}3s8KPN1<=4?93x?=s>S(lxZZxvLwCnD8iT~7OL@h@RHu0@zuVO0RTsVxP}&K8 zg&4Mt>HD)^tLdEzR>xyQ5sZK9kKN{vAM}pEaNu05yaUcO8g}+LIJCL=`AGrK&U}5z z2gi3vocyl(YYRA=A?G%dKMXsl0t*S)&mJG`Z>MW&;|sIkWj_MTUrIA&sW-mbWoG5( z5&;_;9vKPflO7nwl{X9)mcY)DO`{Mv^VKmJ|IR%sNqe~IMtx2W+QmgX(4zOd-_>6o z$lRJL4T=J-OK!~Z(XiHOYG0cPmQb|UE|Gn4F(s^)CrAA2bwi7rsi#XAuIrQR$}Sx? z(uVjgMd{MI5Lm$CP?QKj3 zU-@So2qr2jDvrL1rKhL=n3)MrPG-C{(!d1tRB&1v^Eq*GyekM%G%;ZW(ez0E68Muh zAHvZfpBd-~^8tZiQV` zG}dl<@5TDoFTBiCiDbhf`~))>9yz+3=9LSJB8?*p!LJF4)|F1fs>)8vl@IUjH^??F zfFnCu+>Q#03B*8|Axq$%NGK^oOG`yGF&J41r_cVs_Qgz(5py@{G*Pmq>!3WQv^e@U8>qTuRZ`U(wOgKZBAx zlI)1Fuq8>Qkk6>vJ38J36pCPhCd;m<5T0zqqCClKa&mI;!P7^_4-o#Orly_|5jpBh zaPAvrUM9&t>D(~yq3N0)bT&z(wYi66YWSYOtuF(8m)lE%Yu$FZ0?rC6-{ZFSsPgfs zcNp7s?T`GHYf*Mms7>=YI{;R4PIR=4XV~m=L|&CyRm*GrX{*l#FB|ZyXIp-eAzz~Q z#4p9UWA^hi0+n)>Wvs8Y_l_lf%2ZX|A!KCo4oRsCZIw5=-4uLKz+h?feVZ=gQI>8@ zCrgKRtKyIqS>MlZv&@!(bH`?wE2l%1@eu>w!%PEM9fUeJAQZh6*mlqj> z3R(~9>MppsyNiLbO1V*z4WgN7re(F*9IuSH?d$57&otAfG=BKJ}=k7x%cd ze?X^gatb9;w;`jG6ulcWqMh-jFgN$GeP6{FgZ8R4t8#HW%A1tN^9sJ=iSecYe}^goer)TL7vZTV9R>;Z*b~NiqQaEuek6GuDjS5o144C=G*sGn!G1 z0BTLCYiP7Uae=3$rN>h=!!|ZNAS|>7;|JNW7`t6*9qF`dMM3Pfevx|x- z0IRECy?P0D-T1V$4^e66jDHxDKx-)V22MXhPS zdwazjH(2q2L|CKb`SW9jhKBZ1+pY!}3?(QvNI)@E!<~?eC}<8Ska1a2EUe`hL`6IS_=b z+XL8pOG-*wC2INjMMOGazC*Z5%F4=mm+F}vP%rsKMIo?)y1KgPja0aObt%xj7FXN} zl{Tht{=>KB|8-M39*<1n`afsh{vELg+}MWk=jti}S*a6l9A?cE_`@D9p}0tDWmm?( tdlVjicqlCi@9y{Y|JuL*A@MkVgmTE0rhb{^k%uR7HB_}!N|Y_{{|91^U(Wyl literal 0 HcmV?d00001 diff --git a/doc/userguide/rules/datasets.rst b/doc/userguide/rules/datasets.rst index 5e08350b20..591efe1769 100644 --- a/doc/userguide/rules/datasets.rst +++ b/doc/userguide/rules/datasets.rst @@ -96,6 +96,31 @@ hashsize .. note:: 'load' and 'state' or 'save' and 'state' cannot be mixed. +Example rules could look like: + +1. Detect unique User-Agents: + +.. container:: example-rule + + alert http any any -> any any (msg:"LOCAL HTTP new UA"; http.user_agent; dataset:set,http-ua-seen, type string, state http-ua-seen.csv; sid:8000001; rev:1;) + +2. Detect unique TLDs: + +.. container:: example-rule + + alert dns $HOME_NET any -> any any (msg:"LOCAL DNS unique TLD"; dns.query; pcrexform:"\\.([^\\.]+)$"; dataset:set,dns-tld-seen, type string, state dns-tld-seen.csv; sid:8000002; rev:1;) + +Following image is a pictorial representation of how the ``pcrexform`` works +on domain names to find TLDs in the dataset ``dns-tld-seen``: + +.. image:: dataset-examples/detect-unique-tlds.png + +Notice how it is not possible to do certain operations alone with datasets +(example 2 above), but, it is possible to use a combination of other rule +keywords. Keep in mind the cost of additional keywords though e.g. in the +second example rule above, negative performance impact can be expected due +to ``pcrexform``. + datarep ~~~~~~~