Waa maxay crawler ah? qalab crawler "Yandex" iyo Google

Maalin kasta internetka waxaa jira tiro aad u badan oo ah qalab cusub si loo abuuro website a updated bogagga web jir ah, geliyaan sawiro iyo videos. Haddii aan ka matoorada search qarsoon waa la waayay ee World Wide Web, midkoodna dokumentiyadan. Beddelka sida barnaamijyada joofane waqti kasta la siiyaa aanu jirin. Waa maxay robot search a, sababta aad u baahan tahay iyo sida ay u shaqeeyaan?

Waa maxay robot search a

site Crawler (search engine) - waa barnaamij si toos ah in uu awoodo in uu soo booqdo malaayiinta boggaga web, si deg deg ah u dhex maraan iyada oo loo marayo internet oo aan faragelin shaqaale kasta. Axe joogto ah waa meel scan of World Wide Web, helidda boggaga web cusub oo si joogto ah u soo booqan hore u xaddiday. Magacyada kale ee caarada guurguurtayaal web, guurguurtayaal, Axe.

Maxay u yihiin caarada engine search

Shaqooyinka ugu muhiimsan in oofin caarada search engine - boggaga web-eegid, iyo sidoo kale qoraal, sawir, cod iyo video files yihiin kuwa iyaga on. Axe hubi tixraacyada, goobaha muraayad (koobi) iyo updates. aalado ayaa sidoo kale qabtaan gacanta HTML-code for heerarka waafaqsan Ururka aduunka, kaas oo uu yeesho iyo fulisaa heerarka farsamada for World Wide Web.

Waa maxay tusmaynta, iyo sababta loo baahan yahay

Indexing - waa, in fact, waa hab ka mid ah booqasho bog web gaar ah by matoorada search. Barnaamijka waxaa uu baaritaanku qoraalka on this site, images, videos, links outbound, ka dibna page u muuqataa in natiijooyinka raadinta. Xaaladaha qaarkood, goobta oo aan si toos ah la marsiiyey karo, ka dibna waxa lagu dari karaa si engine search ka gacanta webmaster. Sida caadiga ah, taasi dhacdo in maqnaanshaha of links dibadda si gaar ah (inta badan dhawaan uun ku abuuray Eebe) bog.

Sidee raadin caarada engine

search engine kasta wuxuu leeyahay robot gaar ah la robot raadinta Google ee si weyn u kala duwanaan karaan si waafaqsan hab ay ku shaqaysaa barnaamij la mid ah, "Yandex" ama nidaamyada kale.

Guud ahaan, mabda 'robot shaqeeya waa sida soo socota: barnaamijka "yimaado" goobta iyo links dibadda ka yimid bogga ugu weyn, "akhriyaa" khayraadka Web (oo ay ku jiraan kuwa dusha raadinaya in aanu arki user). Doon waa sida ay u maraan inta u dhaxaysa pages of site a iyo guurto on in ay dadka kale.

Barnaamijku wuxuu dooran doona goobta si index? More inta badan "turaanturoon" caaro bilaabmaa goobaha news ama tusaha waaweyn ee khayraadka iyo aggregators oo leh miisaan weyn tixraaca. Crawler si joogto ah uu baaritaanku bogagga midba mid, on the heerka iyo joogtaynta ee tusmaynta arrimaha soo socda:

• Gudaha: perelinovka (links gudaha u dhexeeya pages of khayraadka isla), size site, code saxda ah, user-saaxiibtinimo iyo wixii la mid ah;
• Dibadda: wadarta miisaanka tixraaca, taas oo horseedaysa in ay goobta.

Waxa ugu horreeya ee ay baaritaan robot search on website kasta oo ay robots.txt. tusmaynta khayraadka dheeraad ah waxaa la sameeyaa oo ku salaysan xogta la helay ay tahay ka dokumentigan. file waxa ku jira tilmaamaha gaarka ah ee "caarada" in ay kordhin karaan fursadaha of page booqasho by matoorada search, iyo, sidaas awgeed, si ay u gaaraan goobta ah oo hit hore ee "Yandex" ama Google.

Barnaamijka analogs guurguurtayaal

Inta badan ereyga "robot search" waxaa khaldin caaqil, user ama wakiilada maamulka, "nooli" ama "Gooryaanka". farqi weyn Hawiyaha oo keliya marka la barbardhigo wakiilada, sharaxyo kale eegtaa noocyada la midka ah ee aalado.

Tusaale ahaan, wakiilada waxay noqon kartaa:

• caqliga, barnaamijka, kuwaas oo ka soo dhaqaaqay goobta goobta, si madax banaan go'aan sidii loo sii wadi; waxay aad caadi u ah oo ku saabsan internetka ma aha;
• Banaan: wakiilada waxay kaa caawin user ee xulashada ah wax soo saarka, search, ama buuxinta foomamka, filtarrada-ka loo yaqaan, kuwaas oo in yar oo la xidhiidha barnaamijyada network,.
• user: barnaamijka ka qayb dhexgalka user la World Wide Web, browser ah (tusaale ahaan, Opera, IE, Google Chrome, Firefox), wargeeyayaal (Viber, telegraam) ama barnaamijyada e-mail (MS Muuqaalka iyo Qualcomm).

"Ants" iyo "Gooryaanka" waa ka badan la mid ah search engine ah "caarada". foomka hore ee network ah iyo si joogto ah ula falgalaan sidan gumeysi qudhaanjada, "Gooryaanka" waa ay awoodaan in ay sii wadaan howlo kale oo la mid ah sida crawler caadiga ah.

Kala duwan ee aalado search engine

Kala saaro noocyo badan oo guurguurtayaal. Iyada oo ku xidhan ujeedada barnaamijka, waxay kala yihiin:

• "Mirror" - Nuqullada Author websites.
• Mobile - focus on versions mobile ee bogagga internetka.
• Quick - hagaajin macluumaad cusub oo si deg deg ah by arkaysid updates ugu dambeeyey.
• Tixraaca - index tixraaca, tirin tirada ay.
• Indexers noocyada kala duwan ee content - barnaamijyo gaar ah qoraalka, audio, video, images.
• "Spyware" - raadinaya boggaga aan weli la soo bandhigay in engine raadinta.
• "Woodpecker" - xilliyo soo booqato goobaha si loo hubiyo in ay ku tacaluqa iyo hufnaan.
• National - Author khayraadka Web ku yaal mid ka mid ah domains dalka (tusaale, .mobi, ama .kz .ua).
• Global - index oo dhan goobaha qaran.

Robots matoorada waaweyn ee search

Waxa kale oo jira qaar ka mid ah caarada search engine. In aragtida, ay ka shaqeynayaan si weyn loo kala duwanaan karaan, laakiin dhab ahaan barnaamijyada ku yihiin ku dhowaad isku mid ah. The kala duwan ugu weyn ee tusmaynta boggaga web aalado laba matoorada waaweyn ee search waa sida soo socota:

• stringency ee imtixaanka. Waxaa la rumeysan yahay in hab ay of crawler "Yandex" qiyaasaha xoogaa adag site for waafaqsanaanta heerarka World Wide Web.
• Ilaalinta daacadnimada goobta. The jirrabo crawler Google goobta oo dhan (oo ay ku jiraan content warbaahinta), "Yandex" ayaa sidoo kale ka arki kartaa content eegista.
• imtixaanka Speed pages cusub. Google ku darayaa khayraadka cusub ee natiijooyinka raadinta ee maalmo yar gudahood, in ay dhacdo "by Yandex" nidaamka qaadan karaa laba toddobaad ama ka badan.
• Inta jeer ee dib-u-tusmaynta. Crawler "Yandex" hubiyo updates laba jeer toddobaadkii, iyo Google - mid ka mid ah 14 maalmood oo kasta.

Internet, dabcan, aan ku koobnayn labada matoorada search. makiinado kale search leeyihiin aalado ay Raacay xuduudaheedu tusmaynta iyaga u gaar ah. Intaa waxaa dheer, waxaa jira dhawr "caarada" kuwaas oo loogu khayraadka ma waaweyn ee search, iyo kooxo gaar ah ama maamulayaasha.

qalad caadi ah

Ku lidi ah aaminaada, "caarada" ha loo farsameeyo macluumaadka. Barnaamijku wuxuu kaliya uu baaritaanku oo qabaneysa boggaga web iyo processing dheeraad ah qaadataa aalado gabi ahaanba ka duwan.

Sidoo kale, dadka isticmaala badan ayaa aaminsan in caarada search engine leeyihiin saamayn taban iyo "waxyeelada leh" Internet. Dhab ahaantii, versions qaar ka mid ah "caarada" si weyn u cuslaynaysaa server ka. Waxa kale oo jira waa arrin aadanaha - webmaster ah, kuwaas oo abuuray barnaamijka, qaladaad sameyn karaan in qaabeynta robot ah. Laakiin Badankoodu mid ah barnaamijyada ka jira waxaa si fiican loogu talagalay iyo xirfad lehna loo maamulo, oo wax dhibaato ah soo baxaya si dhakhso saaro.

Sida loo maamulo tusmaynta ah

aalado Search engine yihiin barnaamijyada iswada, laakiin habka tusmaynta la qayb xakameyn karaa by webmaster ah. Tani aad bay u caawisaa dibadda iyo ayna gudaha ee khayraadka. Intaa waxaa dheer, in aad gacanta ku dari kartaa goob cusub si ay search engine a: khayraadka badan oo ay leeyihiin foom gaar ah oo lagu diiwaan-pages Web.