Shoeisha Crawling hack

Engineers who develop and operate Web applications and Web services must have a basic knowledge of the mechanism and basic knowledge of Web technologies such as HTML / HTTP and Web servers. In this book, based on the development and operation experience of large-scale web crawlers in actual services, points and techniques that web engineers should keep in mind, such as HTTP, character code, HTML, authentication, Ajax / JSON, etc. that support crawlers. To learn. In addition to how the website works from the crawler's point of view and the actual situation, we will also explain practical and efficient crawling methodologies that are realistic. ○ You can deeply understand the mechanism of the Web such as HTTP and HTML based on actual examples ○ You can know the development and operation know-how of large-scale crawlers ○ The website operator (crawler side) also knows about crawlers Can be obtained