ICode9

精准搜索请尝试: 精确搜索
首页 > 其他分享> 文章详细

这也敢爬,你离牢饭不远了,爬虫逆向实战案例

2022-03-02 13:30:00  阅读:200  来源: 互联网

标签:逆向 FR CODE NAME QY 牢饭 REGION 爬虫 COLLECT


重要的事情放在最前面,请您一定看仔细:

本文以学习和交流爬虫技术,尤其是JS逆向技术为目的。大家重点学习其中的思路和过程,请不要模仿爬取文中举例的网站,如果要爬取,请爬取不要超过3页,否则后果自负。

Crawl It!

爬虫技术万年第一步——分析网页结构和数据加载方式:

在这里插入图片描述

结果一打开网页是这样的,看来网站维护人员也过节去了。那就假期回来再继续写吧。。

在这里插入图片描述
本文暂停于 2021.2.11 上午 10:25

春节假期一眨眼就过完了,我又回来“搞事情”了!

在这里插入图片描述

  • 打开url,显示数据页面。

在这里插入图片描述

  • 通过Chrome浏览器的Network选项卡可以得知,翻页是由ajax实现的。

在这里插入图片描述

响应数据如下:

"95780ba0943730051dccb5fe3918f9fe1b6f2130681f99d5620c5497aa480f133ef544506ed439f
4bddb1ca63d7c066964c776ea098085c5cd9112e7cbcf533c5823e8b4bb0d542e5e2591fb6a46134
7bbb39b1ed0de45bce0d2512d6a0834f5c1d80d738d51820fb78c1aa2481fd43c825924bc101ad71
917930074d902df9764fe7f29f5a6cb917ea3e617ca76a348186f29d547a8b2bd4044cc9d9036fe2
b22e953f4aeea13705d9d1583300371749d5a882f6c50176739ca1171a50db15aae8c6550591dd8a
e62ad68dc69935e3fcf57379cc625c94acabb2effc4d5505f214f94a2a0e50110d9647d20bb189f4
4acad6184304fa46dff0048b0f58dd1b5d8a945794bbe9e66898b8e363192a79b84cde408f90a02c
9c9b85df2b6d5de973fc00874b223acd2da7df8642ce1e1e64aa6ff550e7b96db782d348f8209d5f
d59b9e26a2aa7ee9c4621cfc5a6d757d91e3862469c39b3b5152054a50667e0f8dfad00fafe55917
e938398c09fc682becfd9aa54f68ce6fd7e023cc7db416727a1572389c798d85f1cd348ff918a4d0
9e67c41a719ac57b6c06a05e954b84a6f0303192b740d3d5ed9c9cc4fe8f0c4de71aa12edc33069d
55f311894109c8c686745fcd22ed2cb33f5deeb6ca9af98c0097e86b450e98a9b1011587011014a3
5b63a3d55c17d92849f56d5e20e76847238c37620d2098b96c153accc9df0df7722868ff86028415
1bf7d7d0aa78ffeb89c882d592994ff09a235180f6f1ff6092fd60ce7eae7c69fc30ba47239f513e
08b9eb3e530cf08647933ea18d2c34c4cb9515b82d4153172311afb9af5d8bdd770d9d9410e75381
cf0f2aa2ca5cb33ccf3276ed0c7dfeda58b7c953bd7225b4395b07e922b75c26c3b5ee302ac3f0ae
40dc31a55ef92afdc7909c7f2e900e1ae8bf92764571c5a5ff27671fbce00c27fc379d64ca61d84f
95558bd2af9b83dd58349922543b8c8e0dc43dead3a102025ce19f4db76f5346b5bbd8784cefa854
51ef22ccd3967f1f6b2eb6f3a9114cc7f26ce107b25985c35ac9a82f11b637fd4a52c2c4f6966f05
3ec48eea3d056268304e58540c3eac82dd70e668e2b24707daeb5d1026b6ad7e7efb638bd51838f7
18c690d0ca4f421246e09432169a0c84ee96adcd4082f5796f474c636ed6d19702e8e5c45421e98a
56d94324160196673363f729ec7cc207f478fbaa865af04ab243cd1a5d3cf54b22393a04aeff51fd
42d33a3d76d11814a540332db0d9f61172e99b3f08f75b081367da26edae0ee7ff7dcab16883ab28
82e48eb95177cdf3d337a1836c949182e7bd5b155aef25c87f399664a367fbe961f1e532295dbce5
0c5a1f7acd891849bae2e6169eb8b9bfbe62e85e740686bc4d3a1ff47cc224507f7be3578751d571
6f006eebdced3c82eabbd500e55b8ff610cef3d947a317f103bf2cbde3fdfc2a0bfdc7e26bdcfee7
9a6967fd66eb01fc84e5c07718747c603e52719d045fdd19ff01f4a00921db26c1761fb744eebf73
262688e1802da1c1a0320196dc721f1a604512589fd23cc01895760881ba4a7af28e5e66b921a14b
85b7f86cd317e691aadf81a8d2b463318058490b19293a35d2fe87ffdecd220cd24fbee8d4073c26
bc61a3a0ad31377db1e660bb5b15c397d15ff820832ffb46f4008ced73bb1133dc957e6e3ec3d829
309addb1aa1e3a431d06b1c8aef6c43ed040f5ca311910bc0d70c34d8aca94d5b7084cfe46fb793d
e87a027f7bfe4d60489d05d3a0e661e9405a768548cdb69bb8a8e52aa0c775684b3e30f18cc059c9
38b451b3376a1736a21df7809791ed8144a893cd831245c64321064f45c67721413018e6bfef3ebe
572cda88723a970ae924cdde1b0a205dbf80b2e5aa55e6baac55b4a30232ee3bad287a40d4916a42
414f6805955ad4cb32bc1c7a4946d77cc9aa4582add2e68976a9a85003acec58f21327c30750e7c1
0615b94a66912b93237e5cc5568a5a53b7214801d3f9903bff3f445e33cc9fda9d19137917abbee4
c08d178095e5956acbb22dcd91639943ace2c24510b7206d92d011f44c055973c7e25c3de3553a5d
ad8457f0b0023e5cfef2f66ed0288b320f37e8313d061b9b75add7c7848ad722e616cad7c7c12359
018c34f723e37afaf21d466654312495a08beb81cb77e2c2bae6d0151fdaa68df623a70618cd6d3c
b66c1475c8eb24829742f10b90bfb9b5794f5eeba7226f7530ba86a69804cf8fbb4c6985c843c0c4
0a3f5bf88bdf55437b7bfcb0bb52f3bae7ba8ee035923b0dcceb144a30f7d2559103d512b4c786c1
710ebba3cedde63e99eee1c5f6ff7991d4b013ba8f2b223bd791baa5e6962a687415b947efb5dea9
07e3dc7d3eb19d9adbc4e9498c5362460ceb945533e3c4c049e93ce9711ca776d54279a01a226998
c5160efd85151618f565c38e9beed091441d2bd1e7677e032d13a25817f051241ca97995188f45cb
be1602636e855f827f35122b6b8cc887f013402b032fc4d1e24939c7cf2c5e686fdb4c4565022ba6
11726a8b73b5c7f8e77502d73b56b58f14239eebe1fcf356b20367e41a7434ce793b738a6d2b7f7d
3a6753a950641bce3fe01428fd777ddadab33fdde61b9fc56c527d89f52ec4cce77fca7ff693ba1d
7e45225eff225fded5e6708e73c20f6fb72b8ce8b080f3f1f055d5b4b624d0b4020a599b9dbebd9e
88c567cd894a94d846022de3298b171556155bec89323dc12a49137f06acb0f3fbad45aa55f94ebc
31458ccb0289dc403540b98e3a36d9b48a975bbb055dccfc57d87a6213f1fc57db70919dab450c3f
21da6daec9d0df7a847fd632a3e9a882b0e14e15565d06c7f437e7e69e9648f1cb9b022d6118d08e
7a0407d15a820b3249bff0ce986c8dc05ca7eca74b6302080a24c898437e1bbc5d380eb93a783710
407fa1d5389c127ab201e2508cdbc63b51a9d5c546d6bcee9f5e0b3a4b92239509884edc05c27aa1
76f64a21f4a7d4bfe997b4a4fb1b95c9f7e1cf1a85c33df25848be1ae83fe4a948095c38d66dbe10
73d8fa6e1bda1fc8a2913d8731d78b1bd6ed8dda5597da4e63d9dadbb8eadb1612198fc88342afd6
dd2442170683a5d55d27dea2475bf57fec14663906dbb6beec6c91a012f909e01626bf9a1a2a452f
c696b52e4edabe499f349cfeff364b28d8bf82aa9a5cfaae80c4b92e22f03c58b3cca43363f1a64e
cfa7c652de4e676b3d18029e0fb606e8ca0e5f2960b16705eeb7d34f48ef3d6f1a6f2e9dd2ce32fb
44af7cfff512488d7bfe6f57f40b5681a18bf2b6a60827b8aa888c81afbcaff7063fc4aeed89b137
5d35f4f7af1ce3cf9c5b3c81c4175651a981f755f422c30713d3e6ca4f3763f95c0f0b1d1f51c32b
769157a7c0a25f050560bb851fbf581a7186d515bd88e100fc91a56b39b86da71e8d028485358e58
31b00d2e653485ee7f404cf8848a681f3fafceb06138e090fc2caf2b03a4978173e1c6e71821ada8
9ceba3302cc16cfaa4b9bf472fa79156772e5ffdab8b44606e258176fa651d13e7388c13ee17d965
4b3e5564c24ccf478015014da5b0dcc8deb213cb0558c185339b22c490a332b4d4f9cf700cf2662d
ec0f815c8418e2573621713717463e4606b5903f66d7bc0581369a1586e84ebd7ba27277aecbd67f
737e35ab87ac78c428588bd7a1a26e4bc2834779d3399e06790bf1cb1ea77f665a449b9e6b3f459d
bd14bbf1534b4f19292f35fbb5942ec505fcf654235d8e1b556843c87b6b868b4e049733e85f551f
78e3587dd6223183f038be2482259638a0650cf6eff6f1bf5383d4774e5b1e5bd20dc839df78ae23
cca22d23d30f2fa1a49307220474a371666908619339f83cc7ae3fcd37a37db640a06970b39db055
a2a8a0d067fe72516b9b9eccc51a69dcd19e961bfe3b131e8130ec2e648d20aafafaceb7f05f03e5
a4e75cd3dadece5d15cdb26c7e6c39a4fcf400490a668e4221f4c0129cfeb46078775392218c65db
37935c172fc5c10f2b0079a195698acd00f736071593604a1e088db94b29319b75f759cc0e6fb3a4
e226a2305dfb7ad0f0f3b9b02b229953e030ac50fe75560f7a98ccfff5a3dd06617349b4fd1ee5ac
db33789b5724e5c3e4441d13a11974b11406bca931d7077b0b2e1e9c4c7a6eb7245fc5779c1cf922
0f64e3bbd4c1fc1cd5000c879700dcb35e5fd77508fa96063de2c7531ae658bf048156b7678b4b41
bf66f00eb6198503671668115bcfcea46ab1a07a7c077d339bb24f65757fe8cfc944cbc1ab7aae39
b19c40619f28d86e58b1f39e52202ff220092ded48323531264e8271129c644ca7e6c4ee03ccbae9
02b990257b1a55daec1b919f33fff441198653032d1026ac1b404de3562c6f851664fd534b04e544
c43dd82ac7ebeaccc7a9a479678c2371665e99815e6a0c60b8b66a21fb0275dd5ee2c26cd69ddff0
b173b40b90e6a602a30c5f0b33e8b933de6f13520af43adf5b87e22ee5af4c307a27b2f788feaefc
668d87af90bf442df137fd26c8e29a6d40c5cb5fbd551752ef0660477013614528661901067c6d64
a95917b9b419180c4d3d3f0a1edb8521d90700ad70a2f4d0717d5bdc24766802ece3be32dc45bfa0
79cd7ff464608926728f9b395f609f53e42c7ba4136abd566d53d0b1819fbef45358df803eadd071
a250c3ca1abd7c2426975dbdfc5dc8369c79e785a4f6c0ed5a7152ec4912df7bf97914abca3f0b4f
616905ed128809a5ead9d9fe57425a16a32fb0ff3e81e409718814eca8226676d3cc8df61b8b0827
c5c54696a5c5c25f697fb842ca316a36a09dfb48b6fc9f5fdc3717affb3477d98259126821e2fedb
292daba634bff50e1e6136dfc83a4441bc8f2ed5350792b6c4668e76f5db7d64b4f40e94bb749471
19881f0077536a67422278bf0cc97614c2f7800ad08fc3e04e5ea3b010a6c16af0a8e16dfb15b780
e6eb284f197ce9d3d370536bb10c1f48762e7842a651875c0d47c530fc9fd5705fad0b0f3a533500
153e810ad751ff15af761011399a162029a77b4fe937edd5b8987ac01e2c89f1642716fe83627124
7f26c983bea9c09ae415a7d367f5a3bfd1d5cf3e3eafd235d0a222869a6d2f3403937a2f6bb0b7eb
6568c743a39ee90bb7b8efd6d1b5e863da2cae9642eedb758d44f2dd749a6327ee586de842042c7c
ae67b09fd95c57aca18b6a25321440c04bdb056ae12b593fc2f96a45f608d0d0a96e15ad6fff2bbb
bf906b02451af38cfce206ec1004de390f3fb724ddafd903dd31d15fabe97d7a540fda85467014d2
0ea9e3a5a92e88c9718ac6ebd35b7047c333a4a8dcf56b66039b2b77fda17f9e6a741801a43410b6
d4db887d713b83a5db7a1dae485312f452d805da5cf8e385c656b10754359ef3031222fcb3fa3f79
de5acd68630ebc075a86b63d100c05f472057401e9bbc70e931550337f6536b69bda901e8503de8a
b9b0216a0edaea8117381f7e0a91e1abd1e4c3f8009ce0eda0802fc896248754b35d85b76b8ec569
917451a3ec2dadc3a2d07b7f5fb0bcc6c5a9fc5e1280310fbf7dc4bf060228ed3c997f35ab16d44c
0c5dceec43281120738882cc22b1ee5172d41613723c8da59fa6a70438868e2aa216e152435ed196
53866ebcc82721b1a524ebe0dde2a60647924e0d9ea4e391e2fc042f46b93658dc73087360b60969
cf50615daf3a7fe37f24691aad9114b8b8b2c010115236100c007d2e0754aff82e380d9147b18b5d
1489bdfb5753d0ad58854212935801080ad49054b9a599ecaafaa29c2194ef5cf98515e60dfdbce4
61f190b7ea166946b7d3b584b0f137ece880feb9431c5d6732bd3302c1147dbd03edf1aedc8c509e
0a6238a74aae81acd9cab9c1c4d8aeed9a0ac8f9a8a32a93217600e5a43c807677caa2954fb1e065
51b74336c8f37c83edd238f3b2b9434949300a6dc5f00f08ecc2fc8b4ba329c2217bc03e8e6e3457
e92d72c824ef97a8e35ebe395bd4e8944074fd64d6f59e672daba63bb7dcf416ec1d47400d186a4f
882721cbc6a6809b92aab8e0e66f6a6f82b35fc254558afadfa857179c245c626672b8a82c99b194
8761c98707206cbe7ecab8ffee47e9a7577504f78a59059b232a8ed9600a9516cb1619a42e471b23
0b4074d1df1acbb07834ece68090ca8f242fb126e688881004e7751c3f47a18c9b62994e4bbe2aff
fdc007f125234cdddc007b138a4eadaab2bc94f4d645e3adf00d490e4e2283d7dfff39bd3b0c5759
4ca2696c549281887f82776d98e1bab01f031d5767a71053122976a6e59fb2a13f07c98b5e45d530
1bb11d75bb83fb0c8c1090cd5089ff73eb07b6ffc775e5da3194702c2bf8b01f139457158475a8a2
dab9657e4321ebf05f52b64c604e6338fe1801c81db8120c553b641e3b3344d700a91d25412c90fc
981abe075450ed73fd9145d5b9626983272a47d60b087d30a76ede332fdaef790e14583d81dfc89b
8206a120badd4db7b11d73bc2ce882fea80d0ebb86d835c786bf81c63c3855d873535ad17ab183cd
4"

请求返回的一堆乱七八糟的字符就是加密过后的数据。(当然了,你服务器给我加密后的数据,你想要展示到浏览器页面上肯定需要本地解密数据,你肯定不会把这一堆乱码展示出来的。

在这里插入图片描述

  • 那么我们就需要去找对应的 js 解密流程。

在这里插入图片描述

通过九牛二虎之力终于找到了解密的地方,这一步是最耗时,最容易秃头的步骤。

在这里插入图片描述

找逆向切入点主要方法有:

  • 关键字搜索,打断点调试
  • 利用XHR设置断点;
  • 查看调用栈。

这个网页我是通过打XHR断点 + 搜索关键字decrtpt(解密)的方式找到了这个加密处。

在这里插入图片描述

来看看调试出来的数据吧~

{"code":200,"data":
{"list":[{"QY_FR_NAME":"缪存旭","QY_REGION":"350200","QY_NAME":"厦门兴海湾工程管理有限公司","QY_REGION_NAME":"福建省-厦门市","QY_ORG_CODE":"913502112601177740","COLLECT_TIME":1477106448000,"RN":31,"QY_ID":"616160676166636361616466606865656369","QY_SRC_TYPE":"0","OLD_CODE":"260117774"},
{"QY_FR_NAME":"陈有良","QY_REGION":"330800","QY_NAME":"浙江洲道工程管理有限公司","QY_REGION_NAME":"浙江省-衢州市","QY_ORG_CODE":"91330802330012039P","COLLECT_TIME":1477106529000,"RN":32,"QY_ID":"616160676166636361616466606865656261","QY_SRC_TYPE":"0","OLD_CODE":"330012039"},
{"QY_FR_NAME":"徐丙水","QY_REGION":"370100","QY_NAME":"山东省正大建设监理有限公司","QY_REGION_NAME":"山东省-济南市","QY_ORG_CODE":"913701022671825366","COLLECT_TIME":1477106425000,"RN":33,"QY_ID":"616160676166636361616466606865656260","QY_SRC_TYPE":"0","OLD_CODE":"267182536"},
{"QY_FR_NAME":"王仲","QY_REGION":"510100","QY_NAME":"凯邦建设管理集团有限公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"91510105725354703M","COLLECT_TIME":1477106469000,"RN":34,"QY_ID":"616160676166636361616466606865656263","QY_SRC_TYPE":"0","OLD_CODE":"725354703"},
{"QY_FR_NAME":"刘一诺","QY_REGION":"371300","QY_NAME":"临沂宏利建设工程监理咨询有限公司","QY_REGION_NAME":"山东省-临沂市","QY_ORG_CODE":"913713005543587173","COLLECT_TIME":1477106454000,"RN":35,"QY_ID":"616160676166636361616466606865656262","QY_SRC_TYPE":"0","OLD_CODE":"554358717"},
{"QY_FR_NAME":"王红娟","QY_REGION":"410100","QY_NAME":"河南中建工程技术有限公司","QY_REGION_NAME":"河南省-郑州市","QY_ORG_CODE":"914101057919311268","COLLECT_TIME":1477106526000,"RN":36,"QY_ID":"616160676166636361616466606865656265","QY_SRC_TYPE":"0","OLD_CODE":"791931126"},
{"QY_FR_NAME":"林锦","QY_REGION":"350100","QY_NAME":"福建省建信工程管理集团有限公司","QY_REGION_NAME":"福建省-福州市","QY_ORG_CODE":"913501001544135960","COLLECT_TIME":1477106407000,"RN":37,"QY_ID":"616160676166636361616466606865656264","QY_SRC_TYPE":"0","OLD_CODE":"154413596"},
{"QY_FR_NAME":"杨高保","QY_REGION":"421200","QY_NAME":"嘉鱼县嘉泰建设监理有限责任公司","QY_REGION_NAME":"湖北省-咸宁市","QY_ORG_CODE":"91421221737145124M","COLLECT_TIME":1477106486000,"RN":38,"QY_ID":"616160676166636361616466606865656267","QY_SRC_TYPE":"0","OLD_CODE":"737145124"},
{"QY_FR_NAME":"蒋亚斌","QY_REGION":"410800","QY_NAME":"河南协同工程监理有限公司","QY_REGION_NAME":"河南省-焦作市","QY_ORG_CODE":"91410800555731456W","COLLECT_TIME":1477106405000,"RN":39,"QY_ID":"616160676166636361616466606865656266","QY_SRC_TYPE":"0","OLD_CODE":"555731456"},
{"QY_FR_NAME":"黄向军","QY_REGION":"450100","QY_NAME":"广西南宁建科工程监理有限责任公司","QY_REGION_NAME":"广西壮族自治区-南宁市","QY_ORG_CODE":"914501031983159639","COLLECT_TIME":1477106406000,"RN":40,"QY_ID":"616160676166636361616466606865656269","QY_SRC_TYPE":"0","OLD_CODE":"198315963"},
{"QY_FR_NAME":"廖渊","QY_REGION":"510100","QY_NAME":"成都海龙工程项目管理有限公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"91510100201986152W","COLLECT_TIME":1477106437000,"RN":41,"QY_ID":"616160676166636361616466606865656268","QY_SRC_TYPE":"0","OLD_CODE":"201986152"},
{"QY_FR_NAME":"邹赟","QY_REGION":"360500","QY_NAME":"新余市永固工程监理有限责任公司","QY_REGION_NAME":"江西省-新余市","QY_ORG_CODE":"91360500723935352F","COLLECT_TIME":1477109337000,"RN":42,"QY_ID":"616160676166636361616466606865656561","QY_SRC_TYPE":"0","OLD_CODE":"723935352"},
{"QY_FR_NAME":"王文权","QY_REGION":"371300","QY_NAME":"山东政航工程咨询有限公司","QY_REGION_NAME":"山东省-临沂市","QY_ORG_CODE":"91371302092199458A","COLLECT_TIME":1477106426000,"RN":43,"QY_ID":"616160676166636361616466606865656560","QY_SRC_TYPE":"0","OLD_CODE":"092199458"},
{"QY_FR_NAME":"张洪建","QY_REGION":"530100","QY_NAME":"云南金吉安建设咨询监理有限公司","QY_REGION_NAME":"云南省-昆明市","QY_ORG_CODE":"91530000719414685K","COLLECT_TIME":1479833531000,"RN":44,"QY_ID":"616160676166636361616466606865656567","QY_SRC_TYPE":"0","OLD_CODE":"719414685"},
{"QY_FR_NAME":"张志明","QY_REGION":"610700","QY_NAME":"汉中志诚建设工程监理有限责任公司","QY_REGION_NAME":"陕西省-汉中市","QY_ORG_CODE":"91610725667992132H","COLLECT_TIME":1477106417000,"RN":45,"QY_ID":"616160676166636361616466606865656568","QY_SRC_TYPE":"0","OLD_CODE":"667992132"}],
"pageNum":2,"pageSize":15,"total":450},"message":"","success":true}

从上面的图片,我们可以知道解密流程是由一个h函数实现的,h 函数把传进来的参数t(就是那一大串加密的字符),通过AES解密后返回了一个json格式的字符串,这就是我们想要的数据了。

下面我们开始用 python 去模拟这一流程:

在这里插入图片描述

OK,我们可以通过 requests 获得加密后的数据了,下面的步骤就是将字符解密。

在这里插入图片描述

这是逆向过程中第二费时的步骤,抠网页中的 js 代码然后用pyexecjs模块去执行获得解密数据; 或者将网页中的 js 代码用
python 进行重写(就是把 js 代码逻辑改成 python 的)。

在这里插入图片描述

据我所学知识看来,这是个AES加密,模式是CBC,填充模式是Pkcs7,只要再知道偏移量iv和密钥key是多少,这个解密流程就能被完全突破了,按照函数的位置参数来看,f对应的就是秘钥key

在这里插入图片描述

得来全不费工夫~~~就在上面挨着,拿到密钥和vi之后,就可以使用Python中的Crypto模块进行AES解密。详细代码如下:

import requests
from Crypto.Cipher import AES
from binascii import b2a_hex, a2b_hex


# 向url发起请求,参数page_num为请求的页码数
def request(page_num):
  headers = {
    'Connection': 'keep-alive',
    'Pragma': 'no-cache',
    'Cache-Control': 'no-cache',
    'Accept': 'application/json, text/plain, */*',
    'timeout': '30000',
    'accessToken': '',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36',
    'Referer': 'http://jzsc.mohurd.gov.cn/data/company',
    'Accept-Language': 'zh-CN,zh;q=0.9',
  }

  params = (
    ('pg', page_num),
    ('pgsz', '15'),
    ('total', '450'),
  )
  response = requests.get('http://jzsc.mohurd.xxx.cn/api/webApi/dataservice/query/comp/list', headers=headers, params=params, verify=False)
  return response.text


# 对请求回来的响应数据进行解密,参数text为响应数据
def decrypt(text):
  key = 'jo8j9wGw%6HbxfFn'.encode('utf-8')  //密钥
  iv = '0123456789ABCDEF'.encode('utf-8')  //加密偏移量
  mode = AES.MODE_CBC      //加密模式
  cryptos = AES.new(key, mode, iv)
  plain_text = cryptos.decrypt(a2b_hex(text))
  return bytes.decode(plain_text).rstrip('')

# 获取数据,我们观察下打印的数据是否为解密的数据就可以了
def get_data():
  for i in range(3):
    print(decrypt(request(str(i))))

if __name__ == '__main__':
  get_data()

执行代码后,我们得到网页上的数据了:

{"code":200,"data":{"list":[{"QY_FR_NAME":"廖露云","QY_REGION":"360500","QY_NAME":"江西天工水陆工程设计有限公司","QY_REGION_NAME":"江西省-新余市","QY_ORG_CODE":"91360500079012847X","COLLECT_TIME":1477106416000,"RN":1,"QY_ID":"616160676166636361616466606865626860","QY_SRC_TYPE":"0","OLD_CODE":"079012847"},{"QY_FR_NAME":"单俊","QY_REGION":"360500","QY_NAME":"江西中余建筑设计研究院有限公司","QY_REGION_NAME":"江西省-新余市","QY_ORG_CODE":"91360500159864134W","COLLECT_TIME":1477106400000,"RN":2,"QY_ID":"616160676166636361616466606865626863","QY_SRC_TYPE":"0","OLD_CODE":"159864134"},{"QY_FR_NAME":"林茂清","QY_REGION":"420100","QY_NAME":"武汉茂业建设工程有限公司","QY_REGION_NAME":"湖北省-武汉市","QY_ORG_CODE":"91420106074476087K","COLLECT_TIME":1477106413000,"RN":3,"QY_ID":"616160676166636361616466606865626862","QY_SRC_TYPE":"0","OLD_CODE":"074476087"},{"QY_FR_NAME":"涂明","QY_REGION":"360100","QY_NAME":"南昌市水利规划设计院","QY_REGION_NAME":"江西省-南昌市","QY_ORG_CODE":"91360100MA35J54Y86","COLLECT_TIME":1477106440000,"RN":4,"QY_ID":"616160676166636361616466606865626865","QY_SRC_TYPE":"0","OLD_CODE":"491102609"},{"QY_FR_NAME":"郭霖江","QY_REGION":"410100","QY_NAME":"博维筑成设计有限公司","QY_REGION_NAME":"河南省-郑州市","QY_ORG_CODE":"914101025651418126","COLLECT_TIME":1477106411000,"RN":5,"QY_ID":"616160676166636361616466606865626864","QY_SRC_TYPE":"0","OLD_CODE":"565141812"},{"QY_FR_NAME":"顾亚红","QY_REGION":"440100","QY_NAME":"广东铂亚信息技术有限公司","QY_REGION_NAME":"广东省-广州市","QY_ORG_CODE":"914401017163873948","COLLECT_TIME":1477106545000,"RN":6,"QY_ID":"616160676166636361616466606865626867","QY_SRC_TYPE":"0","OLD_CODE":"716387394"},{"QY_FR_NAME":"洪锡场","QY_REGION":"440300","QY_NAME":"深圳大晟建设集团有限公司","QY_REGION_NAME":"广东省-深圳市","QY_ORG_CODE":"91440300564201676L","COLLECT_TIME":1477106522000,"RN":7,"QY_ID":"616160676166636361616466606865626869","QY_SRC_TYPE":"0","OLD_CODE":"564201676"},{"QY_FR_NAME":"郑莉","QY_REGION":"440300","QY_NAME":"深圳市祺盛装饰设计工程有限公司","QY_REGION_NAME":"广东省-深圳市","QY_ORG_CODE":"91440300087044558G","COLLECT_TIME":1477106425000,"RN":8,"QY_ID":"616160676166636361616466606865626868","QY_SRC_TYPE":"0","OLD_CODE":"087044558"},{"QY_FR_NAME":"李胜","QY_REGION":"440100","QY_NAME":"广东倍思控制技术有限公司","QY_REGION_NAME":"广东省-广州市","QY_ORG_CODE":"91440000735024820F","COLLECT_TIME":1477106400000,"RN":9,"QY_ID":"616160676166636361616466606865656160","QY_SRC_TYPE":"0","OLD_CODE":"735024820"},{"QY_FR_NAME":"张晓聪","QY_REGION":"440600","QY_NAME":"广东天元建筑设计有限公司","QY_REGION_NAME":"广东省-佛山市","QY_ORG_CODE":"91440606725095377Y","COLLECT_TIME":1477106406000,"RN":10,"QY_ID":"616160676166636361616466606865656163","QY_SRC_TYPE":"0","OLD_CODE":"725095377"},{"QY_FR_NAME":"陈玢","QY_REGION":"440100","QY_NAME":"深圳市致道景观设计有限公司","QY_REGION_NAME":"广东省-广州市","QY_ORG_CODE":"91440300758618904A","COLLECT_TIME":1477106842000,"RN":11,"QY_ID":"616160676166636361616466606865656162","QY_SRC_TYPE":"0","OLD_CODE":"758618904"},{"QY_FR_NAME":"黄荣辉","QY_REGION":"440500","QY_NAME":"汕头市振侨装修工程总公司","QY_REGION_NAME":"广东省-汕头市","QY_ORG_CODE":"914405001927423079","COLLECT_TIME":1477106439000,"RN":12,"QY_ID":"616160676166636361616466606865656165","QY_SRC_TYPE":"0","OLD_CODE":"192742307"},{"QY_FR_NAME":"李健","QY_REGION":"440300","QY_NAME":"深圳市森磊镒铭设计顾
问有限公司","QY_REGION_NAME":"广东省-深圳市","QY_ORG_CODE":"914403007663577081","COLLECT_TIME":1477106424000,"RN":13,"QY_ID":"616160676166636361616466606865656164","QY_SRC_TYPE":"0","OLD_CODE":"766357708"},{"QY_FR_NAME":"张玉梅","QY_REGION":"450100","QY_NAME":"广西西普工程设计有限公司","QY_REGION_NAME":"广西壮族自治区-南宁市","QY_ORG_CODE":"669733898","COLLECT_TIME":1477106445000,"RN":14,"QY_ID":"616160676166636361616466606865656167","QY_SRC_TYPE":"0","OLD_CODE":"669733898"},{"QY_FR_NAME":"彭伟","QY_REGION":"440100","QY_NAME":"广州市自来水工程有限公司","QY_REGION_NAME":"广东省-广州市","QY_ORG_CODE":"91440101190437587Y","COLLECT_TIME":1477106420000,"RN":15,"QY_ID":"616160676166636361616466606865656061","QY_SRC_TYPE":"0","OLD_CODE":"190437587"}],"pageNum":0,"pageSize":15,"total":513826},"message":"","success":true}
{"code":200,"data":{"list":[{"QY_FR_NAME":"吴凡","QY_REGION":"510100","QY_NAME":"四季精彩设计集团有限公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"915100000858139239","COLLECT_TIME":1477106410000,"RN":16,"QY_ID":"616160676166636361616466606865656060","QY_SRC_TYPE":"0","OLD_CODE":"085813923"},{"QY_FR_NAME":"王红武","QY_REGION":"510100","QY_NAME":"成都城市天地工程设计咨询有限责任公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"91510105762288418C","COLLECT_TIME":1477106422000,"RN":17,"QY_ID":"616160676166636361616466606865656063","QY_SRC_TYPE":"0","OLD_CODE":"762288418"},{"QY_FR_NAME":"袁亮","QY_REGION":"610100","QY_NAME":"陕西首创天成工程技术有限公司","QY_REGION_NAME":"陕西省-西安市","QY_ORG_CODE":"91610000684783500F","COLLECT_TIME":1477106808000,"RN":18,"QY_ID":"616160676166636361616466606865656062","QY_SRC_TYPE":"0","OLD_CODE":"684783500"},{"QY_FR_NAME":"白晋菡","QY_REGION":"510100","QY_NAME":"四川泰兴装饰工程有限责任公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"915100002018730434","COLLECT_TIME":1477106445000,"RN":19,"QY_ID":"616160676166636361616466606865656065","QY_SRC_TYPE":"0","OLD_CODE":"201873043"},{"QY_FR_NAME":"黄腊梅","QY_REGION":"510100","QY_NAME":"成都三元建筑勘察设计有限公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"9151018479216087X2","COLLECT_TIME":1477106448000,"RN":20,"QY_ID":"616160676166636361616466606865656064","QY_SRC_TYPE":"0","OLD_CODE":"202625084"},{"QY_FR_NAME":"段黎","QY_REGION":"510100","QY_NAME":"四川瑞实忠正建筑工程有限公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"91510115592094097F","COLLECT_TIME":1477106696000,"RN":21,"QY_ID":"616160676166636361616466606865656067","QY_SRC_TYPE":"0","OLD_CODE":"592094097"},{"QY_FR_NAME":"娄拥军","QY_REGION":"110000","QY_NAME":"北京华清技科工程管理有限公司","QY_REGION_NAME":"北京市","QY_ORG_CODE":"9111010872146477XF","COLLECT_TIME":1477106408000,"RN":22,"QY_ID":"616160676166636361616466606865656066","QY_SRC_TYPE":"0","OLD_CODE":"72146477X"},{"QY_FR_NAME":"陈积刚","QY_REGION":"210100","QY_NAME":"沈阳华盛工程建设监理有限公司","QY_REGION_NAME":"辽宁省-沈阳市
","QY_ORG_CODE":"91210103242650807A","COLLECT_TIME":1477106423000,"RN":23,"QY_ID":"616160676166636361616466606865656069","QY_SRC_TYPE":"0","OLD_CODE":"242650807"},{"QY_FR_NAME":"刘海富","QY_REGION":"230600","QY_NAME":"大庆市大广工程建设监理有限责任公司","QY_REGION_NAME":"黑龙江省-大庆市","QY_ORG_CODE":"912306047253128885","COLLECT_TIME":1477106451000,"RN":24,"QY_ID":"616160676166636361616466606865656361","QY_SRC_TYPE":"0","OLD_CODE":"725312888"},{"QY_FR_NAME":"李怀斌","QY_REGION":"211400","QY_NAME":"葫芦岛市海威工程
建设监理有限公司","QY_REGION_NAME":"辽宁省-葫芦岛市","QY_ORG_CODE":"912114007164286430","COLLECT_TIME":1477106432000,"RN":25,"QY_ID":"616160676166636361616466606865656360","QY_SRC_TYPE":"0","OLD_CODE":"716428643"},{"QY_FR_NAME":"王凤","QY_REGION":"120000","QY_NAME":"天津港油工程建设监理有限责任公司","QY_REGION_NAME":"天津市","QY_ORG_CODE":"9112011674668236XP","COLLECT_TIME":1477107655000,"RN":26,"QY_ID":"616160676166636361616466606865656363","QY_SRC_TYPE":"0","OLD_CODE":"74668236X"},{"QY_FR_NAME":"卢敏杰","QY_REGION":"310000","QY_NAME":"上海申邑工程咨询有限公司","QY_REGION_NAME":"上海市","QY_ORG_CODE":"913101181345318787","COLLECT_TIME":1477106405000,"RN":27,"QY_ID":"616160676166636361616466606865656362","QY_SRC_TYPE":"0","OLD_CODE":"134531878"},{"QY_FR_NAME":"高双选","QY_REGION":"130100","QY_NAME":"石家庄中天工程建设监理有限公司","QY_REGION_NAME":"河北省-石家庄市","QY_ORG_CODE":"9113010060118700X3","COLLECT_TIME":1477106403000,"RN":28,"QY_ID":"616160676166636361616466606865656365","QY_SRC_TYPE":"0","OLD_CODE":"60118700X"},{"QY_FR_NAME":"范广镒","QY_REGION":"231000","QY_NAME":"牡丹江市隆华工程建设监理有限公司","QY_REGION_NAME":"黑龙江省-牡丹江市","QY_ORG_CODE":"912310007026319900","COLLECT_TIME":1477106420000,"RN":29,"QY_ID":"616160676166636361616466606865656367","QY_SRC_TYPE":"0","OLD_CODE":"702631990"},{"QY_FR_NAME":"王国华","QY_REGION":"320100","QY_NAME":"江苏交通工程咨询监理有限公司","QY_REGION_NAME":"江苏省-南京市","QY_ORG_CODE":"913200001347697633","COLLECT_TIME":1477106402000,"RN":30,"QY_ID":"616160676166636361616466606865656366","QY_SRC_TYPE":"0","OLD_CODE":"134769763"}],"pageNum":1,"pageSize":15,"total":450},"message":"","success":true}

{"code":200,"data":{"list":[{"QY_FR_NAME":"缪存旭","QY_REGION":"350200","QY_NAME":"厦门兴海湾工程管理有限公司","QY_REGION_NAME":"福建省-厦门市","QY_ORG_CODE":"913502112601177740","COLLECT_TIME":1477106448000,"RN":31,"QY_ID":"616160676166636361616466606865656369","QY_SRC_TYPE":"0","OLD_CODE":"260117774"},{"QY_FR_NAME":"陈有良","QY_REGION":"330800","QY_NAME":"浙江洲道工程管理有限公司","QY_REGION_NAME":"浙江省-衢州市","QY_ORG_CODE":"91330802330012039P","COLLECT_TIME":1477106529000,"RN":32,"QY_ID":"616160676166636361616466606865656261","QY_SRC_TYPE":"0","OLD_CODE":"330012039"},{"QY_FR_NAME":"徐丙水","QY_REGION":"370100","QY_NAME":"山东省正大建设监理有限公司","QY_REGION_NAME":"山东省-济南市","QY_ORG_CODE":"913701022671825366","COLLECT_TIME":1477106425000,"RN":33,"QY_ID":"616160676166636361616466606865656260","QY_SRC_TYPE":"0","OLD_CODE":"267182536"},{"QY_FR_NAME":"王仲","QY_REGION":"510100","QY_NAME":"凯邦建设管理集团有限公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"91510105725354703M","COLLECT_TIME":1477106469000,"RN":34,"QY_ID":"616160676166636361616466606865656263","QY_SRC_TYPE":"0","OLD_CODE":"725354703"},{"QY_FR_NAME":"刘一诺","QY_REGION":"371300","QY_NAME":"临沂宏利建设工程监理咨询有限公司","QY_REGION_NAME":"山东省-临沂市","QY_ORG_CODE":"913713005543587173","COLLECT_TIME":1477106454000,"RN":35,"QY_ID":"616160676166636361616466606865656262","QY_SRC_TYPE":"0","OLD_CODE":"554358717"},{"QY_FR_NAME":"王红娟","QY_REGION":"410100","QY_NAME":"河南中建工程技术有限公司","QY_REGION_NAME":"河南省-郑州市","QY_ORG_CODE":"914101057919311268","COLLECT_TIME":1477106526000,"RN":36,"QY_ID":"616160676166636361616466606865656265","QY_SRC_TYPE":"0","OLD_CODE":"791931126"},{"QY_FR_NAME":"林锦","QY_REGION":"350100","QY_NAME":"福建省建信工程管理集团有限公司","QY_REGION_NAME":"福建省-福州市","QY_ORG_CODE":"913501001544135960","COLLECT_TIME":1477106407000,"RN":37,"QY_ID":"616160676166636361616466606865656264","QY_SRC_TYPE":"0","OLD_CODE":"154413596"},{"QY_FR_NAME":"杨高保","QY_REGION":"421200","QY_NAME":"嘉鱼县嘉泰建设监理有限责任公司","QY_REGION_NAME":"湖北省-咸宁市","QY_ORG_CODE":"91421221737145124M","COLLECT_TIME":1477106486000,"RN":38,"QY_ID":"616160676166636361616466606865656267","QY_SRC_TYPE":"0","OLD_CODE":"737145124"},{"QY_FR_NAME":"蒋亚斌","QY_REGION":"410800","QY_NAME":"河南协同工程监理有限公司","QY_REGION_NAME":"河南省-焦作市","QY_ORG_CODE":"91410800555731456W","COLLECT_TIME":1477106405000,"RN":39,"QY_ID":"616160676166636361616466606865656266","QY_SRC_TYPE":"0","OLD_CODE":"555731456"},{"QY_FR_NAME":"黄向军","QY_REGION":"450100","QY_NAME":"广西南宁建科工程监理有限责任公司
","QY_REGION_NAME":"广西壮族自治区-南宁市","QY_ORG_CODE":"914501031983159639","COLLECT_TIME":1477106406000,"RN":40,"QY_ID":"616160676166636361616466606865656269","QY_SRC_TYPE":"0","OLD_CODE":"198315963"},{"QY_FR_NAME":"廖渊","QY_REGION":"510100","QY_NAME":"成都
海龙工程项目管理有限公司","QY_REGION_NAME":"四川省-成都市","QY_ORG_CODE":"91510100201986152W","COLLECT_TIME":1477106437000,"RN":41,"QY_ID":"616160676166636361616466606865656268","QY_SRC_TYPE":"0","OLD_CODE":"201986152"},{"QY_FR_NAME":"邹赟","QY_REGION":"360500","QY_NAME":"新余市永固工程监理有限责任公司","QY_REGION_NAME":"江西省-新余市","QY_ORG_CODE":"91360500723935352F","COLLECT_TIME":1477109337000,"RN":42,"QY_ID":"616160676166636361616466606865656561","QY_SRC_TYPE":"0","OLD_CODE":"723935352"},{"QY_FR_NAME":"王文权","QY_REGION":"371300","QY_NAME":"山东政航工程咨询有限公司","QY_REGION_NAME":"山东省-临沂市","QY_ORG_CODE":"91371302092199458A","COLLECT_TIME":1477106426000,"RN":43,"QY_ID":"616160676166636361616466606865656560","QY_SRC_TYPE":"0","OLD_CODE":"092199458"},{"QY_FR_NAME":"张洪建","QY_REGION":"530100","QY_NAME":"云南金吉安建设咨询监理有限公司","QY_REGION_NAME":"云南省-昆明市","QY_ORG_CODE":"91530000719414685K","COLLECT_TIME":1479833531000,"RN":44,"QY_ID":"616160676166636361616466606865656567","QY_SRC_TYPE":"0","OLD_CODE":"719414685"},{"QY_FR_NAME":"张志明","QY_REGION":"610700","QY_NAME":"汉中志诚建设工程监理有限责任公司","QY_REGION_NAME":"陕西省-汉中市","QY_ORG_CODE":"91610725667992132H","COLLECT_TIME":1477106417000,"RN":45,"QY_ID":"616160676166636361616466606865656568","QY_SRC_TYPE":"0","OLD_CODE":"667992132"}],"pageNum":2,"pageSize":15,"total":450},"message":"","success":true}

到这里,爬取数据部分全部结束。

在这里插入图片描述

关于Python技术储备

学好 Python 不论是就业还是做副业赚钱都不错,但要学会 Python 还是要有一个学习规划。最后大家分享一份全套的 Python 学习资料,给那些想学习 Python 的小伙伴们一点帮助!

一、Python所有方向的学习路线

Python所有方向路线就是把Python常用的技术点做整理,形成各个领域的知识点汇总,它的用处就在于,你可以按照上面的知识点去找对应的学习资源,保证自己学得较为全面。

二、学习软件

工欲善其事必先利其器。学习Python常用的开发软件都在这里了,给大家节省了很多时间。

三、入门学习视频

我们在看视频学习的时候,不能光动眼动脑不动手,比较科学的学习方法是在理解之后运用它们,这时候练手项目就很适合了。

四、实战案例

光学理论是没用的,要学会跟着一起敲,要动手实操,才能将自己的所学运用到实际当中去,这时候可以搞点实战案例来学习。

五、面试资料

我们学习Python必然是为了找到高薪的工作,下面这些面试题是来自阿里、腾讯、字节等一线互联网大厂最新的面试资料,并且有阿里大佬给出了权威的解答,刷完这一套面试资料相信大家都能找到满意的工作。


这份完整版的Python全套学习资料已经上传CSDN,朋友们如果需要可以微信扫描下方CSDN官方认证二维码免费领取【保证100%免费

标签:逆向,FR,CODE,NAME,QY,牢饭,REGION,爬虫,COLLECT
来源: https://blog.csdn.net/m0_67402125/article/details/123227928

本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享;
2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关;
3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关;
4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除;
5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

专注分享技术,共同学习,共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有