node.js - nodejs代理IP發(fā)送HTTP請(qǐng)求
問(wèn)題描述
問(wèn)題解答
回答1:用request庫(kù)可以很方便這種任務(wù),下面是我寫(xiě)的一個(gè)獲取可用代理列表 + 用代理訪問(wèn)目標(biāo)網(wǎng)站的Demo
var request = require('request');var iconv = require(’iconv-lite’);var Promise = require('bluebird');function getProxyList() { var apiURL = ’http://www.66ip.cn/mo.php?sxb=&tqsl=100&port=&export=&ktip=&sxa=&submit=%CC%E1++%C8%A1&textarea=http%3A%2F%2Fwww.66ip.cn%2F%3Fsxb%3D%26tqsl%3D100%26ports%255B%255D2%3D%26ktip%3D%26sxa%3D%26radio%3Dradio%26submit%3D%25CC%25E1%2B%2B%25C8%25A1’; return new Promise((resolve, reject) => {var options = { method: ’GET’, url: apiURL, gzip: true, encoding: null, headers: {’Accept’: ’text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8’,’Accept-Encoding’: ’gzip, deflate’,’Accept-Language’: ’zh-CN,zh;q=0.8,en;q=0.6,zh-TW;q=0.4’,’User-Agent’: ’Mozilla/8.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36’,’referer’: ’http://www.66ip.cn/’ },};request(options, function (error, response, body) { try {if (error) throw error;if (/meta.*charset=gb2312/.test(body)) { body = iconv.decode(body, ’gbk’);}var ret = body.match(/d{1,3}.d{1,3}.d{1,3}.d{1,3}:d{1,4}/g);resolve(ret); } catch (e) {return reject(e); }}); })}getProxyList().then(function (proxyList) { var targetOptions = {method: ’GET’,url: ’http://ip.chinaz.com/getip.aspx’,timeout: 8000,encoding: null, }; //這里修改一下,變成你要訪問(wèn)的目標(biāo)網(wǎng)站 proxyList.forEach(function (proxyurl) {console.log(`testing ${proxyurl}`);targetOptions.proxy = ’http://’ + proxyurl;request(targetOptions, function (error, response, body) { try {if (error) throw error;body = body.toString();console.log(body);eval(`var ret = ${body}`);if (ret) { console.log(`驗(yàn)證成功==>> ${ret.address}`);} } catch (e) {// console.error(e); }}); });}).catch(e => { console.log(e);})
那你首先得找到一堆代理啊,據(jù)我所知這種動(dòng)態(tài)代理一般靠譜的都是收費(fèi)的。具體思路就是請(qǐng)求代理提供商的接口獲得代理服務(wù)器列表,然后每次請(qǐng)求都隨機(jī)選一個(gè)作為代理
相關(guān)文章:
1. javascript - 有什么比較好的網(wǎng)頁(yè)版shell前端組件?2. sql語(yǔ)句 - mysql中關(guān)聯(lián)表查詢問(wèn)題3. javascript - 原生canvas中如何獲取到觸摸事件的canvas內(nèi)坐標(biāo)?4. css - div外層有一圈白色5. javascript - [js]為什么畫(huà)布里不出現(xiàn)圖片呢?在線等6. javascript - vscode alt+shift+f 格式化js代碼,通不過(guò)eslint的代碼風(fēng)格檢查怎么辦。。。7. javascript - 如何將一個(gè)div始終固定在某個(gè)位置;無(wú)論屏幕和分辨率怎么變化;div位置始終不變8. html - vue項(xiàng)目中用到了elementUI問(wèn)題9. javascript - iframe 為什么加載網(wǎng)頁(yè)的時(shí)候滾動(dòng)條這樣顯示?10. javascript - 求解答:實(shí)例對(duì)象調(diào)用constructor,此時(shí)constructor內(nèi)的this的指向?
