找回密码
 会员注册
查看: 39|回复: 0

python爬取携程全国旅游景点信息-2024.4.13

[复制链接]

5

主题

0

回帖

16

积分

新手上路

积分
16
发表于 2024-9-8 17:21:05 | 显示全部楼层 |阅读模式
1.概述         携程网是中国领先的在线旅行服务公司,提供酒店预订、机票预订、旅游度假、商旅管理等服务。携程网上有大量的旅游景点和酒店信息,这些信息对于旅行者和旅游业者都有很大的价值。通过爬虫技术,我们可以从携程网上获取这些信息,并进行数据清洗、数据分析、数据可视化等操作,从而得到有用的洞察和建议。 2.安装requests库                  在开始之前,请确保你已经安装了以下Python库:                requests:用于发送HTTP请求并获取网页内容。                你可以使用pip来安装这些库:pipinstallrequests3.爬取携程旅游网站数据        首先,我们需要确定要爬取的页面。假设我们想要获取携程旅游网站上某个目的地的旅游信息。如下例如北京。   当前接口链接和post参数url='https://m.ctrip.com/restapi/soa2/18109/json/getAttractionList?_fxpcqlniredt=09031015313388236487&x-traceID=09031015313388236487-1712974794650-8267936'data={"index":1,"count":10,"sortType":1,"isShowAggregation":true,"districtId":1,"scene":"DISTRICT","pageId":"214062","traceId":"14f9745c-92ad-f5c5-07bb-171293c80647","extension":[{"name":"osVersion","value":"10"},{"name":"deviceType","value":"windows"}],"filter":{"filterItems":[]},"crnVersion":"2020-09-0122:00:45","isInitialState":true,"head":{"cid":"09031015313388236487","ctok":"","cver":"1.0","lang":"01","sid":"8888","syscode":"09","auth":"","xsid":"","extension":[]}}4.开始正式代码        headers={'User-Agent':'Mozilla/5.0(WindowsNT10.0;Win64;x64)AppleWebKit/537.36(KHTML,likeGecko)Chrome/123.0.0.0Safari/537.36Edg/123.0.0.0','Cookie':你的cookies}html=requests.post(url,headers=headers,json=data).json()attractionList=html['attractionList']forattractioninattractionList:data=attraction['card']commentCount=data['commentCount']commentScore=data['commentScore']coordinate=[data['coordinate']['latitude'],data['coordinate']['longitude']]coverImageUrl=data.get('coverImageUrl','')#距离distanceStr=data.get('distanceStr','')#地点displayField=data.get('displayField',None)heatScore=data.get('heatScore','')#景点名poiName=data['poiName']isFree=data['isFree']ifisFree:price=0#原价marketPrice=0else:price=data.get('price',0)#原价marketPrice=data.get('marketPrice',0)#类别信息sightCategoryInfo=data.get('sightCategoryInfo','')#标签tagNameList=data.get('tagNameList','')#5asightLevelStr=data.get('sightLevelStr',None)5.保存到csvf=open('csv/全国各景点全.csv','w',encoding="utf-8",newline='')csvwrite=csv.writer(f)csvwrite.writerow(['城市','景点名','地点','距离','坐标','评论数','评论分','热评分','封面','是否免费','价格','原价','类别信息','标签','是否5A'])csvwrite.writerow([city,poiName,displayField,distanceStr,coordinate,commentCount,commentScore,heatScore,coverImageUrl,isFree,price,marketPrice,sightCategoryInfo,tagNameList,sightLevelStr])6.通过获取全国cityid,可请求全国景点数据并保存        全国景点数据csv地址 https://download.csdn.net/download/britlee/89115745
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 会员注册

本版积分规则

QQ|手机版|心飞设计-版权所有:微度网络信息技术服务中心 ( 鲁ICP备17032091号-12 )|网站地图

GMT+8, 2025-1-11 01:31 , Processed in 0.464172 second(s), 26 queries .

Powered by Discuz! X3.5

© 2001-2025 Discuz! Team.

快速回复 返回顶部 返回列表