728x90
반응형
라이브러리를 설치하고 아래와 같이 선언
import requests
import pandas as pd
from bs4 import BeautifulSoup
import time
import requests
import pandas as pd
from bs4 import BeautifulSoup
import time
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Whale/3.22.205.18 Safari/537.36'
}
response = requests.get(url, headers = headers).text
print(response)
headers는 내 PC정보를 같이 보내려고 만든거고..없어도 된다...
requests 를 사용하여 티스토리 메인의 html을 긁어와서 response에 담아두고
print로 출력
그럼 소스보기 하면 나오는 것들이 솰라 솰라 솰라 출력된다...
<!doctype html>
<html lang="ko">
<head>
<meta charset="utf-8">
<meta name="google-site-verification" content="Djy29naX64H0z8fGEOEOd-k40Sp65VRnz1sm_thWPhw" />
<meta property="og:url" content="https://www.tistory.com">
<meta property="og:site_name" content="Tistory">
<meta property="og:title" content="Tistory">
<meta property="og:description" content="좀 아는 블로거들의 유용한 이야기">
<meta property="og:image" content="//t1.daumcdn.net/tistory_admin/static/images/openGraph/tistoryOpengraph.png">
<title>TISTORY</title>
<link rel="icon" href="https://t1.daumcdn.net/tistory_admin/favicon/tistory_favicon_32x32.ico" sizes="any">
<link rel="icon" type="image/svg+xml" href="https://t1.daumcdn.net/tistory_admin/top_v2/bi-tistory-favicon.svg" />
<link rel="apple-touch-icon" href="https://t1.daumcdn.net/tistory_admin/top_v2/tistory-apple-touch-favicon.png"> <link rel="stylesheet" type="text/css" href="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/css/ext/swiper.min.css">
<link rel="stylesheet" type="text/css" href="//t1.daumcdn.net/tistory_admin/www/style/top/font.css">
<link rel="stylesheet" type="text/css" href="//t1.daumcdn.net/tistory_admin/assets/tistory-gnb/3.3.0/gnb.min.css">
<link rel="stylesheet" type="text/css" href="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/css/pc/top.css">
<script type="text/javascript" src="https://developers.kakao.com/sdk/js/kakao.min.js" ></script>
<!--[if lt IE 9]>
<script src="https://t1.daumcdn.net/tistory_admin/lib/jquery-1.12.4.min.js"></script>
<![endif]-->
<!--[if gte IE 9]><!-->
<script src="https://t1.daumcdn.net/tistory_admin/lib/jquery-3.1.0.min.js"></script>
<!--<![endif]-->
<script src="https://t1.daumcdn.net/tistory_admin/lib/fingerprint2-1.4.2.min.js"></script>
<script type="text/javascript" src="//m1.daumcdn.net/svc/original/U03/cssjs/jquery/plugin/jquery.cookie-1.4.0.min.js"></script>
<script type="text/javascript" src="//s1.daumcdn.net/svc/original/U03/cssjs/userAgent/userAgent-1.0.14.min.js"></script>
</head>
<body>
<div id="kakaoIndex"><!-- 웹접근성용 바로가기 링크 모음 -->
<a href="#kakaoBody">본문 바로가기</a>
<a href="#kakaoLnb">메뉴 바로가기</a> <!-- 2017-05-31 수정 : href 값 kakaoGnb -> kakaoLnb 로 변경 -->
</div>
<div id="kakaoWrap" class="tistory_type1">
<div id="kakaoHead" role="banner" class="#GNB kakao_head"><!-- head_type1 : 추가타입 -->
<div class="inner_header">
<h1>
<a href="/" id="kakaoServiceLogo" class="#logo img_common_tistory tit_tistory">티스토리</a>
</h1>
<div id="kakaoGnb" role="navigation" class="gnb_tistory">
<h2 class="screen_out">서비스 주요 메뉴</h2>
<ul class="list_gnb">
<li ><a href="/feed" class="link_gnb">피드</a></li>
<li ><a href="/category" class="link_gnb">스토리</a></li>
<li ><a href="/skin" class="link_gnb">스킨</a></li>
<li ><a href="/community" class="link_gnb">포럼</a></li>
</ul>
</div>
<div class="info_tistory">
<div class="logn_tistory" style="display:none">
<h2 class="screen_out">로그인정보</h2>
<a href="/auth/login?redirectUrl=http%3A%2F%2Fwww.tistory.com%2F" class="link_log_info">로그인하기</a>
<a href="/member/join" class="btn_tistory btn_log_info">가입하기</a>
</div>
</div>
</div>
</div>
<hr class="hide">
<div id="kakaoContent" role="main">
<div id="cMain">
<div id="mFeature" class="feture_main feture_top">
<h2 class="screen_out">PRPMOTION</h2>
<div class="swiper-container">
<div class="swiper-wrapper main_slide">
<div class="swiper-slide scene_slide init_slide" style="background-image:url(https://img1.daumcdn.net/thumb/R1920x0/?scode=mtistory2&fname=https%3A%2F%2Ft1.daumcdn.net%2Fsection%2Foc%2Ffd708e84887d4dff9da3cf2a4374e50e)">
<div class="dimm_g_type2"></div>
<div class="scene_content">
<div class="inner_scene">
<div class="scene_tistory scene_tistory_type2">
<strong class="tit_scene"><b>'스토리 크리에이터'</b><br> 제도를 소개합니다.</strong>
<p class="desc_scene">스토리에서 특정 분야에 전문성 있는 창작자를 지원해 드려요.</p>
<div class="wrap_btn"><a href="https://storyhome.kakao.com/storycreator/?t_src=tistory_pc_topbanner" class="btn_tistory btn_tistory_type5">자세히보기</a></div>
</div>
</div>
</div>
</div>
<div class="swiper-slide scene_slide " style="background-image:url(https://img1.daumcdn.net/thumb/R1920x0/?scode=mtistory2&fname=https%3A%2F%2Ft1.daumcdn.net%2Fsection%2Foc%2F3b837b34cc814c7b9b3a12f772e2441f)">
<div class="dimm_g_type2"></div>
<div class="scene_content">
<div class="inner_scene">
<div class="scene_tistory scene_tistory_type2">
<strong class="tit_scene"><b>Story로 뭉쳐</b><br>
더 넓고 깊어집니다</strong>
<p class="desc_scene">티스토리가 브런치스토리, 카카오스토리와 함께<br>
Story로 새롭게 출발합니다.</p>
<div class="wrap_btn"><a href="https://notice.tistory.com/2637" class="btn_tistory btn_tistory_type5">자세히보기</a></div>
</div>
</div>
</div>
</div>
<div class="swiper-slide scene_slide " style="background-image:url(https://img1.daumcdn.net/thumb/R1920x0/?scode=mtistory2&fname=https%3A%2F%2Ft1.daumcdn.net%2Fsection%2Foc%2Ff7ba106243774ba6a8cd65419b3f6d17)">
<div class="dimm_g_type2"></div>
<div class="scene_content">
<div class="inner_scene">
<div class="scene_tistory scene_tistory_type2">
<strong class="tit_scene"><b>티스토리 모바일홈</b>이<br>
새로워졌어요</strong>
<p class="desc_scene">'오늘의 이슈'부터 '주목받는 블로그'까지<br>
새로운 모바일홈에서 살펴보세요.</p>
<div class="wrap_btn"><a href="https://notice.tistory.com/2635" class="btn_tistory btn_tistory_type5">자세히보기</a></div>
</div>
</div>
</div>
</div>
<div class="swiper-slide scene_slide white " style="background-image:url(https://img1.daumcdn.net/thumb/R1920x0/?scode=mtistory2&fname=https%3A%2F%2Ft1.daumcdn.net%2Fsection%2Foc%2Feaa6512fa207497e8a88e6d70fd5aa1b)">
<div class="dimm_g_type2"></div>
<div class="scene_content">
<div class="inner_scene">
<div class="scene_tistory scene_tistory_type2">
<strong class="tit_scene">당신의 이야기가<br>
<b>값진 수익이 됩니다.</b></strong>
<p class="desc_scene">심사부터 광고 설정, 수익 확인까지<br>티스토리에서 바로 할 수 있어요.</p>
<div class="wrap_btn"><a href="https://notice.tistory.com/2572" class="btn_tistory ">자세히 보기</a></div>
</div>
</div>
</div>
</div>
<div class="swiper-slide scene_slide white " style="background-image:url(https://img1.daumcdn.net/thumb/R1920x0/?scode=mtistory2&fname=https%3A%2F%2Ft1.daumcdn.net%2Fsection%2Foc%2F058ac743cf2640fb885bd1f9e9635ab2)">
<div class="dimm_g_type2"></div>
<div class="scene_content">
<div class="inner_scene">
<div class="scene_tistory scene_tistory_type2">
<strong class="tit_scene">이제부터 티스토리를<br>
<b>카카오계정</b>으로<br>
이용해주세요.</strong>
<p class="desc_scene"> </p>
<div class="wrap_btn"><a href="https://notice.tistory.com/2553" class="btn_tistory ">자세히 보기</a></div>
</div>
</div>
</div>
</div>
<div class="swiper-slide scene_slide white " style="background-image:url(https://img1.daumcdn.net/thumb/R1920x0/?scode=mtistory2&fname=https%3A%2F%2Ft1.daumcdn.net%2Fsection%2Foc%2F8532fb82eed84069a36fa0f05170557c)">
<div class="dimm_g_type2"></div>
<div class="scene_content">
<div class="inner_scene">
<div class="scene_tistory scene_tistory_type2">
<strong class="tit_scene">티스토리 앱 2.0<br>
<b>업데이트</b></strong>
<p class="desc_scene">모바일에서도 편리한 블로그 관리를 위해<br>티스토리 앱이 새 옷을 입었습니다.</p>
<div class="wrap_btn"><a href="https://notice.tistory.com/2523" class="btn_tistory ">자세히 보기</a></div>
</div>
</div>
</div>
</div>
</div>
<button type="button" class="btn_scene btn_prev">
<span class="img_top ico_prev ico_prev_type1">이전</span>
<span class="img_top ico_prev ico_prev_type2"></span>
</button>
<button type="button" class="btn_scene btn_next">
<span class="img_top ico_next ico_next_type1">다음</span>
<span class="img_top ico_next ico_next_type2"></span>
</button>
</div>
</div>
<div id="mArticle">
<div class="section_content section_showcase">
<h2 class="screen_out">SHOWCASE</h2>
<div class="wrap_tit">
<strong class="tit_section">이 놀라운 사이트들이<br><em>모두 <span class="txt_da">다</span>티스토리</em>라는 사실</strong>
<p class="desc_section">티스토리는 그저 공간일 뿐이에요. 그 공간에 무슨일이 일어날지는 아무도 상상할 수 없죠.</p>
</div>
<div class="case_list">
<div class="inner_case">
<div class="slide_case" style="width:50000px">
<a class="site_exam" href="http://leeheehoon.tistory.com/" target="_blank">
<div class="wrap_thumb">
<img width="528" height="396" src="https://img1.daumcdn.net/thumb/C1200x900/?scode=mtistory2&fname=https%3A%2F%2Ft1.daumcdn.net%2Ftistory_admin%2Fassets%2Fpromotions%2F72a26c24396048fa9fb668442021a18b" class="thumb_g" alt="" />
</div>
<div class="info_exam">
<strong class="tit_exam">Beyond Perspective</strong>
<p class="txt_exam">Lee Heehoon,<br> photographer for ohmynews Seoul S.Korea</p>
<span class="link_exam">http://leeheehoon.tistory.com/<span class="img_top ico_view"></span></span>
</div>
</a>
<a class="site_exam" href="http://so-oh.tistory.com/category" target="_blank">
<div class="wrap_thumb">
<img width="528" height="396" src="https://img1.daumcdn.net/thumb/C1200x900/?scode=mtistory2&fname=http%3A%2F%2Fcfile1.uf.tistory.com%2Fimage%2F99368F335BB2C64F29335D" class="thumb_g" alt="" />
</div>
<div class="info_exam">
<strong class="tit_exam">yunscent</strong>
<p class="txt_exam">사진이 그림이 됩니다.</p>
<span class="link_exam">http://so-oh.tistory.com/category<span class="img_top ico_view"></span></span>
</div>
</a>
<a class="site_exam" href="http://hotel990.com" target="_blank">
<div class="wrap_thumb">
<img width="528" height="396" src="https://img1.daumcdn.net/thumb/C1200x900/?scode=mtistory2&fname=http%3A%2F%2Fcfile10.uf.tistory.com%2Fimage%2F99A81B3C5BB2C6EF2C9883" class="thumb_g" alt="" />
</div>
<div class="info_exam">
<strong class="tit_exam">HOTEL 990</strong>
<p class="txt_exam"> WELCOME TO THE HOTEL990</p>
<span class="link_exam">http://hotel990.com<span class="img_top ico_view"></span></span>
</div>
</a>
<a class="site_exam" href="http://mistyfriday.kr/" target="_blank">
<div class="wrap_thumb">
<img width="528" height="396" src="https://img1.daumcdn.net/thumb/C1200x900/?scode=mtistory2&fname=http%3A%2F%2Fcfile2.uf.tistory.com%2Fimage%2F996CC03C5BB2C66823DA6C" class="thumb_g" alt="" />
</div>
<div class="info_exam">
<strong class="tit_exam">빛으로 쓴 편지</strong>
<p class="txt_exam">여행하고 글을 씁니다.</p>
<span class="link_exam">http://mistyfriday.kr/<span class="img_top ico_view"></span></span>
</div>
</a>
<a class="site_exam" href="https://hygphoto.tistory.com/" target="_blank">
<div class="wrap_thumb">
<img width="528" height="396" src="https://img1.daumcdn.net/thumb/C1200x900/?scode=mtistory2&fname=http%3A%2F%2Fcfile2.uf.tistory.com%2Fimage%2F99D6CF405BB2C6B824A969" class="thumb_g" alt="" />
</div>
<div class="info_exam">
<strong class="tit_exam">HYGPHOTO</strong>
<p class="txt_exam">아름다운 일상의 한 컷</p>
<span class="link_exam">https://hygphoto.tistory.com/<span class="img_top ico_view"></span></span>
</div>
</a>
</div>
</div>
<div class="area_btn area_prev">
<button type="button" class="btn_tistory_type4 btn_prev slick-arrow" style=""><span class="img_top ico_prev">이전</span></button>
</div>
<div class="area_btn area_next">
<button type="button" class="btn_tistory_type4 btn_next slick-arrow" style=""><span class="img_top ico_next">다음</span></button>
</div>
</div>
<div class="relative_list">
<a class="wrap_logo" style="background-image:url(https://img1.daumcdn.net/thumb/R300x100/?scode=mtistory2&fname=http%3A%2F%2Fcfile27.uf.tistory.com%2Fimage%2F99A6963C5BB2C5C32AB7FC)" target="_blank" href="http://thehyundaiblog.com">현대백화점 공식블로그</a>
<a class="wrap_logo" style="background-image:url(https://img1.daumcdn.net/thumb/R300x100/?scode=mtistory2&fname=http%3A%2F%2Fcfile23.uf.tistory.com%2Fimage%2F991B033C5BB2C59024121E)" target="_blank" href="http://www.witheverland.com">withEverland</a>
<a class="wrap_logo" style="background-image:url(https://img1.daumcdn.net/thumb/R300x100/?scode=mtistory2&fname=http%3A%2F%2Fcfile27.uf.tistory.com%2Fimage%2F9989ED3B5BB2C5D3287867)" target="_blank" href="http://blogdegalleria.tistory.com">갤러리아 공식블로그 'BLOG DE GALLERIA'</a>
<a class="wrap_logo" style="background-image:url(https://img1.daumcdn.net/thumb/R300x100/?scode=mtistory2&fname=http%3A%2F%2Fcfile24.uf.tistory.com%2Fimage%2F9950223C5BB2C57B2225FF)" target="_blank" href="http://blog.ibk.co.kr">IBK 기업은행 블로그</a>
<a class="wrap_logo" style="background-image:url(https://img1.daumcdn.net/thumb/R300x100/?scode=mtistory2&fname=http%3A%2F%2Fcfile21.uf.tistory.com%2Fimage%2F995901365BB2C53C27D162)" target="_blank" href="http://if-blog.tistory.com">교육부 공식 블로그</a>
<a class="wrap_logo" style="background-image:url(https://img1.daumcdn.net/thumb/R300x100/?scode=mtistory2&fname=http%3A%2F%2Fcfile29.uf.tistory.com%2Fimage%2F9988F0395BB2C5B11F8B83)" target="_blank" href="http://blog.paradise.co.kr">파라다이스 블로그</a>
</div>
</div>
<div class="section_content section_skin">
<h2 class="screen_out">스킨고르기</h2>
<div class="wrap_tit wrap_tit_type1">
<strong class="tit_section">마음에 드는<em><span class="img_square">스킨</span>고르기</em>부터 시작해보세요</strong>
<p class="desc_section">티스토리는 얼굴이 없어요. <br>당신이 얼굴을 직접 제작할 수 있도록 다양한 스킨을 제공할 뿐이에요.</p>
</div>
<div class="icon_tistory icon_expression"></div>
<div class="wrap_tit">
<strong class="tit_section tit_section_type2">마음대로<br><em>조립하는</em><br>내 콘텐츠</strong>
<p class="desc_section">내 마음대로 위치를 바꿔<br>하나의 커버로도 전혀 다른 스킨을<br> 만들 수 있어요.</p>
</div>
<div class="box_skin">
<div class="wrap_thumb_skin_01">
<span class="" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin1_1.jpg);"></span>
<span class="" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin1_2.jpg);"></span>
<span class="" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin1_3.jpg);"></span>
<span class="" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin1_1.jpg);"></span>
<span class="" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin1_2.jpg);"></span>
<span class="" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin1_3.jpg);"></span>
</div>
<div class="wrap_thumb_skin_02">
<div class="inner_thumb">
<span class=" _left" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin2_1.jpg);height:140px" data-index="1"></span>
<span class=" _left" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin2_2.jpg);height:150px" data-index="2"></span>
<span class=" _left" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin2_3.jpg);height:432px" data-index="3"></span>
<span class=" _right" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin2_3.jpg);height:432px" data-index="1"></span>
<span class=" _right" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin2_2.jpg);height:150px" data-index="2"></span>
<span class=" _right" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/thumb_skin2_1.jpg);height:140px" data-index="3"></span>
</div>
</div>
</div>
<a href="/skin/pg_Poster" class="link_section">
<strong class="tit_link">Poster</strong>
<span class="desc_link">스킨 보기</span>
<span class="img_top ico_link_arrow"></span>
</a>
</div>
<div class="section_content section_photo">
<h2 class="screen_out">사진변경</h2>
<div class="wrap_tit">
<strong class="tit_section tit_section_type2"><em>사진만 바꾸어도</em><br>다른 분위기</strong>
<p class="desc_section">스킨은 그저 캔버스일 뿐입니다.<br>스킨 속 같은 커버라도 이미지에 따라<br>전혀 다른 분위기를 연출할 수 있어요. </p>
</div>
<div class="wrap_photo_skin">
<div class="photo_skin">
<div class="skin_move">
<span class="bg_img" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/photo_skin01.jpg);"></span>
<span class="bg_img" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/photo_skin03.jpg);"></span>
</div>
<div class="box_img_skin">
<div class="wrap_skin">
<div class="skin_move">
<span class="bg_img_sm" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin01_sm.jpg);"></span>
<span class="bg_img_sm" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin03_sm.jpg);"></span>
</div>
</div>
<div class="wrap_skin">
<div class="txt_img_skin" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin_txt02.png);"></div>
<div class="txt_img_skin" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin_txt04.png);"></div>
</div>
</div>
</div>
<div class="photo_skin">
<div class="skin_move">
<span class="bg_img" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/photo_skin02.jpg);"></span>
<span class="bg_img" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/photo_skin04.jpg);"></span>
</div>
<div class="box_img_skin">
<div class="wrap_skin" >
<div class="txt_img_skin" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin_txt01.png);"></div>
<div class="txt_img_skin" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin_txt03.png);"></div>
</div>
<div class="wrap_skin">
<div class="skin_move">
<span class="bg_img_sm" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin02_sm.jpg);"></span>
<span class="bg_img_sm" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/photo_skin04_sm.jpg);"></span>
</div>
</div>
</div>
</div>
</div>
<a href="/skin/pg_Whatever" class="link_section">
<strong class="tit_link">Whatever</strong>
<span class="desc_link">스킨 보기</span>
<span class="img_top ico_link_arrow"></span>
</a>
</div>
<div class="section_content section_magazine"><!-- 모션 진행시 'rolling_up' 클래스 추가 -->
<h2 class="screen_out">블로그와 매거진처럼 </h2>
<div class="wrap_tit">
<strong class="tit_section tit_section_type2">때로는 <em>블로그</em>처럼,<br>때로는 <em>매거진</em>처럼</strong>
<p class="desc_section">복잡하지 않으면서 깔끔한 스킨 커버로<br>콘텐츠에 힘을 실어 줄 수 있습니다.</p>
</div>
<div class="bg_magazine" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/magazin_01.jpg);">
<div class="bg_magazine inner_bg" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pc/magazin_02.jpg);"></div>
</div>
<div class="box_magazine" style="background-image:url(//t1.daumcdn.net/tistory_admin/static/top/pcrtn/magazine_header.jpg">
<div class="inner_box">
<img src="//t1.daumcdn.net/tistory_admin/static/top/pc/magazine_sm.jpg" class="thumb_magazine" width="540" alt="">
</div>
</div>
<a href="/skin/xf_Letter" class="link_section">
<strong class="tit_link">Letter</strong>
<span class="desc_link">스킨 보기</span>
<span class="img_top ico_link_arrow"></span>
</a>
</div>
<div class="section_content section_guide">
<h2 class="screen_out">이용 가이드</h2>
<div class="inner_section">
<strong class="tit_guide">더 잘 쓰고 싶어요?</strong>
<div class="info_guide">
<a href="https://notice.tistory.com/1797" target="_blank" class="link_guide">당신의 티스토리에 다양한 <em>플러그인</em>을 <br>사용할 수 있어요.</a>
<a href="https://notice.tistory.com/2444" target="_blank" class="link_guide">사이트 보안을 위해 더 안전한 <br><em>암호화 접속</em>을 사용해보세요.</a>
<a href="https://notice.tistory.com/2400" target="_blank" class="link_guide">콘텐츠를 고정된 메뉴로 유지하고 싶다면 <br><em>페이지</em>를 이용해보세요.</a>
<a href="https://notice.tistory.com/1784" target="_blank" class="link_guide">나만의 <em>도메인</em>을 연결해보세요.<br>당신의 티스토리에 엣지가 생길거예요.</a>
<a href="https://notice.tistory.com/2384" target="_blank" class="link_guide"><em>로그인 보안</em>기능으로 당신의 정보와<br>콘텐츠를 안전하게 지키세요! </a>
</div>
</div>
</div>
</div>
</div>
</div>
<hr class="hide">
<div id="daumFoot" class="footer_comm">
<div class="inner_foot">
<div class="cont_tistory">
<strong class="img_common_tistory txt_tistory">TISTORY</strong>
<p class="desc_footer">티스토리는 카카오에서 <em>♥</em>을 담아 만듭니다.<br>© Kakao Corp.</p>
<div class="search_tistory" id="footerSearchContainer">
<h2 class="screen_out">검색</h2>
<form action="//search.daum.net/search" target="_blank" role="search" method="get">
<fieldset>
<legend class="screen_out">티스토리 검색</legend>
<input type="hidden" name="w" value="blog">
<input type="hidden" name="f" value="section">
<input type="hidden" name="SA" value="tistory">
<input type="hidden" name="lpp" value="10">
<input type="hidden" name="nil_profile" value="vsearch">
<input type="hidden" name="nil_src" value="tistory">
<div class="inp_text">
<input type="text" id="footerSearchInput" placeholder="검색어입력" title="검색" name="q">
</div>
<button type="submit" class="img_common_tistory btn_search">검색</button>
</fieldset>
</form>
</div>
</div>
<div class="question_tistory">
<h2 class="screen_out">문의목록</h2>
<div class="wrap_question">
<span class="tit_question">메뉴가 궁금할 땐</span>
<ul class="list_question">
<li><a href="/skin" class="link_txt">스킨</a></li>
<li><a href="/community/forum" class="link_txt">포럼</a></li>
<li><a href="/category" class="link_txt">스토리</a></li>
</ul>
</div>
<div class="wrap_question">
<span class="tit_question">사용하다 궁금할 땐</span>
<ul class="list_question">
<li><a href="/guide/api/manage/register" class="link_txt">오픈 API</a></li>
<li><a href="https://tistory.github.io/document-tistory-skin/" class="link_txt" target="_blank">스킨가이드</a></li>
<li><a href="https://cs.kakao.com/requests?service=175&locale=ko" class="link_txt" target="_blank">고객센터</a></li>
</ul>
</div>
<div class="wrap_question">
<span class="tit_question">정책이 궁금할 땐</span>
<ul class="list_question">
<li><a href="/info/contract" class="link_txt">이용약관</a></li>
<li><a href="https://www.kakao.com/policy/privacy?lang=ko" class="link_txt"><em>카카오 개인정보처리방침</em></a></li>
<li><a href="https://www.kakao.com/policy/safeguard" class="link_txt">청소년보호정책</a></li>
<li><a href="/info/openApi" class="link_txt">오픈API 이용약관</a></li>
<li><a href="/info/emailPolicy" class="link_txt">Email 수집거부정책</a></li>
</ul>
</div>
<div class="wrap_question">
<span class="tit_question">도움이 필요할 땐</span>
<ul class="list_question">
<li><a href="https://www.kakao.com/policy/right" class="link_txt" target="_blank">권리침해신고</a></li>
<li><a href="https://cs.kakao.com/conflict" class="link_txt" target="_blank">상거래 피해 구제신청</a></li>
</ul>
</div>
</div>
</div>
</div>
</div>
<script type="text/javascript" src="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/js/ext/underscore-1.9.0.min.js"></script>
<script src="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/js/ext/swiper.min.js"></script>
<script src="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/js/T.js"></script>
<script src="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/js/T.util.js"></script>
<script src="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/js/T.auth.js"></script>
<script src="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/js/pc/T.p.top.js"></script>
<script type="text/javascript" src="//t1.daumcdn.net/tiara/js/v1/tiara.min.js"></script>
<script type="text/javascript">
window.addEventListener('load', function (){
TiaraTracker.getInstance()
.setSvcDomain('user.tistory.com')
.trackPage('user')
.setSection('top')
.setPage('user')
.setKakaoAppKey('3e6ddd834b023f24221217e370daed18')
.setAppUserId('1682679362')
.setAutoClick(true)
.setDisableSendLegacy(true)
.track();
});
</script>
<script type="text/javascript">
var Config = {
production: true,
service: 'tistory',
useWriteButton: false,
useBlogList: true
};
var appInfo = {
domain: 'tistory.com',
topUrl: 'https://www.tistory.com',
loginUrl: 'https://www.tistory.com/auth/login',
logoutUrl: 'https://www.tistory.com/auth/logout',
memberUrl: 'https://www.tistory.com/member',
gnbBaseUrl: 'https://www.tistory.com/accio',
thumbnailBaseUrl: 'https://i1.daumcdn.net/thumb',
kakaoAccountUrl: 'https://accounts.kakao.com/weblogin/account/info'
};
var initData = {
};
var production = true;
initData.user = {"id":4847188,"loginId":"tmlhk@nate.com","accountType":"kakao","name":"그미s","description":"","level":1,"defaultBlogUrl":null,"emailAccepted":true,"blogCreateCount":1,"defaultBlogId":4663977,"joined":null,"profileImageUrl":"https://tistory1.daumcdn.net/tistory/4663977/attach/0455ed9b052a475381ec57bc44d3548b","imageCache":true,"defaultProfileImageUrl":"https://t1.daumcdn.net/tistory_admin/blog/admin/profile_default_04.png"};
initData.userActionLimitation = {"level":"BEGINNER","dailyWriteLimit":50,"dailyOpenPublishLimit":15,"createBlogLimit":10,"activeBlogLimit":5};
</script>
<script src="https://t1.daumcdn.net/tistory_admin/assets/tistory-gnb/3.3.0/gnb.min.js"></script>
<script src="//t1.daumcdn.net/tistory_admin/assets/tistory-web-top/1694410865/static/js/pc/T.p.top.home.js"></script>
<!-- IE9 이하에서 영상 슬라이드 제거 위한 조건부 스크립트 적용 -->
<!--[if lte IE9]>
<script type="text/javascript">
(function($) {
var slideNum = $('.scene_slide').length,
isVideo = $('.scene_slide').children("video").length,
videoItemSlide = $("video").parent(),
videoItem = $('video');
if(slideNum == 1){
if(isVideo == 1){
videoItem.remove();
}
}else{
if(isVideo == 1){
videoItemSlide.remove();
}
}
})(jQuery);
</script>
<![endif]-->
</body>
</html>이렇게 데이터를 가지고 오고나면
담아둘 변수 = soup.find_all("div", class_="클래스명")
이렇게 우리가 필요한 부분의 div를 긁어서 다시 변수에 보관한다.
그럼 뭘 가져올지 지정해주자
문자를 새로 보관 = 가져올 변수.find("a", class_="클래스명_").text.strip()
이렇게 하면 HTML에서 <A CLASS = "클래스명_") 이렇게 된 곳의 글짜를 긁어서 문자를 새로 보관 이라는 변수에 넣어준다.
이런식으로 쓴다
import requests
import pandas as pd
from bs4 import BeautifulSoup
import time
# 검색할 키워드 설정
search_keyword = "키워드"
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Whale/3.22.205.18 Safari/537.36'
}
# 검색 결과 페이지 수 설정
num_pages = 3
for page in range(1, num_pages + 1):
#url = f"https://search.shopping.naver.com/search/all?query={search_keyword}&pagingIndex={page}"
response = requests.get(url, headers = headers)
soup = BeautifulSoup(response.text, "html.parser")
products = soup.find_all("div", class_="클래스명") #div에 클래스명이라는 class를 가진 구간을 긁어 products에 넣는다
for product in products: #반복문
title = product.find("a", class_="클래스명").text.strip() #a태그의 class가 클래스명인 것의 글짜를 긁어서 title에 넣는다
print(title) #긁어온걸 출력해본다
이렇게 크롤링한 데이터는
엑셀, PDF, 워드, 등으로 출력도 가능하다.
라이브러리가 참 편리하게 잘 준비가 되어 있다!!!
728x90
반응형
'교육 > PYTHON' 카테고리의 다른 글
| [파이썬]특정시간 후 PC가 자동으로 종료되도록 해보자 (47) | 2023.10.04 |
|---|---|
| [PYTHON] 크롬 브라우저 아이디/비밀번호 자동 입력 (2) | 2022.10.12 |