-
- EXPLORE
-
-
-
-
-
-
-
-
How to Use Selenium to Web-Scrape with Example
Here's an example of how to use Selenium to web-scrape a website:
Let's say you want to scrape the top 10 posts from the front page of Reddit. You could do this using Selenium in Python with the following code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
# start a new browser session
driver = webdriver.Chrome()
# navigate to Reddit's front page
driver.get("https://www.reddit.com")
# find the top 10 post titles
post_titles = driver.find_elements_by_class_name("title")
# print each title
for post_title in post_titles[:10]:
print(post_title.text)
# close the browser session
driver.quit()
This code uses the webdriver
module to start a new Chrome browser session. It then navigates to Reddit's front page using the get
method. Next, it finds all the post titles on the page using the find_elements_by_class_name
method, which returns a list of WebElement
objects representing the post titles. Finally, it loops through the first 10 post titles and prints their text using the text
attribute of each WebElement
.
Selenium Training in Chennai is a leading provider of Selenium training. They offer up-to-date and practical training programs so that you can get certified. If you are looking for a job, or if you want to move up the ranks, then this course will be perfect for you. With Selenium Training in Chennai‘s certification program, your skillset will be top notch and you will be able to land the position you deserve in your career field.
You could modify this code to extract other data from the Reddit front page, such as the URLs of the top 10 posts, or the number of upvotes each post has received. The key is to use the various methods and properties provided by Selenium to locate and interact with the relevant web elements on the page.
- Art
- Causes
- Crafts
- Dance
- Drinks
- Film
- Fitness
- Food
- Games
- Gardening
- Health
- Home
- Literature
- Music
- Networking
- Other
- Party
- Religion
- Shopping
- Sports
- Theater
- Wellness