Monthly Archives: June 2016

Automation With Python Python Codes To Create Dropbox Apps

- - Python, Tutorials
As promised in the article earlier on Automate DropBox Signups using python, I have come up with an article along with the codes to create an app and fetch the API keys for it which then allows us to access the files in dropbox. Well, again we stick to selenium module for an ease. In the last article, I’ve explained a python script to automate the signups for dropbox. Now that we have enough cloud space in different accounts. We now need to access the files in those spaces so we can use it as a file server. DropBox provides a feature to create apps on dropbox and gives API keys to hence access the files in the account. Since we’ve got multiple dropbox accounts we would stick towards automating the procedure to get the api key for accessing the files.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Firefox()
browser.get("https://dropbox.com/login")
list_of_inputs = browser.find_elements_by_xpath("//div/input[starts-with(@id,'pyxl')]")
list_of_inputs[0].send_keys("email@domain.com")
list_of_inputs[1].send_keys("password")
sign_in = browser.find_elements_by_xpath("//*[contains(text(),'Sign in')]")
sign_in[len(sign_in)-1].click()
time.sleep(10)
browser.get("https://dropbox.com/developers/apps/create")
time.sleep(3)
type_of_app = browser.find_elements_by_xpath("//*[contains(text(),'Dropbox API app')]")
type_of_app[0].click()
file_access = browser.find_elements_by_xpath("//*[contains(text(),'My app needs access to files already on Dropbox.')]")
file_access[0].click()
type_of_file_access = browser.find_elements_by_xpath("//*[contains(text(),'My app needs access to a user')]")
type_of_file_access[0].click()
app_name = browser.find_element_by_name("name")
app_name.send_keys("appnamewhichisuniquelolo")
create_app = browser.find_elements_by_xpath("//*[contains(text(),'Create app')]")
create_app[1].click()
time.sleep(7)
app_key_item = browser.find_element_by_class_name("app-key")
app_key = str(app_key_item.get_attribute('innerHTML'))
app_secret_item = browser.find_element_by_class_name("app-secret")
app_secret = app_secret_item.get_attribute('data-app-secret')
print app_key, app_secret

General Idea of Automation

The general idea for automation is to mimic the manual workflow and put it in a loop or assign a cron job(it’s kind of same thing but not really). For creating apps on dropbox, I did the same thing. The codes are self-explanatory. We’ve used selenium and time module throughout our program. We use selenium for initiating as well as interacting with the browser. You can see, we’ve used time.time(time_in_seconds) method from time module. Depending on the speed of the internet, we need to set this up. Failing to do so will lead the program to misbehave since it will start looking for some element even when the page hasn’t been completely loaded. We fuel our program with the varieties of methods selenium provides. The above codes however shows only the procedure to create an app for a single account and print the api keys. You should loop over some file containing email id’s and password and save the api keys to some file in real usage. Hint: Place a loop over the codes and once done with getting api keys, logout from the current account.

Do comment below how you felt the article was. Any queries, please mention below.

Announcement

I’ve joined twitter @bbhishan

Google Search Using Selenium And Python – Selenium Python Basics

- - Applications, Python, Tutorials

The intentions of this blog is to show through examples some of the most common methods of selenium. Selenium is a library used for automated browser testing. However, in this post we will discuss about using selenium module in python to make a google search. The post breaks down into various blocks explaining on how to open a url in the browser via selenium python, search presence of a url in a page, click links present in a page. These are the necessities to get started with selenium.

Prerequisites
  1. Python
  2. selenium module in Python
  3. Chrome driver (http://chromedriver.chromium.org/downloads)
Installation of selenium through pip in both Linux and Windows

pip install selenium

Google search using selenium python
from selenium import webdriver

search_query = input("Enter the search query")
search_query = search_query.replace(' ', '+') #structuring our search query for search url.
executable_path = "/path/to/chromedriver"
browser = webdriver.Chrome(executable_path=executable_path)


for i in range(20):
    browser.get("https://www.google.com/search?q=" + search_query + "&start=" + str(10 * i))
    matched_elements = browser.find_elements_by_xpath('//a[starts-with(@href, "https://www.thetaranights.com")]')
    if matched_elements:
        matched_elements[0].click()
        break
1. Import statements (Line 1)

It is the import statements that is required for initiating a browser later in our program and passing url parameters to the address bar in the browser. It can be thought of as a driver for the browser. We use various methods on an instance from webdriver.Chrome() instance to control interaction with the browser.

2. Get query for google search (Line 3 and 4)

Here, we are taking a query for the google search via input() in Python3(raw_input() for Python2). Here is an example url for a google search which requires the spaces between the words to be replaced by “+” , an additional parameter start=0 is seen which specifies the search result of page 1. Similarly start=10 gives the search result of page 2.
https://www.google.com/search?q=bhishan+bhandari&start=0“
Hence, after taking the input from the user, we replaces the spaces with +.

3. Instantiate a browser (Line 5)

The statement browser = webdriver.Chrome() opens up a new browser window. We can also customize the browser capabilities such as download location, etc.

4. Opening a url in the browser (Line 9)

For opening a url in the browser, all you need to do is pass the url as an argument to the browser.get method. Remember I’ve given browser.get because we instantiated the browser earlier with browser = webdriver.Chrome(executable_path=executable_path).

5. Searching for a presence of certain url/text in the search result (Line 10 to 15)

The following methods returns the browser elements which match the criteria that the href attribute of the anchor element starts with https://www.thetaranights.com

browser.find_elements_by_xpath('//a[starts-with(@href, "http://www.thetaranights.com"]')

There is also an alternative method find_element_by_xpath for getting the first element that matches the given xpath construct. Then we make a check whether or not any there was at least an element returned from the above statement, which if true we click using click() method on the first element that matched the criteria. This will open the link on the browser. Since the result we are looking for is found and clicked, we exit loop. Else continue searching for the link with the above criteria until 20 pages if not found. You can quit the browser using browser.quit() method.

We generally covered how to open a browser, search for link in the body of the page and click the link. You may also like to read my article on how to login to a website using selenium python.

How To Split And Merge Pdf Documents

- - Uncategorized

Not the type of posts I usually produce. A promotional review of a tool.

Everyone knows that PDF files are hard to work with. Apart from figuring out how to convert PDF documents, oftentimes we’re also trying to put together the best PDF document possible from other content.

But when those content sources are already in the PDF format, it can seem like an uphill battle just to get the content separated. More often than not, we need to figure out how to manipulate PDF documents at the page level.

Sometimes we may need to rework a PDF document by adding or removing a few pages. Manipulating PDF documents like this can seem intimidating at first.

If you have legal PDF documents your concern may be preserving the integrity of the PDF pages, or if you’re working with reports, you may be worried about deleting the original PDF pages for good.

Normally, you’d have to convert the entire PDF file into a Microsoft Word document, delete or insert the pages accordingly, and then convert it back to PDF. But there’s an easier way to do it.

With a tool like Able2Extract 10 from Investintech.com, you can merge and split your PDFs as easily as you can select a page. This latest version comes with features for converting, creating and editing PDF documents.

Under the latter category, Able2Extract 10 has added the ability to merge and split PDF files. It does this by letting you extract or insert PDF pages to your currently opened PDF document.

For instance, if you have blank pages or full page images in a PDF you’d like to remove or collect into one file, you can extract them into a completely separate file. Or, if you’d like to add some supplementary information to compliment your existing PDF content, you can add them page by page into an existing PDF document easily.

Here’s a look at how this can be done with Able2Extract 10’s latest PDF splitting and merging feature.

To Merge PDFs:

1. Open the PDF you wish to add pages to in Able2Extract 10.

2. Click on Edit from the toolbar

 

3. From the side editing panel, select Insert From PDF

4. From the dialog that appears, select your PDF file from which you want to insert pages from. Click on Open.

How To Download Udemy Videos Script For Downloading Udemy Videos

- - Web

This short post will walk through simple steps to download udemy videos which are not downloadable from the website. Most of the paid udemy courses as well as some free courses are unavailable to download at udemy.com . I personally have around 200 courses in my account. Now most of these courses were not available for download. Fortunately I found a python script on the internet which solved my problem easily. It is udemy-dl.

Installation of udemydl
  1. Install Python for your operating system, preferably Python3
  2. Install pip (python package installer) if not already installed with Python.
  3. Install udemy-dl via
    pip install udemy-dl

The udemy-dl installed from pip doesn’t work. Therefore we need to download it from github. Follow the steps below

    • Clone the repository or download the repository as a zip and extract it from https://github.com/r0oth3x49/udemy-dl.
    • Navigate to the folder through command line/terminal. cd udemydlfolderpath
    • Install dependencies of udemy-dlpip install -r requirements.txt

The above command will install all the prerequisites for udemy-dl to run correctly.

Using udemydl to download courses from Udemy account

Open command line/terminal and navigate to the udemy-dl folder that you downloaded and enter the command python udemy-dl.py link_to_the_course_on_website

python udemy-dl.py https://www.udemy.com/COURSE_NAME

Example

python udemy-dl.py https://www.udemy.com/learn-how-to-deploy-docker-applications-to-production

You will be asked for the username and password for your udemy account. Once credentials are entered, it begins the download.

Thanks for reading