Grab Whois Information And Write To Google Spreadsheet

- - Uncategorized

Hello Guys, Here I am with yet another program that can benefit you and many search engine optimizers. By the end of this read you will be able to write a program to extract the whois information of a number of domains stored in a text file and write the information about the domain in a google spreadsheet which has now been a medium to share data and findings online.

As a Search Engine Optimizer, you need to keep track a number of websites including your competitions. Here I offer you a simple python program to keep track of. On the other hand if you are not a SEO expert like myself, you can still use this script to track various websites you are used to.

Prerequisites before beginning to code

We are going to have two files one of which is a .py file where we code our program. The other is a text file with .txt extention where we store the domain names we want to find whois information for. The text file must contain a domian name in a format www.domain.com one per each line.

Next, we need to create a google spreadsheet where we intend to write the whois information so we can share with others. Direct your browser to https://docs.google.com/spreadsheets/ and create a new spreadsheet named “Whois Info”. Once done, create three rows namely “website”, “whoisinformation” and “date”. The name of the domain name will be under the row website, the whois information will be under the row whoisinformation and the date we queried the whois information will remain under the row date.

Python code to extract whois information and write to google spreadsheet

from bs4 import BeautifulSoup
from urllib2 import urlopen
import gdata.spreadsheet.service
import datetime
rowdict = {}
rowdict['date'] = str(datetime.date.today())
spread_sheet_id = '1zE8Qe8wmC271hG2uW4XE68btUks79xX0OG-O4KDl_Mo'
worksheet_id = 'od6'
client = gdata.spreadsheet.service.SpreadsheetsService()
client.debug = True
client.email = "email@domain.com"
client.password = 'password'
client.source = 'whoisinfo'
client.ProgrammaticLogin()
with open('websitesforwhois.txt') as f:
    for line in f:
        soup = BeautifulSoup(urlopen("http://www.checkdomain.com/cgi-bin/checkdomain.pl?domain=" + str(line)).read())
        for pre in soup.find_all("pre"):
            whois_info = str(pre.string)
        #print whois_info
        rowdict['website'] = str(line)
        rowdict['whoisinformation'] = whois_info
        client.InsertRow(rowdict,spread_sheet_id, worksheet_id)

1. Line 1 to 4

These are the import statements. We use BeautifulSoup to make a soup object out of a url response. Urlopen to get the response of a url. Gdata to access the google spreadsheet. Datetime to get the current system time.

2. Line 5 and 6

In our program, we require to access the google spreadsheet and write to it hence we are using gdata module. Now in order to write to spreadsheet, we need to pass the data as a dictionary or generally known as json which has data as a key:value pair. Rowdict is a variable storing the data to pass to google spreadsheet. On line 6, we store the current date to the key “date” which if you remember is a row at our spreadsheet.

3. Line 7 to 14

Line 7 to 14 is a procedure to connect/access a specific google spreadsheet. We require spread_sheet_id and worksheet_id. Take a look to the url of your spreadsheet. The url looks something like this one

https://docs.google.com/spreadsheets/d/1VbNph0TfFetKLU8hphrEyuNXlJ-7m628p8Sbu82o8lU/edit#gid=0

The spreadsheet id(mentioned earlier) is present in the url. “1VbNph0TfFetKLU8hphrEyuNXlJ-7m628p8Sbu82o8lU” in the above url is the spreadsheet id we need. By default the worksheet id is ‘od6‘.

On line 13 is the client.source assigned to string ‘whoisinfo’. This is the file name or the spreadsheet name. Remember we named our spreadsheet “Whois Info”. The client.source is the spreadsheet name which is written in small alphabets excluding white spaces.

4. Line 15 to 16

Line 15 opens the text file where we’ve stored the names of the domain. Line 16 helps iterate through each lines in the file. At each iteration, the domain name at each line is stored to variable line.

5 Line 17

On line 17, we query the page giving the whois information for us and make a soup object out of it by invoking the BeautifulSoup method over the url response. The reason we are making a soup object is that we can access required data via tags and the data we need is inside a <pre></pre> tag.

6 Line 18 to 19

Now we know that there is only one “pre” tag in the soup element. We therefore iterate to find a pre tag and store the information inside of the pre tag to a variable whois_info.

7 Line 21 to 23

On line 21, we are assigning the domain name to the key “website” of the dictionary rowdict. On line 22, we are assigning the whois information stored in the variable whois_info to the key “whoisinformation” of the dictionary rowdict. Note that the key of the dictionary must match to the row name in our spreadsheet. Line 23 pushes the dictionary to the google spreadsheet and writes to it. The iteration goes until the domain names at a text file is finished.

If you have any questions/confusions regarding the article or code, please mention below in comments so we can discuss. Thanks for reading :)

Gui Automation With Python

- - Uncategorized

Hello Readers. It has been a bit longer delay in publishing my article. However today I will present to my awesome readers, an introduction to a GUI automation module in python (I.e pyautogui). Pyautogui is a GUI automation module for python2 and python3 which provides methods for controlling mouse and keystrokes. This decent module can be used to create bots to automate the repetitive tasks while you can enjoy your coffee.

Pyautogui can do anything a human user sitting at the computer can do, except spill coffee on the keyboard” says the geek responsible for this cool module.

Follow the link below to have pyautogui installed on your machine.

https://pyautogui.readthedocs.org/en/latest/install.html

With no further iteration about the introduction, I would like to present few basics about the module.

1. Locating coordinates of the mouse cursor.

>>> import pyautogui

>>> pyautogui.position()

(850, 504)

>>>

It returns the current x and y coordinate of the mouse cursor position. In a computer screen the left top point is the origin or (0,0)

2. Moving the mouse cursor

>>> pyautogui.moveTo(10,10)

>>> pyautogui.moveTo(10,10,duration=1)

The moveTo function takes x-coordinate and y-coordinate as parameters while duration can be passed as the third parameter which is optional used to specify the amount of time in seconds to reach to the specified coordinate. The second one is humanly approach while the first is an instant movement of cursor.

3. Clicking

>>> pyautogui.click(80,80)

>>> pyautogui.doubleClick(80,80)

>>> pyautogui.rightClick(80,80)

Clicking on a certain coordinate on the screen is possible via click method while it also provides doubleClick, rightClick methods taking parameter x-coordinate and y-coordinate in all cases.

4. Keystrokes

For typing, we will first need to locate an appropriate type area. Therefore, you might want to use this method after click on some coordinate which is writable. You can use two or more statements to run simultaneously one after another by separating each statement by semicolon. For instance, I’ve specified the coordinates of the url bar on my browser and then typed my name on it via following commands/statements

>>> pyautogui.click(50,80);pyautogui.typewrite(“Bhishan”)

>>> pyautogui.click(50,80);pyautogui.typewrite(“Bhishan”, interval=0.2)

We can pass an optional parameter interval in seconds to specify the time in seconds between each letter or keystroke.

5. Hot Key

The hotkey method can be used in cases we need to press two or more keys at the same time. A handy example is Ctrl + S to save a file or Ctrl + Shift + q to quit

>>> pyautogui.hotkey(‘Ctrl’,’Shift’,’q’)

You can see all the possible mapping keys of the keystrokes via this method

>>> pyautogui.KEYBOARD_KEYS

Well that’s enough to get you started and good at GUI automation via pyautogui. Below is a bot I have made using the module to automate a boring task for myself. Iterating the story behind the need for the bot. I am a fourth semester CS undergrad student(I mean lazy student). I never take notes in any of the classes I attend. At the time of exams, I rely on the photos of my friend’s notes which they send me. As always I got the photos but this time all the pictures were at landscape mode by some chance(near about 100-110 images). It would be kind of distracting to rotate each image to read it. So I wrote some 7-8 lines of code to make a bot that would open each image file rotate it and save it while I have my dinner. I’ve used time module along with pyautogui to keep some time gap between the statements.

import pyautogui
import time
pyautogui.click(450,450);pyautogui.typewrite('graphicsnotes');pyautogui.press('enter')
time.sleep(2)
for i in range(107):
  pyautogui.press('right');pyautogui.press('enter')
  pyautogui.hotkey('ctrl','r');pyautogui.hotkey('ctrl','s')
  time.sleep(2)
  pyautogui.press('esc')
  time.sleep(2)
  time.sleep(2)


The concept is to click anywhere on the desktop screen. I choose some random coordinate (450,450). Then typing folder name to locate the folder followed by enter to open it. Then iteratively clicking right to select the image file, opening the image file by pressing enter, followed by hotkey ‘Ctrl’ + ‘r’ to rotate the image clockwise then ‘Ctrl’ + ‘s’ to save it. Finally pressing esc to close the file and repeating the process to go to the next image file. I had total of 107 images so I’ve iterated 107 times in my program to reach up to all the image files. Tell me how you felt the article was in the comments section below so I can come up with a cool set of articles for the next week. Till then, happy automation with pyautogui :)

Here read the docs https://pyautogui.readthedocs.org

Python Selenium Time Tkinter Pyvirtualdisplay

- - Uncategorized

Making use of libraries available and building something useful has always been my favorite. However, I haven’t yet been involved in making/contributing to one. I don’t even know the procedure to be involved in building libraries. My latest program also makes use of various libraries such as selenium, time, tkinter. In a nutshell, the program fetches jokes from www.laughfactory.com and displays a message box with a joke in a timely interval. I usually run the program while I am at work(internship). I’ve set 10 minutes interval between the jokes to be shown. By the way, I used time module and it’s time.sleep(time_in_seconds) method for dealing with intervals between each message box to appear. The program isn’t full fledged and you can make additions to it on your own for customization according to your need. The earlier statement, I guess seemed very professional for a small program. Please bear with me ????

 

I made use of selenium module along with pyvirtualdisplay to fetch the jokes and mute the display of the browser, since it would be distracting. The program only fetches 20 latest jokes from laughfactory.com and periodically displays one at a time in a messagebox. A little humor is always good at work. Anyway that’s my personal thought you may disagree to. I used tkinter module for displaying the message box. That’s all about the program. Have a look at the codes. Thank god, the import statements < program statements.

Python codes to get timely jokes

import tkinter
import tkinter.messagebox as mbox
from time import sleep
from selenium import webdriver
from pyvirtualdisplay import Display

display = Display(visible=0, size=(800, 600))
display.start()
browser = webdriver.Firefox()
browser.get("http://www.laughfactory.com/jokes/latest-jokes")

sleep(40)
#print("finding jokes")
jokes = [str(joke.text) for joke in browser.find_elements_by_xpath("//div/p[starts-with(@id,'joke_')]")]

#print("found")
browser.quit()
display.stop()

window = tkinter.Tk()
window.wm_withdraw()

for joke in jokes:
    sleep(600)
    mbox.showinfo('Bored? Enjoy the JOk3!', joke)

Thank you for spending your time here. I have always tried to make my articles short and maybe now succeeded. You can utilize the saved time for another read in this blog, may be :p .Tell me how you felt the article was in the comments section below. This keeps me motivated to publish good content.

Login To A Website Using Selenium Python Python Selenium Example

- - Uncategorized

Through this post, we will discuss about an important and most frequently used module available in python and many other languages which is used for automated browser testing. In this post, we will only touch a little of the selenium’s actual capability.

I researched about selenium when I received an email from one of the readers on this website who was unable to login to a desired website using mechanize which I had explained in the post named Login to any website using mechanize module in python.

Digging into the problem, I finally found out that it was caused by the dynamically added input fields. Some of the websites dynamically add the login fields based on the user’s country or some other criteria. In such cases, using mechanize is not applicable since it has no support for javascript driven websites at current time.

Selenium provides an excellent surfing of the browser automatically. Below is the codes which I suggested my reader to login to his desired website. In this article, I’m using that very website as an example.

Selenium Python Example to login to a website

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Firefox()
browser.get("https://lbjcrs.ust.hk/primo/authen.php") 
time.sleep(10)
username = browser.find_element_by_id("extpatid")
password = browser.find_element_by_id("extpatpw")
username.send_keys("username")
password.send_keys("password")
login_attempt = browser.find_element_by_xpath("//*[@type='submit']")
login_attempt.submit()

Explanation of python codes to login to website using selenium

1. Line 1 to 3

These are the import statements. We import webdriver from selenium which is used to instantiate a browser. The other line imports Keys which is used to pass the username and password to the appropriate fields in the website running on a browser instantiated using selenium webdriver. Next we have imported time which I’ve used to create a delay after opening a url in the browser so that the javascript is loaded competely with all the required input fields.

2. Line 4 to 6

In line 4, we instantiate a Firefox webbrowser. This line will open a browser.
Line 5 is used to open a specific url in the webbrowser we’ve opened. This is the login page which has javascript used to get the input fields dynamically loaded. Hence in line 6 we have a time.sleep(time_in_seconds), which means we want to wait for some seconds specified in the parenthesis before the another line is executed. This is necessary because the javascript requires a while to be loaded so that the input fields are placed well.

3. Line 7 to 10

For placing the username and password to the appropriate fields, we need a prior knowledge of the input field. Right click on the field and click on inspect element to reveal it’s id. Once we have the id for both username and password fields, we can proceed to programatically adding the values to these fields. In line 7 and 8, we’re assigning variables to hold the fields(username and password). In line 9 and 10, we’re using the methods send_keys over these fields to pass the username and passwords respectively. By the end of line 10 the username and password will be already set in the browser.

4. Line 11

In line 11, we’re using the find_element_by_xpath to find a submit button on the webpage being displayed in the webbrowser. We assign this find to login_attempt.

5. Line 12

Finally, we trigger a click event on the submit button which will then login to the website for us.
This is all it takes to login to a website using javascript to load the input fields dynamically with the help of selenium python. If you have any questions regarding the codes/article, please write below so we can discuss. Happy coding :)

Compiling My Year With An Interpreter – Python Programming

- - Tutorials
To my dear readers, without whom this blog would have been a dead planet. Thanks for spending your valuable time here. I wish you a Happy New Year. 2015 is gone with almost an end to my sophomore year. I wouldn’t say year 2015 was any new to me than the others. Few achievements, life lessons and never ending learning curve. My very first article for 2016 will sum up my pythonic experience from 2015. I used some python throughout this semester/year to ease my life. Bear with novice’s automations :)

1. Enroll in 100% off courses at Udemy automatically.

from json import loads
from bs4 import BeautifulSoup
import mechanize
api_key = "8def4868-509c-4f34-8667-f28684483810%3AS7obmNY1SsOfHLhP%2Fft6Z%2Fwc46x8B2W3BaHpa5aK2vJwy8VSTHvaPVuUpSLimHkn%2BLqSjT6NERzxqdvQ%2BpQfYA%3D%3D"
growth_coupon_url = "https://api.import.io/store/data/a5ef05a9-784e-410c-9f84-51e1e8ff413c/_query?input/webpage/url=http%3A%2F%2Fgrowthcoupon.com%2Fcoupon-category%2F100-discount%2F&_user=8def4868-509c-4f34-8667-f28684483810&_apikey=" + api_key
br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [("User-agent","Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13")]
sign_in = br.open("https://www.udemy.com/join/login-popup/")
br.select_form(nr=3)
br["email"] = "email@domain.com"
br["password"] = "password"
logged_in = br.submit()

growth_coupon = br.open(growth_coupon_url)
json_obj = loads(growth_coupon.read())

for course_link in json_obj["results"]:
    try:
        course_page = br.open(str(course_link["couponcode_link"]))
        soup = BeautifulSoup(course_page)
        for link in soup.find_all("a"):
            req_link = link.get('href')
            if 'https://www.udemy.com/payment/checkout' in str(req_link):
                print req_link
                br.open(str(req_link))
                print "success"
                break
    except (mechanize.HTTPError,mechanize.URLError) as e:
        print e.code

This has been my favorite automation throughout the semester. The program checks growthcoupon.com for 100% off coupon codes for paid courses offered at Udemy and enrolls me to those courses. I have uploaded the program to pythonanywhere.com which allows me to have the script run daily(for free accounts) without me having to worry about it. At this time I have over 800 courses at my Udemy account, each courses on an average costs 75$.

2. Conversation between two cleverbots

import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

browser = webdriver.Firefox()
browser2 = webdriver.Firefox()
browser.get("http://cleverbot.com")
browser2.get("http://cleverbot.com")

time.sleep(15)

input_for_bot = browser.find_element_by_name("stimulus")
input_for_bot2 = browser2.find_element_by_name("stimulus")


output_from_bot = ""
output_from_bot2 = "Hey, friend, what's up"

for i in range(0, 200):    
    input_for_bot.send_keys(output_from_bot2)
    input_for_bot.send_keys(Keys.RETURN)
    output_from_bot = ""
    time.sleep(5)
    for elem in browser.find_elements_by_xpath('.//span[@class="bot"]'):
        output_from_bot = elem.text
    input_for_bot2.send_keys(output_from_bot)
    input_for_bot2.send_keys(Keys.RETURN)
    output_from_bot2 = ""
    time.sleep(5)
    for elem in browser2.find_elements_by_xpath('.//span[@class="bot"]'):
        output_from_bot2 = elem.text
    
    

This semester, I had to take Cognitive Science course. I enjoyed it. I had this assignment where I had to submit a page of conversation with cleverbot (www.cleverbot.com). I submitted the assignment and later decided to bridge a conversation between two cleverbots. I used selenium module in python to have this done. It was great and kind of felt like an achievement.

3. Using pyautogui before my exams

import pyautogui
import time
pyautogui.click(450,450);pyautogui.typewrite('graphicsnotes');pyautogui.press('enter')
time.sleep(2)
for i in range(107):
  pyautogui.press('right');pyautogui.press('enter')
  pyautogui.hotkey('ctrl','r');pyautogui.hotkey('ctrl','s')
  time.sleep(2)
  pyautogui.press('esc')
  time.sleep(2)

Being lazy to copy notes at class, I had to rely on photos of the notes sent by my friend. I discovered that the photos were all in landscape and were 107 pictures. I had come across pyautogui in A.I Sweigart’s course at Udemy and quickly wrote some 5-7 lines of code to open the picture and rotate the image and save it while I had my dinner. By the way I had no clue I had been enrolled in A.I Sweigart’s course until I opened my account to check which courses I had. All thanks to pythonanywhere.com which runs my program to enroll to Udemy courses on a daily basis.

4. Automate signups and making apps at Dropbox for unlimited space

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

browser= webdriver.Firefox()
browser.get("https://dropbox.com/register")

list_of_inputs = browser.find_elements_by_xpath("//div/input[starts-with(@id, 'pyxl')]")

list_of_inputs[7].send_keys("first name")

list_of_inputs[8].send_keys("last name")

list_of_inputs[9].send_keys("email")

list_of_inputs[10].send_keys("password")

list_of_inputs[11].click()

password.send_keys(Keys.RETURN)

I had been involved with some people from IIT for some app ideas. We needed cloud space and agreed to use Dropbox. I had been given a bunch of emails with common password. I wrote a program in python to do the signups and later wrote a program to make apps at dropbox and get the API keys and secret keys to access the files in there. Unfortunately the project never continued for some reasons.

Overall, I had a good year but most of the time was spent at college and internship (doing mobile apps using Apache Cordova). For the new year, I will talk to my manager about switching to some python projects. My new year resolution would be to “write more codes and continue blogging about it.” Well that’s all I can think for now.

I would want some suggestions for my new year resolution. Do comment below. Once again A Happy New Year :)

Is It A WordPress Website Checker Script In Python

- - Uncategorized

By the end of this read, you will be able to code a program that will check for a number of domain names stored in a text file to verify whether or not a website is powered by wordpress. In this program we will write the result for each website in a google spreadsheet for later use but you can apply certain code if a website is wordpress powered at the mean time.

Python script to check if a bunch website is wordpress powered

Below is the program written in python programming language which reads a text file storing the names of the domain one per each line and checks if it a wordpress website and writes the status in a google spreadsheet.

from bs4 import BeautifulSoup
import mechanize
import gdata.spreadsheet.service
import datetime
rowdict = {}
rowdict['date'] = str(datetime.date.today())
spread_sheet_id = '1mvebf95F5c_QUCg4Oep_oRkDRe40QRDzVfAzt29Y_QA'
worksheet_id = 'od6'
client = gdata.spreadsheet.service.SpreadsheetsService()
client.debug = True
client.email = "email@domain.com"
client.password = 'password'
client.source = 'iswordpress'
client.ProgrammaticLogin()
br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [("User-agent","Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13")]
base_url = br.open("http://www.isitwp.com/")
with open('websitesforwpcheck.txt') as f:
    for line in f:
        rowdict['website'] = str(line)
        br.select_form(nr=0)
        br["q"] = str(line)
        isitwp_response = br.submit()
        isitwp_response = isitwp_response.read()
        if "Good news everyone" in a:
            rowdict['iswordpresswebsite'] = "yes"
        else:
            rowdict['iswordpresswebsite'] = "no"
        client.InsertRow(rowdict,spread_sheet_id, worksheet_id)

Isitwordpresswebsite code explanation

Before beginning , we need to create a spreadsheet and make three rows with names website, date, isitwordpresswebsite

1. Line 1 to 4

These are the import statements for the libraries we will use in our program. We use BeautifulSoup to convert a response from a request as a soup object. Mechanize is used in order to skip the length of code for sessions and cookies. Gdata is used to connect to our google account in order to access the spreadsheet we want to work with. Datetime is used to get the current date at the time of script run.

2. Line 5

In line 5, we create an empty dictionary where we later create a key:value pairs for date, website name and status of a website(is it wordpress or not). Also google spreadsheet accepts a dictionary item/json which is then written to the spreadsheet. In line 6 we store the current date at the time of script run to a key “date” which is later in our program pushed to the google spreadsheet.

3. Line 7 to 14

Now, before proceeding forward, we need to create a google spreadsheet. Now another step is to take a look at the url of the spreadsheet we created. We need the spreadsheet id to access the spreadsheet via our program. Below is a screenshot of how to get the spreadsheet id. The worksheet id is ‘od6‘ by default. Line 9 to 14 gets us logged in to our google account and accesses the spreadsheet we want to work with.

4. Line 15 to 18

In line 15, we use the mechanize module’s method to initiate a browser. Line 16 states to ignore the robots.txt file. Line 17 adds a user agent to the browser. Line 18 lets us open the website “isitwp.com” which we will be using to check if a website is wordpress powered or not.

5. Line 19

Line 19 allows us to open the text file where we have the names of the domain(one per each line).

6. Line 20 to 21

In line 20, we iterate through the lines of the file where we have stored domain names. At each iteration, the domain name at the current line is stored to the variable line. In line 21, we create a key:value pair where we store the domain name at each iteration to the key “website.

7. Line 22 to 30

Line 22 is a statement to select the form of index 0 i.e the first form present in the website isitwp.com. Now value of the name attribute of the seachbox is “q” which is where we store the name of the domain before submitting the form. Line 24 submits the form after passing the value of domain name in the br[‘q’] and stores the response in a variable isitwp_response. Line 25 is the complete webpage response stored in variable isitwp_response. In line 26 we check if “Good news everyone” substring is present in the response which means the website is powered by wordpress else it is not. Line 27 then makes a key:value pair where the key “iswordpresswebsite” is given value “yes” if the condition on line 26 passed and “no” if the condition failed. Now remember the key of the dictionary rowdict must be the name of the row in our spreadsheet.

This way we can test if a website is wordpress powered for a number of website names stored in a text file(one at each line). Thanks for reading :) . If you have any questions regarding the article or the code, comment below so we can discuss on that part.

Cryptography With An Interpreter

- - Uncategorized

Hey Guys, it’s been a long time since I published my last article. Apologies for the delay. Anyway, straight into the topic, Cryptography. Well, cryptography with python. This semester(V of Bsc CS) I choose Cryptography as an elective over Neural Network and I am enjoying it. So far I have learned Ceasar Cipher, Playfair Cipher, Vigenere Cipher, Vernom Cipher, Hill Cipher and Rail Fence Cipher. It’s just the beginning anyway. I have implemented these learnings via python programming language. Here is link to the github repository containing the implementations. The repo will be updated as I go further into learning throughout the semester.

Quick walk

Cryptography is the science that is used to hide the information. Mathematically defined as a tuple of five members (M, C, E, D, K) where,

M → Message (Plain Text)

C → Cipher Text

K → Set of Keys

E → Encryption Algorithm E: M*K → C

D → Decryption Algorithm D: C*K → M

On the other hand, cryptanalysis is the study of cipher systems with the view of finding the weaknesses in them that will enable us to retrieve the plain text.

Furthermore, ciphers can be classified into various types based on their properties. Well cipher in general is an algorithm to perform encryption and decryption. We could group Substitution and Transposition into Historical ciphers. Substitution cipher may further be monoalphabetic or polyalphabetic. Modern ciphers are either based on input data or based on key. Stream cipher and Block cipher are the types of modern cipher based on input data. Based on key there are symmetric(private) and asymmetric(public) ciphers.

Following are one of the implementations of ciphers I’ve learned so far. For complete package, follow this link to the repository.

Vernom Cipher

#sample plain text : hello
#sample key : axhjb

def make_chunks(text, text_length, key_length):
    for i in range(0, text_length, key_length):
        yield text[i : i + key_length]


def encryptdecrypt(cipher_generator):
    final_text = ""
    for item in cipher_generator:
        for i in range(0, len(item)):
            final_text += alphabets[alphabets.index(key[i]) ^ alphabets.index(item[i])]
    return final_text

alphabets = "abcdefghijklmnopqrstuvwxyz"

plain_text = raw_input("Enter the plain text: ")

key = raw_input("Enter the key: ")

plain_text = plain_text.replace(" ", "")

p_generator = make_chunks(plain_text, len(plain_text), len(key))

cipher_text = encryptdecrypt(p_generator)

print "The cipher text is : ", cipher_text

c_generator = make_chunks(cipher_text, len(cipher_text), len(key))

decrypted_text = encryptdecrypt(c_generator)

print "The decrypted text is : ", decrypted_text


Tell me how you felt the article was in the comments section below or shoot me a message at bbhishan@gmail.com . And there is always thanks for reading. Cheers ????

Enroll In 100 Off Courses At Udemy Automatically Python Codes To Get Paid Courses

- - Uncategorized

Udemy is a teaching and learning platform with loads of courses in various categories. Now very often different coupon codes are available for purchasing courses in minimal amount or with a 100% discount. Various websites serve these coupon codes. One of those websites which I rely on is GrowthCoupon.com

Now, I am not writing a review of 100% off coupon providers. Through this post I will explain my code which I am using to extract the 100% off coupon codes from growthcoupon.com and then get those courses automatically. I have automated my code so that I do not need to worry about new coupon codes available and can save my time. The below code enrolls you in 10 latest 100% off courses available at growthcoupon.com when run a single time. You may wish to automate the script every hour or so.

Get 100%off Udemy courses automatically using python

from json import loads
from bs4 import BeautifulSoup
import mechanize
api_key = "8def4868-509c-4f34-8667-f28684483810%3AS7obmNY1SsOfHLhP%2Fft6Z%2Fwc46x8B2W3BaHpa5aK2vJwy8VSTHvaPVuUpSLimHkn%2BLqSjT6NERzxqdvQ%2BpQfYA%3D%3D"
growth_coupon_url = "https://api.import.io/store/data/a5ef05a9-784e-410c-9f84-51e1e8ff413c/_query?input/webpage/url=http%3A%2F%2Fgrowthcoupon.com%2Fcoupon-category%2F100-discount%2F&_user=8def4868-509c-4f34-8667-f28684483810&_apikey=" + api_key
br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [("User-agent","Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13")]
sign_in = br.open("https://www.udemy.com/join/login-popup/")
br.select_form(nr=3)
br["email"] = "email@domain.com"
br["password"] = "password"
logged_in = br.submit()

growth_coupon = br.open(growth_coupon_url)
json_obj = loads(growth_coupon.read())

for course_link in json_obj["results"]:
    try:
        course_page = br.open(str(course_link["couponcode_link"]))
        soup = BeautifulSoup(course_page)
        for link in soup.find_all("a"):
            req_link = link.get('href')
            if 'https://www.udemy.com/payment/checkout' in str(req_link):
                print req_link
                br.open(str(req_link))
                print "success"
                break
    except (mechanize.HTTPError,mechanize.URLError) as e:
        print e.code

The above program is a pure python code that extracts 10 latest 100% off coupon codes from GrowthCoupon.com and then enrolls you in those courses automatically.

1. Line 1 to 3

The first three lines are the import statements. In our program, we are using three python libraries. Amongst them, mechanize is used to login to the udemy account. BeautifulSoup is used to get the data on the basis of tags. Here in our program we use BeautifulSoup to get the links in certain page. Json’s loads is used to load the json response.

2. Line 4 and 5

We are using import.io API in order to extract data from growthcoupon. I got to know about this very cool resource in my Programming Synthesis class at my college. Here’s How to get and use import.io API. We store the API in a variable api_key. Then concatenate to the growth_coupon_url which is the standard post request url to get data in json format from growthcoupon.

3. Line 6 to 13

From line 6 to 13 is the procedure to login to a website (udemy in our case). Line 6 initializes a browser. Line 7 says ignore the robots.txt file. Line 8 adds a user agent to the browser. Line 9 opens the login url in the browser we initiated earlier.

The next thing you will need is the form you want to work with. By this I mean this is the login form. All you need to do is go to the username box ->> right click on it->> go to the inspect elements option. Now scroll up until you find the first form tag. In most cases you will find the form name attribute but some of the websites do not have this. If there exists then the value given to the name attribute under the form tag is the thing you need to access the form. Another way to access forms is by their index. The first form is indexed 0. Now in case the form name is not available, you will need to find how many forms are present in the login url(basically most of the websites have only one form because all you want the login page to do is login if authenticated). In this case the form index is 3.

Now you need to know the variable name that is assigned to take the value you enter to the email/username and password section. To get these values inspect element when you are inside the fields email/username and password. Below is a snapshot to give you insights of the variables you want to take care of.

udemyloginform

4. Line 15 and 16

Here on line 15, we are opening the url that gives us the data from growthcoupon in json format. Line 16 loads the url as a json object.

5. Line 18

Our json object is stored in json_obj variable. But the data we need is stored inside an array which is the value for the key “results” Hence we are iterating through this array.

6. Line 20

Now we open the couponcode link which is the value of the key “couponcode_link”. This url is present in each index of the array. On each loop the particular index’s url’s response is stored in variable course_page.

7. Line 21

We then convert the page response to a soup element by invoking the BeautifulSoup method over course_page.

8. Line 22 to 27

Now we want to iterate through each links found in the soup element. The url for enrolling in the udemy course starts with the string “https://www.udemy.com/payment/checkout”. Hence we check if the string is a substring of the link at each iteration. If the condition satisfies, we open that link to enroll ourselves in that course. Well that’s the end of the code that works.

Thanks for reading :) Enjoy ! If you have any questions regarding the codes/post, comment below.

How To Swap Values Without Temporary Variable

- - Web
My dear readers, this article will be invested in explaining about swapping two integers without the use of temporary variable. The reason I am writing this article is because it has been asked several times in an interview for a CS internship/job. In fact, a friend of mine who applied for an internship at a local company was asked the same question. The problem tends to test your analytical skills as well as understanding of programming fundamentals. However, in practice one should never use the following approaches which I will explain why towards the end of the article.

 

Starting with no further delay, there are two approaches to the problem.

1. Using XOR operator

2. Using addition and subtraction operators

Throughout the article we use ‘^’ symbol for representing XOR.

Below is a table showing the results of XOR operation

A B A^B
0 0 0
0 1 1
1 0 1
1 1 0

This concludes that for different value of A and B, the XOR operation gives 1 while it gives 0 for the same value of A and B.

XOR also termed as exclusive-or has following properties

1. Self – inversed

A ^ A = 0

Any value XOR’d with itself gives 0.

2. Identity

A ^ 0 = A

Any value XOR’d with 0 remains unchanged.

3. Associative

A ^ (B ^ C) = (A ^ B) ^ C

The order does not matter.

4. Commutative

A ^ B = B ^ A

How to swap two integers using XOR

It is simple and underlies on the above mentioned properties/principles.

Say for an instance a = 2 and b = 3

The XOR operation is performed between two corresponding bits of the numbers.

In binary 2 = 0010 and 3 = 0011

For instance, while performing a ^ b, the most significant bit of a is XOR’d with the most significant bit of b and so on.

The following three statements swaps values of a and b.

a = a ^ b

b = a ^ b

a = a ^ b

Congratulations, you’ve swapped values in a and b without the use of a temporary variable. How?

What going on in the first statement, lets see it in a table.

a b Final value of a = a ^ b
0 0 0 (from self-inversed)
0 0 0 (from self-inversed)
1 1 0 (from self-inversed)
0 1 1 (from Identity)

At the end of the first statement, the value of a = 0001 = 1

Now the value of a = a ^ b,

What’s going on in the second statement, lets see it in a table.

a = a ^ b b Final value of b = (a ^ b) ^ b
0 0 0 (from self-inversed)
0 0 0 (from self-inversed)
0 1 1 (from Identity)
1 1 0 (from self-inversed)

Voila, we’ve got the initial value of a I.e 2 to be held by b now. It worked but how? It’s nothing but the combination of self-inversed property followed by identity over the integers. For the second statement

b = a ^ b // After the first statement the value of a is equal to a ^ b, so

b = (a ^ b) ^ b

This can be rearranged by associative property and written as

b = a ^ (b ^ b)

We know from self-inversed rule, anything XOR’d with itself gives 0 . So we may now write

b = a ^ (0)

From Identity rule, we know something XOR’d to 0 will leave the number unchanged. Therefore we get

b = a

That’s how we got the initial value of a to be stored to b at the end of second statement.

Moving on to the third statement which is

a = a ^ b // By now the value of a is a ^ b and value of b is the initial value of a

so we may also write

a = a ^ b ^ a

Now this can be written as

a = a ^ a ^ b

Using self-inversed rule

a = 0 ^ b

Using identity rule

a = b

Through Table

A = a ^ b B = a A = a ^ b ^ a
0 0 0
0 0 0
0 1 1
1 0 1

At the end of the computation, we have b = 0010 = 2 and

a = 0011 = 3

Why you shouldn’t use XOR for swapping values?

For an instance, say

int a = 2

int *aptr = &a

int *bptr = &a

In the above set of statements, aptr and bptr are pointers to a. Using what we just learned

*aptr = *aptr ^ *bptr

*bptr = *aptr ^ *bptr

*aptr = *aptr ^ *bptr

Now since both the pointers are pointing to the number in the same location so all the above statements are making change at the same location whose result is data loss.

Swapping using addition and subtraction operators

a = a + b

b = a – b

a = a – b

This is pretty straight forward. Say for an instance

a = 5 and b = 10

Then the result of the first statement will be

a = a + b yields

a = 5 + 10

a = 15 and b is unchanged I.e b = 10

Now when the second statement is run, we have

b = a -b yields

b = 15 – 10

b = 5 (which is the initial value of a. Note that the value of a has already been changed to 15)

Running the third statement

a = a – b

a = 15 – 5

a = 10 (which is the initial value of b. Note that the value of b has already changed to 5)

So what do we have at the end

a = 10 and b = 5 (values swapped)

Why addition and subtraction shouldn’t be used to swap values

Say for an instance the values of a and b are large integers. We are performing addition on the first statement which can lead to overflow since we have large integers.

Should you have any questions, do comment below.

Fill Online Form Using Python

- - Uncategorized

Use mechanize module to fill an online form

By the end of this read, you will be able to fill up an online form using python. For this thing to be done, I would like to introduce you to a module “mechanize”. Mechanize is essentially designed for this purpose. One advantage while working with mechanize is- we need not worry about things like session handling.

Let’s start with github signup form itself.

Base url of the github page

https://www.github.com/

Index of the form to fill-up

Index of a form is the count of forms that exists before it. A webpage may contain several forms. If the form you want to work with is present after two other forms while viewing the source code then the form index would be 2 (2 forms are before it). This is the case for sign up form in github.

Value of name attribute for each text entry box

On each text entry box, right click and inspect elements and look for the value of name attribute.

In case of github ::

  1. For username field

The username text entry area in github’s signup page is completely same to the above text entry area. Right click and inspect element to see the value assigned to the name attribute. All we want is the value assigned to the name attribute. In this case it is “user[login]”.

Similarly for the email address text field, the value of the name attribute is “user[email]”. And for the password entry field, the value of the name attribute is “user[password]”.

Thats all the information we need. The below code should sign up for a git account.

import mechanize #sudo pip install python-mechanize

br = mechanize.Browser() #initiating a browser

br.set_handle_robots(False) #ignore robots.txt

br.addheaders = [("User-agent","Mozilla/5.0")] #our identity

gitbot = br.open("https://github.com") #requesting the github base url

br.select_form(nr=2) #the sign up form in github is in third position(search and sign in

formscome before signup)

br["user[login]"] = username #username for github

br["user[email]"] = email #email for github

br["user[password]"] = password #password for github

sign_up = br.submit()

Possibilities

  1. To automate sign-up (Tip : Use a .txt file to store the username,email. Hardcode password in the program. Use a loop to sign-up for all in .txt file)
  2. Bruteforce attack
  3. Fetch google search results

Limitations

  1. Exception handling is not considered (This may result in 503 error code(service unavailable) while making numerous requests to a website. Tip:looping over different user agents may help)