Python

Learn Python from Coursera’s Introduction to Interactive Programming in Python and here are some other project I did in Python.

concurrent.futures: a simple multi-thread execution

Recently I was expose to a way to launch parallel tasks in python by using concurrent.futures.  Let say you have a function call doSomething(i) that do something with a value, i and return a result. You can setup a pool of 8 threads as following:

executor = concurrent.futures.ThreadPoolExecutor(max_workers=8)

If we need to doSomething(i) on a range(64) values, we can map the result using the executor as follow:

results = executor.map(doSomething, range(64))

This will return an array of 64 result. Be sure to signal the executor to free any resources by having the following line.

executor.shutdown()

Additional References:

CHR and ORD

I was working with pkcs#7 padding from cryptopals.com and was dancing around between hex() and unhexlify to convert the padding size into a one byte character.

>>>binascii.unhexlify(bytes('0'+hex(4)[2:], encoding='utf-8')
b'\x04'

In short, I must add a string zero in front if the size is less than 16. Very ugly. Then I ran into this function chr() and ord() that does what I needed. Convert a integer value into a character and back.

>>> chr(4)
'\x04'
>>> ord('\x04')
4

Simple and clean. The only thing is to make sure what encoding is the string I am appending this at the end so I can put it into the right encoding for it.

Additional References:

Converting between bytes and hex

I was working on set 1 in the matasano crypto challenges and coding along using python2 and later found out from here that python2 is legacy and I should start using python3.

I had the code for converting hex string working in python2 but was having trouble doing it in python3 due to some of the quark between bytes and string. Here are the summary of what I found.

# create a string
>>> s='hello'
>>> s
'hello'

# string to byte encoding in ascii
>>> bs=bytes(s, encoding='utf-8')
>>> bs
b'hello'

# byte encoding in ascii to string
>>> bs.decode()
'hello'

# byte encoding in ascii to hex representation in byte encoding in ascii
>>> hb=binascii.hexlify(bs)
b'68656c6c6f'

# hex representation in byte encoding in ascii to byte encoding in ascii
>>> binascii.unhexlify(hb)
b'hello'

# hex representation in byte encoding in ascii to hex representation string
>>> hs = hb.decode()
>>> hs
'68656c6c6f'

# hex representation string to byte encoding in ascii
>>> binascii.unhexlify(hs)
b'hello'

# string to hex representation in byte encoding in ascii : attempt in error 
>>> binascii.hexlify(s)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: 'str' does not support the buffer interface

Notice that both hex representation in byte encoding of ascii (hb) and hex representation string (hs) can use unhexlify and get the byte encoding in ascii of ‘hello’. However, we can only hexlify the byte encoding in ascii of ‘hello’ and not the string ‘hello’ itself. To get a string convert to a hex representation of byte encoding in ascii, we need to do it in 2 steps.

>>> binascii.hexlify(bytes(s, encoding='ascii'))
b'68656c6c6f'

bytes/strings in python 2 and 3

I was trying to convert some python code from 2 to 3 and found this documentation regarding string and bytes representation. Some of the examples really help make it clear are:

>>>> b''=''
False

>>>> b'xyz'[0]
120

>>>> b'xyz'[0:1]
b'x'

>>>> b'xyz'[3:4]
b''

The first show comparing byte is not the same as unicode of the same string. The second shows that indexing a byte string return an integer. The third one show how to correctly index a byte character out of a byte string by using slicing. The last one show that there is no index out of range warning if slicing outside the length of a byte string, just a empty byte string.

See bytes/unicode comparison section for more explanation.