TypeError: can't concat str to bytes when converting Python 2 to 3 with Encryption Function

Question

I am trying to transfer a code from python2 to 3. The problem happens. "pad * chr(pad)" looks like a string but when I print it out it shows . I dont know what it is really is.

<ipython-input-26-6c9679723473> in aesEncrypt(text, secKey)
     43 def aesEncrypt(text, secKey):
     44     pad = 16 - len(text) % 16
---> 45     text =text +  pad * chr(pad)
     46     encryptor = AES.new(secKey, 2, '0102030405060708')
     47     ciphertext = encryptor.encrypt(text)

TypeError: can't concat str to bytes

I then tried encode() but it didnt work. I am wonder how can concat two string in python3.

<ipython-input-53-e9f33b00348a> in aesEncrypt(text, secKey)
     43 def aesEncrypt(text, secKey):
     44     pad = 16 - len(text) % 16
---> 45     text = text.encode("utf-8") + (pad * chr(pad)).encode("utf-8")
     46     encryptor = AES.new(secKey, 2, '0102030405060708')
     47     ciphertext = encryptor.encrypt(text)

AttributeError:'bytes' object has no attribute 'encode'

For the reference the original code is

作者：路人甲
链接：https://www.zhihu.com/question/31677442/answer/119959112
#encoding=utf8
import requests
from bs4 import BeautifulSoup
import re,time
import os,json
import base64 
from Crypto.Cipher import AES
from pprint import pprint 

Default_Header = {
                  'Referer':'http://music.163.com/',
                  'Host':'music.163.com',
                  'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.3.0',
                  'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
                  'Accept-Encoding':'gzip, deflate'
                }

BASE_URL = 'http://music.163.com'

_session = requests.session()
_session.headers.update(Default_Header)

def getPage(pageIndex):
    pageUrl = 'http://music.163.com/discover/playlist/?order=hot&cat=全部&limit=35&offset='+pageIndex
    soup = BeautifulSoup(_session.get(pageUrl).content)
    songList = soup.findAll('a',attrs = {'class':'tit f-thide s-fc0'})
    for i in songList:
        print i['href']
        getPlayList(i['href'])

def getPlayList(playListId):
    playListUrl = BASE_URL + playListId
    soup = BeautifulSoup(_session.get(playListUrl).content)
    songList = soup.find('ul',attrs = {'class':'f-hide'})
    for i in songList.findAll('li'):
        startIndex = (i.find('a'))['href']
        songId = startIndex.split('=')[1]
        readEver(songId)

def getSongInfo(songId):
    pass

def aesEncrypt(text, secKey): 
    pad = 16 - len(text) % 16 
    text = text + pad * chr(pad) 
    encryptor = AES.new(secKey, 2, '0102030405060708') 
    ciphertext = encryptor.encrypt(text) 
    ciphertext = base64.b64encode(ciphertext) 
    return ciphertext 
def rsaEncrypt(text, pubKey, modulus): 
    text = text[::-1] 
    rs = int(text.encode('hex'), 16)**int(pubKey, 16) % int(modulus, 16) 
    return format(rs, 'x').zfill(256) 
def createSecretKey(size): 
    return (''.join(map(lambda xx: (hex(ord(xx))[2:]), os.urandom(size))))[0:16] 

def readEver(songId):
    url = 'http://music.163.com/weapi/v1/resource/comments/R_SO_4_'+str(songId)+'/?csrf_token=' 
    headers = { 'Cookie': 'appver=1.5.0.75771;', 'Referer': 'http://music.163.com/' } 
    text = { 'username': '', 'password': '', 'rememberLogin': 'true' } 
    modulus = '00e0b509f6259df8642dbc35662901477df22677ec152b5ff68ace615bb7b725152b3ab17a876aea8a5aa76d2e417629ec4ee341f56135fccf695280104e0312ecbda92557c93870114af6c9d05c4f7f0c3685b7a46bee255932575cce10b424d813cfe4875d3e82047b97ddef52741d546b8e289dc6935b3ece0462db0a22b8e7' 
    nonce = '0CoJUm6Qyw8W8jud' 
    pubKey = '010001' 
    text = json.dumps(text) 
    secKey = createSecretKey(16) 
    encText = aesEncrypt(aesEncrypt(text, nonce), secKey) 
    encSecKey = rsaEncrypt(secKey, pubKey, modulus) 
    data = { 'params': encText, 'encSecKey': encSecKey } 
    req = requests.post(url, headers=headers, data=data) 
    total = req.json()['total']
    if int(total) > 10000:
        print songId,total
    else:
        pass


if __name__=='__main__':
    for i in range(1,43):
        getPage(str(i*35))

Distaff

On line 28 you have a print statement without parenthesis - you'll have to use parenthesis with print if you're migrating to python 3 @RoyDai – Nostology

@Nostology thanks for the pointing out.The code was in Python 2. I noticed that and changed accordingly.Now I stuck at the TypeError: can't concat str to bytes – Distaff

I'm guessing that you've barely started conversion to python 3 you should read what I wrote about the differences between P2 vs P3 strings. – Nostology

Are you using an IDE that you can use to debug this? Set a breakpoint in the aesEncrypt(text, secKey) function and see what the values of text and secKey are. – Nostology

"I dont know what it is really is." Ask Python: print out its type. – Gasolier

@RoyDai return (''.join(map(lambda xx: (hex(ord(xx))[2:]), os.urandom(size))))[0:16] you have a problem with this line. You might want to break that statement into one operation per line to step through it. You're trying to apply ord() to an integer value. – Nostology

@Nostology For aesEncrypt(text, secKey), Textis {"username": "", "password": "", "rememberLogin": "true"}as changed by json.dumps. secKey isnonce = '0CoJUm6Qyw8W8jud', I have change hex(ord(xx)) to hex(xx) as it already a integer in python. – Distaff

encText = aesEncrypt(aesEncrypt(text, nonce), secKey) this line calls aesEncrypt twice. inner call with a string, outer call with bytes object. On Python 2 it wouldn't matter because str on py2 is bytes.. but py2, str is unicode – Nostology

@Nostology thank for the clarification. Do you mean ciphertext is bytes? – Distaff

@RoyDai secKey is a string first time through aesEncrypt - second time it's a list of strings. and text is str first time, and it's bytes 2nd time. – Nostology

@Nostology seckey I suppose it is still a string after ? – Distaff

@RoyDai I don't know. But I'm certain you don't want it to change types like that. I put it in an answer so I could format it. – Nostology

Answer 1

around line 85: encText = aesEncrypt(aesEncrypt(text, nonce), secKey)

aesEncrypt() is called first time with data as shown with subscript '_1' and 2nd time with data shown with subscript '_2'. Notice that the type of secKey changes from a string to a list of strings, and text from a string to a bytes object.

>>> secKey_1
'0CoJUm6Qyw8W8jud'
>>> secKey_2
['e4', '1a', '61', '7c', '1e', '62', '76', '5', '94', '62', '5a', '92', '9', 'fd', '2f', '4a']
>>> 
>>> text_1
'{"username": "", "password": "", "rememberLogin": "true"}'
>>> text_2
b'qjTTWCVgh3v45StLOhGtNtY3zzoImIeGkRry1Vq0LzNSgr9hDHkkh19ujd+iqbvXnzjmHDhOIA5H0z/Wf3uU5Q=='
>>> 

Nostology

thanks very much for the testing. Now I get where the issue lays – Distaff

What IDE do you use for debugging? @RoyDai – Nostology

,I am using jupyter notebook...I tried spyder but not very familiar with it. – Distaff

I don't know how good that is as a debugger. You should be able to run a script from your IDE and trap exceptions - execution stops at the event and it shows you what line caused it and you can step through it and see variable values.. you can walk up and down the stack examining values. Strongly recommend becoming as familiar as you can with such a debugger. Eclipse and PyCharm work well for debugging. @RoyDai – Nostology

yeah okay.. i downloaded Jupyter.. ugh.. it's barely even an editor. You really should try out PyCharm. Saying it's a world of difference would be a huge understatement. jetbrains.com/pycharm ... or use Eclipse eclipse.org .. it's really a disservice if people get stuck with jupyter and don't know what else is out there. – Nostology

Thanks I get jupyter is just a easy notebook for newbees. I will take your advice to try pycharm. thanks. – Distaff

New developers will be stuck at newbie level forever if they don't move beyond Jupyter ;-) Yes.. do try PyCharm, you'll find it's very good. – Nostology

Thanks for the advice. I exceed my limits for question today but I actually get another problem for this code. Myrequest.post(url).json()would return a JSONDecodeError: Expecting value: line 1 column 1 (char 0). No sure if you have any experience on that. Thanks – Distaff

Training wheels are off my friend. Download PyCharm and move up. ;-) @RoyDai – Nostology

Answer 2

Thanks to @Todd. He has found the issue. aesEncrypt()has been called twice and it returns bytes while it receives str, which is acceptable in Python2 but not for Python3.

In the end, I change return ciphertext to return str(ciphertext).

Distaff