Python UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 ordinal not in range(128)
Asked Answered
K

3

12

I'm trying to write my first app in Google App Engine with Python (link of the app: http://contractpy.appspot.com/ - it's just an experimental app). The entire code is bellow.

But, when I submit the data, I'm getting this error (showed on logs):

(...) line 265, in get "contractType":geted_contractType
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 949: ordinal not in range(128)

Line 265 is in this if block:

self.response.out.write(yourcontract % {"resident":geted_resident_name,
                                      "nacionality":geted_user_nacionality,
                                      "SSN":geted_user_SSN,
                                      "SSN_error":geted_SSN_error,
                                      "driverLicense":geted_user_driverLicense,
                                      "email":geted_user_email,
                                      "witness ":geted_witness ,
                                      "owner":geted_owner,
                                      "contractType":geted_contractType
                                      })

I tried to make changes, read similar awnswers (like this or this), but nothing helped. I can't figure out in this code what is going wrong. Can someone figure out what is causing such error and how to fix it?

I'm using Python 2.7.

Thanks in advance for any help!

# -*- coding: utf-8 -*-
# -*- coding: utf-8 -*-
#!/usr/bin/env python
#
# Copyright 2007 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

    import webapp2

    form = """

    <html>
      <head>
        <title>Contract with Python</title>
        <style type="text/css">
          .label {text-align: right}
          .error {color: red}
        </style>

      </head>

      <body>
        <h2>Contract with Python</h2>
        <form method="post">
          <table>
            <tr>
              <td class="label">
                resident
              </td>
              <td>
                <input type="text" name="resident" value= "%(resident)s">
              </td>
              <td class="error"> %(resident_error)s

              </td>
            </tr>

             <tr>
              <td class="label">
                nacionality
              </td>
              <td>
                <input type="text" name="nacionality" value= " ">
              </td>
            </tr>

            <tr>
              <td class="label">
                License
              </td>
              <td>
                <input type="text" name="driverLicense" value="">
              </td>
            </tr>

            <tr>
              <td class="label">
                SSN
              </td>
              <td>
                <input type="text" name="SSN" value="%(SSN)s">
              </td>
              <td class="error"> %(SSN_error)s
              </td>
            </tr>


            <tr>
              <td class="label">
                Email (optional)
              </td>
              <td>
                <input type="text" name="email" value="%(email)s">
              </td>
              <td class="error"> %(email_error)s

              </td>
            </tr>
          </table>

    <br>
            <td class="label">
              Contract Type
              <select name = "contractType">
                  <option>House  Rental Contract</option>
                  <option>Car Rental Contract</option>
                  <option>Other</option>
              </select>
              </td>
    <br>
    <br>

    <br>
              owner
              <select name = "owner">
                  <option>House owner</option>
                  <option>Car owner</option>
                  <option>Other owner</option>
              </select>
    <br>
    <br>

    <br>
              Witness
              <select name = "witness">
                  <option>Carl Sagan</option>
                  <option>Mahatma Gandhi</option>
              </select>
    <br>
    <br>

          <input type="submit">
        </form>
      </body>

    </html>
    """

    yourcontract = """
    <html>
      <head>
        <title>Unit 2 Signup</title>
      </head>

      <body>

    %(contractType)s

    In consideration of the agreements of the Resident(s), known as: %(resident)s The owner hereby rents them the dwelling located at %(residentAdress)s, for the period commencing on the %(dateStarts)s, and monthly thereafter until the last day of %(dateEnds)s, at which time this Agreement is terminated. Resident(s), in consideration of owners permitting them to occupy the above property, hereby agrees to the following terms:

    RENT: To pay as rental the sum of $ (rentalSum) per month, due and payable in advance from the first day of every month. Failure to pay rent when due will result in the owner taking immediate legal action to evict the Resident from the premises and seize the security deposit.
    LATE FEE: Rent received after the first of the month will be subject to a late fee of 10% plus (3.00) dollars per day.

    ACCEPTED THIS (dateContract), at (localContract).

    ___________________________________________________
    %(resident)s - Resident

    ___________________________________________________
    %(owner)s – owner

    ___________________________________________________
    %(witness)s – Witness

      </body>
    </html>

    """

    import re

    USER_RE = re.compile(r"^[a-zA-Z0-9_-]{3,20}$")
    def valid_resident(resident):
        return USER_RE.match(resident)

    PASS_RE = re.compile(r"^.{3,20}$")
    def valid_SSN(SSN):
        return PASS_RE.match(SSN)

    EMAIL_RE = re.compile(r"^[\S]+@[\S]+\.[\S]+$")
    def valid_email(email):
        return EMAIL_RE.match(email)

    def escape_html(s):
        for (i,o) in (("&","&"), (">",">"), ("<","<"), ('"','"')):
            s = s.replace(i,o)
            return s

    import time

    import datetime

    def dateToday():
        today = datetime.datetime.today()
        todayDay = str(today.day)
        todayMonth = str(today.month)
        monthExt = {'1':' January ', '2':'February', '3':' March ', '4':'April', '5':'May', '6':'June', '7 ':' July ', '8':'August', '9':'September', '10':'October', '11':'November ', '12':'December'}
        todayYear = str(today.year)
        return(todayDay + ' of  ' + monthExtenso[todaymonth] + ' of ' + todayYear)

    class MainHandler(webapp2.RequestHandler):
        def get(self):
           self.response.out.write(form %{"resident": "",
                                           "SSN": "",
                                           "driverLicense": "",
                                           "email":"",
                                           "resident_error": "",
                                           "SSN_error": "",
                                           "driverLicense_error": "",
                                           "email_error": ""})

        def write_form(self, text_write):
           self.response.out.write(text_write)

        def post(self):
            resident_name = self.request.get(escape_html("resident"))
            user_nacionality = self.request.get("nacionality")
            user_SSN = self.request.get(escape_html('SSN'))
            user_email = self.request.get(escape_html('email'))
            user_driverLicense = self.request.get(escape_html('driverLicense'))
            resident_error = ""
            SSN_error = ""
            driverLicense_error = ""
            contract_type = self.request.get("contractType")
            owner = self.request.get("owner")
            witness  = self.request.get("witness ")

            if (resident_name and valid_resident(resident_name)) \
            and (user_SSN and valid_SSN(user_SSN)) \
            and ((not user_email) or (user_email and valid_email(user_email))):
                self.redirect('/yourcontract?resident=%s&nacionality=%s&SSN=%s&driverLicense=%s&email=%s&witness=%s&owner=%s' % (resident_name, user_nacionality, user_SSN, user_driverLicense, user_email,
    witness, owner))
            else:
                if not valid_resident(resident_name):
                    resident_error = "Oh no!!! this resident name isn't valid!"
                if not valid_SSN(user_SSN):
                    SSN_error = "Oh no!!! SSN isn't valid!"
                if user_email and not valid_email(user_email):
                    email_error = "Oh no!!! e-mail isn't valid!"
                self.write_form(form % {"resident":resident_name,
                                          "resident_error":resident_error,
                                          "SSN":user_SSN,
                                          "SSN_error":SSN_error,
                                          "driverLicense":user_driverLicense,
                                          "email":user_email,
                                          })

    class yourcontractHandler(webapp2.RequestHandler):
        def get(self):
            geted_resident_name = self.request.get('resident')
            geted_user_nacionality = self.request.get("nacionality")
            geted_user_SSN = self.request.get('SSN')
            geted_user_email = self.request.get('email')
            geted_user_driverLicense = self.request.get('driverLicense')
            geted_resident_error = ""
            geted_SSN_error = ""
            geted_driverLicense_error = ""
            #geted_contract_type = self.request.get("contractType")
            geted_owner = self.request.get("owner")
            geted_witness  = self.request.get("witness")
            geted_contractType = self.request.get("contractType")



            self.response.out.write(yourcontract % {"resident":geted_resident_name,
                                          "nacionality":geted_user_nacionality,
                                          "SSN":geted_user_SSN,
                                          "SSN_error":geted_SSN_error,
                                          "driverLicense":geted_user_driverLicense,
                                          "email":geted_user_email,
                                          "witness ":geted_witness ,
                                          "owner":geted_owner,
                                          "contractType":geted_contractType
                                          })


    app = webapp2.WSGIApplication([('/', MainHandler), ('/yourcontract', yourcontractHandler)],
                                  debug=True)
Kenna answered 7/6, 2012 at 14:48 Comment(2)
Check this: bit.ly/unipainBranny
have you tried encode('utf-8'), it may solve your problemSphingosine
T
6

You should really use a proper templating system. Jinja2 is included with AppEngine.

However in the meantime your problem is that your templates are ASCII but your data is not (can't tell if it's utf-8 or unicode). Easy solution is to prefix each template string with u to make it Unicode.

But, you should really use a proper templating system.

Taxable answered 7/6, 2012 at 14:51 Comment(1)
Hi, @Daniel Roseman, I start using jinja2, but, know, I'm stucked with this error: #11072978 Could you provide some help? Thanks again!Kenna
R
3

Here you got solution.

>>> "t".decode("utf-8")
u't'
>>> "\x81".decode("utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "encodings/utf_8.py", line 7799, in decode
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 0: unexpected code byte

>>> "a\x81b".decode("utf-8", "replace")  # this will decode better and the right way.
u'a\ufffdb'
Rainbow answered 10/9, 2012 at 5:23 Comment(1)
The encoding in your example hasn't gone better. All you've achieved is replacing a non-ASCII characters with Unicode character FFFD. I can't imagine that the asker would want this character turning up in his/her data.Utrillo
G
2

Set default encoder at the top of your code

In appcfg.py which resides inside /google/appengine/tools/appcfg.py

on line 73 add

import sys
reload(sys)
sys.setdefaultencoding("ISO-8859-1")
Guano answered 9/2, 2015 at 12:28 Comment(3)
this was the only solution that worked for me, I tried the other two and they did not work. thank you.Schaffner
The same, adding this to my script resolved this error. Nothing else worked. I set to 'utf-8'.Inebriant
with python 3.9.7 i am getting NameError: name 'reload' is not definedPuppy

© 2022 - 2024 — McMap. All rights reserved.