How to access file in /tmp directory of AWS Lambda

Question

I have downloaded a file from a URL into the /tmp directory of AWS Lambda(as this was the only writable path in Lambda).

My motive is to create an Alexa Skill which will download file from an URL. Hence I created a lambda function.

How can I access the downloaded file from /tmp folder in lambda?

My code is : -

#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import print_function
import xml.etree.ElementTree as etree
from datetime import datetime as dt
import os
import urllib
import requests

from urllib.parse import urlparse


def lambda_handler(event, context):
    """ Route the incoming request based on type (LaunchRequest, IntentRequest,
    etc.) The JSON body of the request is provided in the event parameter.
    """

    print('event.session.application.applicationId=' + event['session'
          ]['application']['applicationId'])

    # if (event['session']['application']['applicationId'] !=
    #         "amzn1.echo-sdk-ams.app.[unique-value-here]"):
    #     raise ValueError("Invalid Application ID")

    if event['session']['new']:
        on_session_started({'requestId': event['request']['requestId'
                           ]}, event['session'])

    if event['request']['type'] == 'LaunchRequest':
        return on_launch(event['request'], event['session'])
    elif event['request']['type'] == 'IntentRequest':
        return on_intent(event['request'], event['session'])
    elif event['request']['type'] == 'SessionEndedRequest':
        return on_session_ended(event['request'], event['session'])


def on_session_started(session_started_request, session):
    """ Called when the session starts """

    print('on_session_started requestId='
          + session_started_request['requestId'] + ', sessionId='
          + session['sessionId'])


def on_launch(launch_request, session):
    """ Called when the user launches the skill without specifying what they
    want
    """

    print('on_launch requestId=' + launch_request['requestId']
          + ', sessionId=' + session['sessionId'])

    # Dispatch to your skill's launch

    return get_welcome_response()


def on_intent(intent_request, session):
    """ Called when the user specifies an intent for this skill """

    print('on_intent requestId=' + intent_request['requestId']
          + ', sessionId=' + session['sessionId'])

    intent = intent_request['intent']
    intent_name = intent_request['intent']['name']

    # Dispatch to your skill's intent handlers

    if intent_name == 'DownloadFiles':
        return get_file(intent, session)
    elif intent_name == 'AMAZON.HelpIntent':
        return get_welcome_response()
    else:
        raise ValueError('Invalid intent')


def on_session_ended(session_ended_request, session):
    """ Called when the user ends the session.Is not called when the skill returns should_end_session=true """

    print('on_session_ended requestId='
          + session_ended_request['requestId'] + ', sessionId='
          + session['sessionId'])


    # add cleanup logic here

# --------------- Functions that control the skill's behavior ------------------

def get_welcome_response():
    """ If we wanted to initialize the session to have some attributes we could add those here """

    session_attributes = {}
    card_title = 'Welcome'
    speech_output = \
        "Welcome to file download Application. Please ask me to download files by saying, Ask auto downloader for download"

    # If the user either does not reply to the welcome message or says something
    # that is not understood, they will be prompted again with this text.

    reprompt_text = \
        "Please ask me to download files by saying, Ask auto downloader for download"
    should_end_session = False
    return build_response(session_attributes,
                          build_speechlet_response(card_title,
                          speech_output, reprompt_text,
                          should_end_session))


def get_file(intent, session):
    """ Grabs the files from the path that have to be downloaded """

    card_title = intent['name']
    session_attributes = {}
    should_end_session = True
    username = '*******'
    password = '*******'

    url = 'https://drive.google.com/drive/my-drive/abc.pdf'
    filename = os.path.basename(urlparse(url).path)

    # urllib.urlretrieve(url, "code.zip")

    r = requests.get(url, auth=(username, password))

    if r.status_code == 200:
        with open("/tmp/" + filename, 'wb') as out:
            for bits in r.iter_content():
                out.write(bits)

    speech_output = 'The file filename has been downloaded'
    return build_response(session_attributes,
                          build_speechlet_response(card_title,
                          speech_output, reprompt_text,
                          should_end_session))


# --------------- Helpers that build all of the responses ----------------------

def build_speechlet_response(
    title,
    output,
    reprompt_text,
    should_end_session,
    ):
    return {
        'outputSpeech': {'type': 'PlainText', 'text': output},
        'card': {'type': 'Simple', 'title': 'SessionSpeechlet - ' \
                 + title, 'content': 'SessionSpeechlet - ' + output},
        'reprompt': {'outputSpeech': {'type': 'PlainText',
                     'text': reprompt_text}},
        'shouldEndSession': should_end_session,
        }


def build_response(session_attributes, speechlet_response):
    return {'version': '1.0', 'sessionAttributes': session_attributes,
            'response': speechlet_response}

Ultra

Do you download the file in one invocation and want to access it in another invocation? – Dimitris

I want to download and access the file in the same invocation. How can I do that? @blueCat – Ultra

Answer 1

Just open the file as you would do normally:

with open('/tmp/'+ filename, 'rb') as file:
    ...

This works all the time for me. Have you tried this and experienced any issues?

Please note that Lambda runs in a container. When you download once, the file will be in the /tmp folder until this container lives. After a container is launched to serve your function, it lives for tipically 10-30 minutes (could be less or more, it's not an official fixed time). So, instead of always downloading the file, you should check if the file isn't already in the /tmp directory. If yes, you obviously don't have to download again! ;)

To make this check use:

if not os.path.isfile('/tmp/' + filename):
    download...

Indebtedness

Answer 2

For anyone who ends up here and is using a Docker image as the Lambda function: AWS cleans out /tmp either when the Docker image is uploaded to ECS or when the Lambda function is executed.

This means that if you are relying on anything being in /tmp (e.g. you are copying a file to /tmp in your Dockerfile) you will observe that the Docker image runs fine locally, and contains the file in /tmp as expected, but you get an error when you try to run this same Docker image as a Lambda function in AWS because that file is not in /tmp.

I would suggest putting the file in the LAMBDA_TASK_ROOT (these days that is the /var/task directory), which is read-only. If you need to modify this file, then I would read it from the LAMBDA_TASK_ROOT directory, and write it to /tmp.

Polygynist

if /var/task folder in lambda is read-only, how will you put a file? Could you please clarify? – Panga

You can place files in /var/task via the Dockerfile. It's only read-only on AWS. – Polygynist

Okay that makes sense, thanks! However OP was asking about performing the same on AWS env I think. – Panga

True. OP's question was already answered, and that accepted answer is still valid. I answered here because this is where Google took me when trying to figure out a similar issue. – Polygynist

@Polygynist This is exactly what I needed to know, thanks! – Lennon

this answer was a big help to me. thanks. – Unmistakable

Answer 3

Just another perspective. For security reason, AWS shows access denied error instead of not found error in such cases. So it could be that the file was not written correctly and actually doesn't exist.

Allhallowtide