Get list of ALL offers from Amazon Product Advertising API

Question

I need to be able to get a listing of all offers for a product using the amazon API. This is the request I'm sending:

http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&Operation=ItemLookup&SubscriptionId=&AssociateTag=&Version=2011-08-01&ItemId=B007IJQDQQ&IdType=ASIN&ResponseGroup=Offers&Condition=All&MerchantId=All

This is returning the follow under offers:

<Offers>
<TotalOffers>1</TotalOffers>
<TotalOfferPages>1</TotalOfferPages>
<MoreOffersUrl>
http://www.amazon.com/gp/offer-listing/B007IJQDQQ%3FSubscriptionId%3DAKIAJTZ6VROOTPJAPPWQ%26tag%3Damazoautom-20%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3DB007IJQDQQ
</MoreOffersUrl>
<Offer>
<OfferAttributes>
<Condition>New</Condition>
</OfferAttributes>
<OfferListing>
<OfferListingId>
TR5sygI0VR7cwy3Hg0DBHwgCWFnkqXLyCR70jjMhy1h8gWlEisKYt5cqDbS2Fu1SEqDtfBILdxYDziJdFzfsqMpPJkBqcYV3DFovHx1nXWRy9wHS6CFZXknBvsCo1bxYS%2BsxAeYrZHrS6g6aakEJQA%3D%3D
</OfferListingId>
<Price>
<Amount>375</Amount>
<CurrencyCode>USD</CurrencyCode>
<FormattedPrice>$3.75</FormattedPrice>
</Price>
<AmountSaved>
<PercentageSaved>56</PercentageSaved>
<Availability>Usually ships in 1-2 business days</Availability>
<AvailabilityAttributes>
<IsEligibleForSuperSaverShipping>0</IsEligibleForSuperSaverShipping>
</OfferListing>
</Offer>
</Offers>

As you can see it only returns the 1 offer even though it says there are 3 new offers. Does anyone know of a way to get all the offers and not just the lowest one? Merchant ID='ALL' doesn't do it, neither do any of the other response groups like offerFull, offerSummary, or offers.

Khedive

Same issue - I don't know why Amazon even bothers to offer this API, it's restrictive nearly to the point of pointlessness. – Deyoung

zinc.io (my company) has an API to return the full list of offers. See docs.zincapi.com. – Pinnate

Answer 1

After doing some research this not possible in the API, however you can parse the HTML of the standard page to get the details:

1) Use http://simplehtmldom.sourceforge.net/ for parsing HTML (If your not using php, you'll need find something similar)

2) CURL (make sure you set your header or amazon will error) both http://www.amazon.com/gp/offer-listing/B007IJQDQQ/?condition=new and http://www.amazon.com/gp/offer-listing/B007IJQDQQ/?condition=used

3) To handle pages count total number of offers (found #new span.numberreturned) and divide by 15 to work out number of pages.

4) You'll need to parse each page, URLs would be like some:

eg page 2 http://www.amazon.com/gp/offer-listing/B005IMB5SG/?condition=used&startIndex=15

eg page 3 http://www.amazon.com/gp/offer-listing/B005IMB5SG/?condition=used&startIndex=30

Hope this is enough info to get you started, sorry I don't have access to the working script I wrote a while ago.

Equity

Thank you for responding. Yeah, I ended up using httpclient with java to parse the page a few months ago. It's unfortunate that amazon would limit their API in this way. Only issue I had in doing this was amazon puts a limit on the number of requests you can make (1 request per second) otherwise it blocks you for a few minutes. Only way I could find a way around this was to put a thread.sleep in my code so I only make one request a second. – Khedive

Is scraping really the only way to get the information from Amazon? I thought Amazon expressly forbids web scraping. – Doing

Is offerlisting id's are available in this page currently, Can anyone confirm it please.. – Irretentive

which headers should be added? – Bantam

Does this actually still work? It seems that page is loaded with JS so not all the information is there. Is there a better way of getting all the offers now? – Seminal

Answer 2

The Product API scratch pad returns an "All Offers" link which you can manually fetch over HTTP and parse to provide a list of DOM nodes which contain the information you need.

It's like the http://www.amazon.com/gp/offer-listing/B007IJQDQQ/?condition=used link @dciso mentioned several years ago but with the API information contained to help follow the amazon rules closer.

<ItemLookupResponse>
    <Items>
        <Item>
        <ASIN>B00I8BICB2</ASIN>
        <ParentASIN>B077PSDB4X</ParentASIN>
        <DetailPageURL>https://www.amazon.com/Sony-Mirrorless-Digitial-3-0-Inch-16-50mm/dp/B00I8BICB2?psc=1&SubscriptionId_____tag=_____&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B00I8BICB2</DetailPageURL>
        <ItemLinks>
            ...
            <ItemLink>
                <Description>All Offers</Description>
                <URL>https://www.amazon.com/gp/offer-listing/B00I8BICB2?SubscriptionId=______&tag=_____&linkCode=xm2&camp=2025&creative=386001&creativeASIN=B00I8BICB2</URL>
            </ItemLink>
        </ItemLinks>

Quarrelsome