The Camelizer could not detect the main product in this page

Edit Subject

What happened there is, we didn't have the product in our database, so the ASIN was queued for processing. We do said processing in the background so that the Camelizer request can return immediately, but that makes us unable to tell the extension if the product exists or not (thus the two scenario error message) since we haven't asked the Amazon API for its data yet.

If you go back to that page and try loading the Camelizer again, you get a "We don't have enough data to chart" message and the usual UI for tracking the product.

I agree that the user should be able to track a product even if we have no data, but in this situation it's hard to say if users should be allowed to track products that aren't even (and may never be) in the DB.

> that makes us unable to tell the extension if the product exists
> or not (thus the two scenario error message) since we haven't
> asked the Amazon API for its data yet.

I see.

To me, "could not detect the main product" implies a parsing operation...where does this parsing happen? Does the extension parse out the info? For that matter why is parsing happening at all? Isn't the product identifier right in the URL?

I would run some basic sanity checks on the parsed ID, and display the first part of that error only if it fails.

> We do said processing in the background...

And so why doesn't the extension put up a "retrieving information..." message and asynchronously update the display once the data has been obtained by your servers?

Parsing generally happens client-side. Parsing is required because you still have to extract the ASIN from the URL, and there are a number of ways it can be put into the URL. I hear you on the error message though, it could be clearer.

The reason we don't update asynchronously is, it can take many minutes for a product to be added to our system, depending on certain factors (search load, # of active and higher priority background jobs [wishlist importing, delivery of price drop alerts], etc). Having the extension ping the server while waiting on the product to be processed would likely just result in a bunch of wasted requests when the user navigates away from the product page prior to the server being finished.

Perhaps that points to the need for faster background processing...

> Perhaps that points to the need for faster background
> processing...

If turnaround on an Amazon web service request proxied through your servers is taking "many minutes," then yes.

Or a simplified version for this scenario. The only thing the extension really needs to know in a matter of seconds is whether the product ID is valid. Once you have that, you can at least display the UI to capture tracking information.

Another approach would be to have the extension talk directly to the Amazon service to validate the ID and retrieve current price info to display in the tracking form. (Can you securely store your AWS key in the extension?)

> Having the extension ping the server...

Make one request and let it hang until the data is available.

(Better ways of addressing this in HTML 5.)

Not all Amazon API requests take minutes to process, but we get so many Camelizer requests that we can't always process those immediately and quickly. Particularly given the number of other API requests we may be doing at any given time, and the request limits Amazon imposes on us.

You can't track a product we don't yet have in our database, so having the extension access the API directly might not be the way to go. Relying on product data returned by the Camelizer would potentially leave our system open to being filled with fake data by Mean People.

Leaving a connection open means one less connection for some other user. Server resources aren't unlimited, much to my chagrin =/

> Relying on product data returned by the Camelizer would
> potentially leave our system open to being filled with fake data...

You wouldn't collect product data via the Camelizer. Only user specified price targets. The data retrieved by the Camelizer would only be used to complete the presentation in the Camelizer UI.

Those submitted price targets would go into a price watch record as it does now, but instead of linking to a populated product record, it would link to a provisional product record, which presumably would be turned into a populated record after your background process collects the data from Amazon.

You'd need a cleanup job to periodically check for stranded provisional records, maybe retry the request with Amazon again, and if the product can't be retrieved, delete the record and email the user that their price watch was deleted and why.

And you'd have to add some UI sugar to the site to explain why the user can't see any product info for a provisional product, but I think your code already handles the "no data" situation.

Good ideas, though I'm not sure implementing all of that is a high priority in the near term. But I think it will help to add some polish to the error messages at least.

> ...I'm not sure implementing all of that is a high priority in
> the near term.

It's a pretty big usability roadblock to not be able to track an item from the extension. I rarely use the extension to just view pricing graphs, so if it can't be used to facilitate price tracking, then I'd be better off using a bookmarklet or something that kicks me over to your site. (Maybe I need to create a greasemonkey script that accomplishes prompting for price targets when an item gets added to my wishlist.)

You could at least show a link in the extension's panel that says something like "we don't have any data on this product yet, but you can set a price watch using _camelcamelcamel_."

I'm guessing Amazon's product database is vast compared to the number of products you've loaded into your database so far, so this won't exactly be a rare occurrence.

We've got over 3 million entries in our products db for Amazon, so I think we're probably covering the majority of popular items, particularly since we add any that users request data on (via camelizer) or find via search (on our site.)

So, while it isn't something that never happens, it might be more rare than you think. Then again, maybe you're right. How often do you experience this particular situation?

> How often do you experience this particular situation?

I've seen "no data" on the site several times, but so far only once via the Camelizer, though I only just started using the extension as a means to initiate price tracking, and rarely used it prior.