Unless I'm missing something, it seems that none of the APIs I've looked at will tell you how many objects are in an S3 bucket / folder(prefix). Is there any way to get a count?
A:
None of the APIs will give you a count because there really isn't any Amazon specific API to do that. You have to just run a list-contents and count the number of results that are returned.
webdestroya
2010-05-19 04:15:30
For some reason, the ruby libs (right_aws/appoxy_aws) won't list more than the the first 1000 objects in a bucket. Are there others that will list all of the objects?
fields
2010-05-19 12:39:48
When you request the list, they provide a "NextToken" field, which you can use to send the request again with the token, and it will list more.
webdestroya
2010-05-19 17:58:51
A:
The api will return the list in increments of 1000. Check the IsTruncated property to see if there are still more. If there are, you need to make another call and pass the last key that you got as the Marker property on the next call. You would then continue to loop like this until IsTruncated is false.
See this Amazon doc for more info: Iterating Through Multi-Page Results
BigJoe714
2010-05-19 17:49:17
A:
There is no way, unless you
A) list them all in batches of 1000 (which can be slow and suck bandwidth - amazon seems to never compress the XML responses), or
B) log into your account on S3, and go Account - Usage. It seems the billing dept knows exactly how many objects you have stored!
Simply downloading the list of all your objects will actually take some time and cost some money if you have 50 million objects stored.
Also see this thread about StorageObjectCount - which is in the usage data. http://developer.amazonwebservices.com/connect/thread.jspa;jsessionid=4AEDE60D33DBA609CEFD070612D9E6C0?messageID=104493𙠭
An S3 API to get at least the basics, even if it was hours old, would be great.
Tom Andersen
2010-05-20 17:44:24
Sorry: http://developer.amazonwebservices.com/connect/thread.jspa?messageID=164220
Tom Andersen
2010-06-02 15:27:33