Listing S3 Top level folders
using simple Python code.

Some time back I had to export list of top level folders in a S3 bucket.
I used python and boto to do this task.
I think following code is self explanatory.

from boto.s3.connection import S3Connection

# Following 3/4 lines need to be changed to make this work.
aws_key = 'SET AWS KEY here'
aws_secret = 'SET AWS SECRET here'
bucket_name = 'SET BUCKET name'
output_file = 'folders.txt'

conn = S3Connection(aws_key, aws_secret)
bucket  = conn.get_bucket(bucket_name)
folders = bucket.list("","/")
count = 0
with open(output_file, 'w') as outfile:
    for folder in folders:
        count += 1
        outfile.write(folder.name[:-1] + '\n')
        print '.',

print('\n\n Completed. Total folders: ' + count)

Above program does two things,

  • Prints ... on system output to show progress and at the end prints count of folders in the given bucket_name.
  • Write folder names to the output_file separated by \n.

Apparently this is an icebreaking post for me to start blogging again.
Just started with simple one, more to follow.

Share

Great!! You read till this point, just go ahead and share this post to your followers, collegues and friends. Thanks!

About Author

Sakthi Priyan H
Passionate Programmer

  • I am passionate about building excellent teams, processes and systems.
  • Primarily I use Java, Scala and Python for building various systems and tools.
  • Building API services, Big data processing and Machine Learning systems in Crayon Data.
  • Also, interested in Golang and building web apps using Javascript ecosystem.
  • I wrote my first program in BASIC in 1998, Passionate about computers since then.