Kinesis, the streaming data service that Amazon Web Services announced in November, is now publicly available. The service is comparable in theory to popular open source technologies such as Apache Storm, only Kinesis brings with the fully managed experience that’s becoming par for the course within AWS.
Like Storm, Kinesis can process data in real time before shipping it into another data store — most likely Elastic MapReduce, Redshift or DynamoDB within the AWS platform. Unlike Storm, however, Kinesis can maintain data for up to 24 hours and isautomatically scalable up to hundreds of terabytes per hour via a software development kit, or SDK. Kinesis does include a connector for porting data to Storm, which AWS General Manager for Data Science Matt Wood said is a possibility in cases where existing Storm users want to keep using it for processing data while automating the collection with Kinesis.