https://www.idata.net/ IData Corporation today released an open-source no-code Pipeline for AWS that can save companies hundreds of hours of development time. The Pipeline can easily transform and load data files into a data lake or into a data warehouse such as Snowflake or Redshift, all with no coding.
The Pipeline can perform data quality checks, data deduplication, transformations and load data directly into its destination (S3, Snowflake, or Redshift). Notifications are automatically sent when data lands, allowing downstream users or systems to process it immediately.
“By working with the open-source Pipeline, companies can move their expensive programming talent to focus on AI and machine learning tasks, instead of spending months building a data pipeline first”, explains Todd Fearn, CEO of IData. “Our customers can have this Pipeline up and running in their own AWS account in just a couple of hours, streaming data downstream to internal and external systems”.
Additionally, the Pipeline employs Apache Iceberg technology. Apache Iceberg is an open-source project that enables building a Lakehouse architecture on top of data lakes. Apache Iceberg provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing on top of existing data lakes, such as S3.
The open-source IData Pipeline software is available on GitHub at https://github.com/idata-corporation/pipeline-server. Technical documentation is available at https://docs.idata.net
IData Corporation is a leading cloud software company based out of Hackensack, NJ. The company has primarily worked in the financial services sector with companies such as Freddie Mac, Ameriprise, Bank of America, Credit Suisse and a number of others.
Visit AITechPark for cutting-edge Tech Trends around AI, ML, Cybersecurity, along with AITech News, and timely updates from industry professionals!
