Integrating Spring Batch and Spring Integration

This talk is for everyone who wants to efficiently use Spring Batch and Spring Integration together. Users of Spring Batch often have the requirements to interact with other systems, to schedule the periodic execution Batch jobs and to monitor the execution of Batch jobs. Conversely, Spring Integration users periodically have Big Data processing requirements, be it for example the handling of large traditional batch files or the execution of Apache Hadoop jobs. For these scenarios, Spring Batch is the ideal solution. This session will introduce Spring Batch Integration, a project that provides support to easily tie Spring Batch and Spring Integration together. We will cover the following scenarios: Launch Batch Jobs through Spring Integration Messages Generate Informational Messages Externalize Batch Process Execution using Spring Integration Create Big Data Pipelines with Spring Batch and Spring Integration.



Source: https://www.youtube.com/watch?v=8tiqeV07XlI

Getting started with Spring Data and Apache Hadoop

Big data and Hadoop is widely considered to be the next generation data platform. Hadoop is notoriously difficult to work with and just diving in and starting coding can easily lead to frustration. A better way is to leverage your existing Java and Spring skills to take advantage of these new technologies. In this presentation we will introduce Spring Data for Apache Hadoop and see how it can make working with Hadoop easier. We will also cover several ways to install a small Hadoop cluster that can be used to test your new Hadoop applications.



Source: https://www.youtube.com/watch?v=tIPA6vMZomQ&index=3&list=PLgGXSWYM2FpOXky1eHFbX0Eb0I58YmCNS

In-memory data and compute on top of Hadoop

Hadoop gives us dramatic volume scalability at a cheap price. But core Hadoop is designed for sequential access - write once and read many times; making it impossible to use hadoop from a real-time/online application. Add a distributed in-memory tier in front and you could get the best of two worlds - very high speed, concurrency and the ability to scale to very large volume. We present the seamless integration of in-memory data grids with hadoop to achieve interesting new design patterns - ingesting raw or processed data into hadoop, random read-writes on operational data in memory or massive historical data in Hadoop with O(1) lookup times, zero ETL Map-reduce processing, enabling deep-scale SQL processing on data in Hadoop or the ability to easily output analytic models from hadoop into memory. We introduce and present the ideas and code samples through Pivotal in-memory real-time and the Hadoop platform.

Your Data, Your Search, Elasticsearch

Finding relevant information fast has always been a challenge, even more so in today's growing "oceans" of data. This talk explores the area of real-time full text search, using Elasticsearch, an open-source, distributed search engine built on top of Apache Lucene. The session will showcase how to perform real-time searches on structured and non-structured data alike, how to cope with types and suggestions, do social graph filters and aggregations for efficient analytics. All from a Spring perspective Last but not least, the presentation focuses on the Hadoop platform and how Map/Reduce, Hive, Pig or Cascading jobs can leverage a search engine to significantly speed up execution and enhance their capabilities.

The presentation covers architectural topics such as index scalability, data locality and partitioning, using off and on-premise storages (HDFS, S3, local file-systems) and multi-tenancy.

Hadoop - Just the Basics for Big Data Rookies

This session assumes absolutely no knowledge of Apache Hadoop and will provide a complete introduction to all the major aspects of the Hadoop ecosystem of projects and tools. If you are looking to get up to speed on Hadoop, trying to work out what all the Big Data fuss is about, or just interested in brushing up your understanding of MapReduce, then this is the session for you. We will cover all the basics with detailed discussion about HDFS, MapReduce, YARN (MRv2), and a broad overview of the Hadoop ecosystem including Hive, Pig, HBase, ZooKeeper and more.