Splitting Large Files to Smaller Files in NiFi

See the complete Video or scroll down to continue reading.

Thumb Rule is, You should always use split record processor to split your total records divided by 10. So for example, If you have a file with 1,000,000 records and you want final output with only 10 records per FlowFile. Then you should first split it to 100,000 and then split it to 10,000 and then split it to 1000 and then split it to 100 and then finally to 10 records.

Input FlowFile is having 10,000 records per FlowFile. This is processed by NiFi Split Record Processors and Splitted into from 10,000 records to 1,000 records, then to 100 records and then finally to 10 records per FlowFile

Final Output FlowFile is having only 10 Records Per FlowFile

Download The Template Here

Leave a Comment