9585a3e7-0171-1000-54bb-76ca2da624ec
Splitting Large File To Smaller File Example
f7323462-6796-3e9b-0000-000000000000
fa748cf5-9ec9-30e7-0000-000000000000
0.0
0.0
007c616c-1963-3982-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
1 GB
10000
f7323462-6796-3e9b-0000-000000000000
8c50a6d4-3057-38fc-0000-000000000000
PROCESSOR
0 sec
1
DO_NOT_COMPRESS
LOAD_BALANCE_NOT_CONFIGURED
DO_NOT_LOAD_BALANCE
splits
0
104ed780-4e83-399e-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
1 GB
10000
f7323462-6796-3e9b-0000-000000000000
e090a445-aaac-34bf-0000-000000000000
FUNNEL
0 sec
1
DO_NOT_COMPRESS
LOAD_BALANCE_NOT_CONFIGURED
DO_NOT_LOAD_BALANCE
splits
0
43e83912-97ee-3775-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
1 GB
10000
f7323462-6796-3e9b-0000-000000000000
4daef289-5527-305f-0000-000000000000
PROCESSOR
0 sec
1
DO_NOT_COMPRESS
LOAD_BALANCE_NOT_CONFIGURED
DO_NOT_LOAD_BALANCE
splits
0
62ab8ad5-4627-38e2-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
1 GB
10000
f7323462-6796-3e9b-0000-000000000000
4a078634-c420-3297-0000-000000000000
PROCESSOR
0 sec
1
DO_NOT_COMPRESS
LOAD_BALANCE_NOT_CONFIGURED
DO_NOT_LOAD_BALANCE
merged
0
6b24bcaf-5ef5-3290-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
1 GB
10000
f7323462-6796-3e9b-0000-000000000000
dd11244d-58ff-3517-0000-000000000000
PROCESSOR
0 sec
1
DO_NOT_COMPRESS
LOAD_BALANCE_NOT_CONFIGURED
DO_NOT_LOAD_BALANCE
success
0
370c73db-3932-3160-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
nifi-record-serialization-services-nar
org.apache.nifi
1.11.3
schema-access-strategy
schema-access-strategy
schema-registry
org.apache.nifi.schemaregistry.services.SchemaRegistry
schema-registry
schema-name
schema-name
schema-version
schema-version
schema-branch
schema-branch
schema-text
schema-text
csv-reader-csv-parser
csv-reader-csv-parser
Date Format
Date Format
Time Format
Time Format
Timestamp Format
Timestamp Format
CSV Format
CSV Format
Value Separator
Value Separator
Skip Header Line
Skip Header Line
ignore-csv-header
ignore-csv-header
Quote Character
Quote Character
Escape Character
Escape Character
Comment Marker
Comment Marker
Null String
Null String
Trim Fields
Trim Fields
csvutils-character-set
csvutils-character-set
CSVReader_InferSchema
false
schema-access-strategy
schema-registry
schema-name
schema-version
schema-branch
schema-text
csv-reader-csv-parser
Date Format
Time Format
Timestamp Format
CSV Format
Value Separator
Skip Header Line
ignore-csv-header
Quote Character
Escape Character
Comment Marker
Null String
Trim Fields
csvutils-character-set
ENABLED
org.apache.nifi.csv.CSVReader
8725e7dd-08e1-3029-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
nifi-record-serialization-services-nar
org.apache.nifi
1.11.3
Schema Write Strategy
Schema Write Strategy
schema-cache
org.apache.nifi.serialization.RecordSchemaCacheService
schema-cache
schema-access-strategy
schema-access-strategy
schema-registry
org.apache.nifi.schemaregistry.services.SchemaRegistry
schema-registry
schema-name
schema-name
schema-version
schema-version
schema-branch
schema-branch
schema-text
schema-text
Date Format
Date Format
Time Format
Time Format
Timestamp Format
Timestamp Format
CSV Format
CSV Format
Value Separator
Value Separator
Include Header Line
Include Header Line
Quote Character
Quote Character
Escape Character
Escape Character
Comment Marker
Comment Marker
Null String
Null String
Trim Fields
Trim Fields
Quote Mode
Quote Mode
Record Separator
Record Separator
Include Trailing Delimiter
Include Trailing Delimiter
csvutils-character-set
csvutils-character-set
CSVRecordSetWriter_InferSchema
false
Schema Write Strategy
schema-cache
schema-access-strategy
schema-registry
schema-name
schema-version
schema-branch
schema-text
Date Format
Time Format
Timestamp Format
CSV Format
Value Separator
Include Header Line
Quote Character
Escape Character
Comment Marker
Null String
Trim Fields
Quote Mode
Record Separator
Include Trailing Delimiter
csvutils-character-set
ENABLED
org.apache.nifi.csv.CSVRecordSetWriter
e090a445-aaac-34bf-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
1216.0
296.0
d9cbb7c0-7489-3cd8-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
-56.0
-120.0
464.0
544.0
0d4d863b-0791-37ba-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
24.0
-56.0
nifi-standard-nar
org.apache.nifi
1.11.3
WARN
1
File Size
File Size
Batch Size
Batch Size
Data Format
Data Format
Unique FlowFiles
Unique FlowFiles
generate-ff-custom-text
generate-ff-custom-text
character-set
character-set
ALL
false
30 sec
File Size
1KB
Batch Size
1
Data Format
Text
Unique FlowFiles
false
generate-ff-custom-text
01,john,smith,5000
02,harry,potter,3000
03,amir,khaan,5000
04,selenia,williams,3000
05,john,abraham,6000
character-set
UTF-8
0
0 sec
TIMER_DRIVEN
1 sec
false
GenerageFiles
false
success
STOPPED
org.apache.nifi.processors.standard.GenerateFlowFile
4a078634-c420-3297-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
576.0
-112.0
nifi-standard-nar
org.apache.nifi
1.11.3
WARN
1
Record Reader
org.apache.nifi.serialization.RecordReaderFactory
Record Reader
Record Writer
org.apache.nifi.serialization.RecordSetWriterFactory
Record Writer
Records Per Split
Records Per Split
ALL
false
30 sec
Record Reader
370c73db-3932-3160-0000-000000000000
Record Writer
8725e7dd-08e1-3029-0000-000000000000
Records Per Split
1000
0
0 sec
TIMER_DRIVEN
1 sec
false
10,000 to 1000
true
failure
true
original
false
splits
STOPPED
org.apache.nifi.processors.standard.SplitRecord
4daef289-5527-305f-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
576.0
88.0
nifi-standard-nar
org.apache.nifi
1.11.3
WARN
1
Record Reader
org.apache.nifi.serialization.RecordReaderFactory
Record Reader
Record Writer
org.apache.nifi.serialization.RecordSetWriterFactory
Record Writer
Records Per Split
Records Per Split
ALL
false
30 sec
Record Reader
370c73db-3932-3160-0000-000000000000
Record Writer
8725e7dd-08e1-3029-0000-000000000000
Records Per Split
100
0
0 sec
TIMER_DRIVEN
1 sec
false
1000 to 100
true
failure
true
original
false
splits
STOPPED
org.apache.nifi.processors.standard.SplitRecord
8c50a6d4-3057-38fc-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
592.0
280.0
nifi-standard-nar
org.apache.nifi
1.11.3
WARN
1
Record Reader
org.apache.nifi.serialization.RecordReaderFactory
Record Reader
Record Writer
org.apache.nifi.serialization.RecordSetWriterFactory
Record Writer
Records Per Split
Records Per Split
ALL
false
30 sec
Record Reader
370c73db-3932-3160-0000-000000000000
Record Writer
8725e7dd-08e1-3029-0000-000000000000
Records Per Split
10
0
0 sec
TIMER_DRIVEN
1 sec
false
100 to 10
true
failure
true
original
false
splits
STOPPED
org.apache.nifi.processors.standard.SplitRecord
dd11244d-58ff-3517-0000-000000000000
f7323462-6796-3e9b-0000-000000000000
16.0
144.0
nifi-standard-nar
org.apache.nifi
1.11.3
WARN
1
record-reader
org.apache.nifi.serialization.RecordReaderFactory
record-reader
record-writer
org.apache.nifi.serialization.RecordSetWriterFactory
record-writer
merge-strategy
merge-strategy
correlation-attribute-name
correlation-attribute-name
Attribute Strategy
Attribute Strategy
min-records
min-records
max-records
max-records
min-bin-size
min-bin-size
max-bin-size
max-bin-size
max-bin-age
max-bin-age
max.bin.count
max.bin.count
ALL
false
30 sec
record-reader
370c73db-3932-3160-0000-000000000000
record-writer
8725e7dd-08e1-3029-0000-000000000000
merge-strategy
Bin-Packing Algorithm
correlation-attribute-name
Attribute Strategy
Keep Only Common Attributes
min-records
10000
max-records
10000
min-bin-size
0 B
max-bin-size
max-bin-age
max.bin.count
10
0
0 sec
TIMER_DRIVEN
1 sec
false
MergeRecord
true
failure
false
merged
true
original
STOPPED
org.apache.nifi.processors.standard.MergeRecord
Splitting Large Files Example
05/13/2020 04:48:46 ICT