OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: InvalidPayloadException when sorting lines in a csv file


Thanks Claus!

Both options work.


For some reason the files are stored in a .camel directory also, and an exception is thrown for each file because it cannot be deleted.

This is not a problem as i clean out the working directory anyway.



2018-06-08 01:27:38 WARN  verne-prophet org.apache.camel.component.file.GenericFileOnCompletion -   -  - Error during commit. Exchange[ID-LP0641-58359-1528457245685-0-11186]. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Error renaming file from C:\Users\ivana.cace\workspace\camel-verne-prophet-splitter\data\output\2018\05-mei\Prophet\PRD_Base_shortest_NW_9605_1_10.rpt to data\output\2018\05-mei\Prophet\.camel\PRD_Base_shortest_NW_9605_1_10.rpt]
org.apache.camel.component.file.GenericFileOperationFailedException: Error renaming file from C:\Users\ivana.cace\workspace\camel-verne-prophet-splitter\data\output\2018\05-mei\Prophet\PRD_Base_shortest_NW_9605_1_10.rpt to data\output\2018\05-mei\Prophet\.camel\PRD_Base_shortest_NW_9605_1_10.rpt
 at org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:81)
 at org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:113)
 at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:88)
 at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:127)
 at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
 at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:57)
 at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:104)
 at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:230)
 at org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:65)
 at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:674)
 at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:629)
 at org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:246)
 at org.apache.camel.processor.Pipeline.process(Pipeline.java:109)
 at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
 at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:460)
 at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:227)
 at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:191)
 at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
 at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Renaming file from 'C:\Users\ivana.cace\workspace\camel-verne-prophet-splitter\data\output\2018\05-mei\Prophet\PRD_Base_shortest_NW_9605_1_10.rpt' to 'data\output\2018\05-mei\Prophet\.camel\PRD_Base_shortest_NW_9605_1_10.rpt' failed: Cannot delete file 'C:\Users\ivana.cace\workspace\camel-verne-prophet-splitter\data\output\2018\05-mei\Prophet\PRD_Base_shortest_NW_9605_1_10.rpt' after copy succeeded
 at org.apache.camel.util.FileUtil.renameFileUsingCopy(FileUtil.java:510)
 at org.apache.camel.util.FileUtil.renameFile(FileUtil.java:482)
 at org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:78)
 ... 25 common frames omitted


________________________________
Van: Claus Ibsen <claus.ibsen@xxxxxxxxx>
Verzonden: vrijdag 8 juni 2018 12:12:27
Aan: users@xxxxxxxxxxxxxxxx
Onderwerp: Re: InvalidPayloadException when sorting lines in a csv file

Hi

Ah okay its not a String type but an ArrayList, which is the output of
the Sort EIP.

a)
You can try adding

.convertBodyTo(String.class)

before the to("file:xxx")


b)
If (a) does not work (I frankly dont think so) then you can split it,
via splitter EIP then each item in the array list can be appended to
the file.


.split(body())
   .to("file:xxx")




On Fri, Jun 8, 2018 at 11:34 AM, Ivana Cace <ivana.cace@xxxxxxxxx> wrote:
> Hi Claus,
>
>
> I'm using camel version 2.28 with Spring Boot 1.5.10.RELEASE
>
>
> Stacktrace:
>
>
> 2018-06-08 10:04:09,976 ERROR [Camel (camel-1) thread #1 - file://data/output/2018/05-mei/Prophet] org.apache.camel.processor.DefaultErrorHandler -  -  - Failed delivery for (MessageId: ID-LP0641-62840-1528445046486-0-375 on ExchangeId: ID-LP0641-62840-1528445046486-0-378). Exhausted after delivery attempt: 1 caught: org.apache.camel.component.file.GenericFileOperationFailedException: Cannot store file: data\output\2018\05-mei\Prophet\sorted\PRD_Base_shortest_HELICA.rpt
> Message History
> ---------------------------------------------------------------------------------------------------------------------------------------
> RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
> [route2            ] [route2            ] [file://data/output/2018/05-mei/Prophet?filterFile=%24%7Bfile%3Asize%7D+%3C+600] [         8]
> [route2            ] [log4              ] [log                                                                           ] [         0]
> [route2            ] [sort1             ] [sort[{tokenize(simple{Simple: ${body}},
> )}]                                  ] [         1]
> [route2            ] [to8               ] [file:data/output/2018/05-mei/Prophet/sorted?fileExist=Append                  ] [         5]
> Stacktrace
> --------------------------------------------------------------------------------------------------------------------------------------- org.apache.camel.component.file.GenericFileOperationFailedException: Cannot store file: data\output\2018\05-mei\Prophet\sorted\PRD_Base_shortest_HELICA.rpt
>  at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:346)
>  at org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:277)
>  at org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:165)
>  at org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:79)
>  at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>  at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:145)
>  at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
>  at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542)
>  at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
>  at org.apache.camel.processor.Pipeline.process(Pipeline.java:120)
>  at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)
>  at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
>  at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:460)
>  at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:227)
>  at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:191)
>  at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
>  at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>  at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>  at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.camel.InvalidPayloadException: No body available of type: java.io.InputStream but has value: [!1,SPCODE,MP_Product_KEY,MP_Weight,MP_Policy_ID,MP_Policy_Number,MP_Status_KEY,MP_Policy_Start_DT,MP_Policy_End_DT,MP_Premium_End_DT,MP_Premium_FRQ,MP_Date_Of_Birth_1st_DT,MP_Gender_1st_KEY,MP_Age_Corr_1st_MNTHS,MP_Insured_1st_ID,MP_Interest_PRC,MP_Benefits_Bas_FRQ,MP_Benefits_Exp_FRQ,MP_Benefits_Bas,MP_Benefits_Exp,MP_Benefits_Prf,MP_Benefits_Terminal_Withdrawn,MP_Prem_Gross_Rgl_Sum_Paid,MP_Prem_Gross_Rgl,MP_Prem_Net_Bas_Rgl,MP_Prem_Net_Exp_Rgl,MP_Prem_Load_Ini_Rgl,MP_Prem_Load_Coll_Rgl,MP_Prem_Load_Term_Rgl,MP_Prem_Load_Solv_Rgl,MP_Prem_Load_Disc_Rgl,MP_Prem_Net_Bas_Corr_Rgl,MP_Prem_Gross_Sgl_Sum_Paid,MP_Prem_Gross_Sgl,MP_Prem_Net_Bas_Sgl,MP_Prem_Net_Exp_Sgl,MP_Prem_Load_Ini_Sgl,MP_Prem_Load_Coll_Sgl,MP_Prem_Load_Term_Sgl,MP_Prem_Load_Solv_Sgl,MP_Premiums_Paid_Known_YN,MP_Keuzerecht_KEY,MP_Ledenvoordeel_PRC,MP_Action_Code_KEY,MP_Status_Current_DT
> ] of type: java.util.ArrayList on: PRD_Base_shortest_HELICA.rpt. Caused by: Error during type conversion from type: java.lang.String to the required type: java.io.InputStream with value !1,SPCODE,MP_Product_KEY,MP_Weight,MP_Policy_ID,MP_Policy_Number,MP_Status_KEY,MP_Policy_Start_DT,MP_Policy_End_DT,MP_Premium_End_DT,MP_Premium_FRQ,MP_Date_Of_Birth_1st_DT,MP_Gender_1st_KEY,MP_Age_Corr_1st_MNTHS,MP_Insured_1st_ID,MP_Interest_PRC,MP_Benefits_Bas_FRQ,MP_Benefits_Exp_FRQ,MP_Benefits_Bas,MP_Benefits_Exp,MP_Benefits_Prf,MP_Benefits_Terminal_Withdrawn,MP_Prem_Gross_Rgl_Sum_Paid,MP_Prem_Gross_Rgl,MP_Prem_Net_Bas_Rgl,MP_Prem_Net_Exp_Rgl,MP_Prem_Load_Ini_Rgl,MP_Prem_Load_Coll_Rgl,MP_Prem_Load_Term_Rgl,MP_Prem_Load_Solv_Rgl,MP_Prem_Load_Disc_Rgl,MP_Prem_Net_Bas_Corr_Rgl,MP_Prem_Gross_Sgl_Sum_Paid,MP_Prem_Gross_Sgl,MP_Prem_Net_Bas_Sgl,MP_Prem_Net_Exp_Sgl,MP_Prem_Load_Ini_Sgl,MP_Prem_Load_Coll_Sgl,MP_Prem_Load_Term_Sgl,MP_Prem_Load_Solv_Sgl,MP_Premiums_Paid_Known_YN,MP_Keuzerecht_KEY,MP_Ledenvoordeel_PRC,MP_Action_Code_KEY,MP_Status_Current_DT
>  due Failed to convert from type [java.util.ArrayList<?>] to type [java.io.InputStream] for value '[!1,SPCODE,MP_Product_KEY,MP_Weight,MP_Policy_ID,MP_Policy_Number,MP_Status_KEY,MP_Policy_Start_DT,MP_Policy_End_DT,MP_Premium_End_DT,MP_Premium_FRQ,MP_Date_Of_Birth_1st_DT,MP_Gender_1st_KEY,MP_Age_Corr_1st_MNTHS,MP_Insured_1st_ID,MP_Interest_PRC,MP_Benefits_Bas_FRQ,MP_Benefits_Exp_FRQ,MP_Benefits_Bas,MP_Benefits_Exp,MP_Benefits_Prf,MP_Benefits_Terminal_Withdrawn,MP_Prem_Gross_Rgl_Sum_Paid,MP_Prem_Gross_Rgl,MP_Prem_Net_Bas_Rgl,MP_Prem_Net_Exp_Rgl,MP_Prem_Load_Ini_Rgl,MP_Prem_Load_Coll_Rgl,MP_Prem_Load_Term_Rgl,MP_Prem_Load_Solv_Rgl,MP_Prem_Load_Disc_Rgl,MP_Prem_Net_Bas_Corr_Rgl,MP_Prem_Gross_Sgl_Sum_Paid,MP_Prem_Gross_Sgl,MP_Prem_Net_Bas_Sgl,MP_Prem_Net_Exp_Sgl,MP_Prem_Load_Ini_Sgl,MP_Prem_Load_Coll_Sgl,MP_Prem_Load_Term_Sgl,MP_Prem_Load_Solv_Sgl,MP_Premiums_Paid_Known_YN,MP_Keuzerecht_KEY,MP_Ledenvoordeel_PRC,MP_Action_Code_KEY,MP_Status_Current_DT
> ]'; nested exception is org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.String] to type [java.io.InputStream]. Exchange[ID-LP0641-62840-1528445046486-0-378]. Caused by: [org.apache.camel.TypeConversionException - Error during type conversion from type: java.lang.String to the required type: java.io.InputStream with value !1,SPCODE,MP_Product_KEY,MP_Weight,MP_Policy_ID,MP_Policy_Number,MP_Status_KEY,MP_Policy_Start_DT,MP_Policy_End_DT,MP_Premium_End_DT,MP_Premium_FRQ,MP_Date_Of_Birth_1st_DT,MP_Gender_1st_KEY,MP_Age_Corr_1st_MNTHS,MP_Insured_1st_ID,MP_Interest_PRC,MP_Benefits_Bas_FRQ,MP_Benefits_Exp_FRQ,MP_Benefits_Bas,MP_Benefits_Exp,MP_Benefits_Prf,MP_Benefits_Terminal_Withdrawn,MP_Prem_Gross_Rgl_Sum_Paid,MP_Prem_Gross_Rgl,MP_Prem_Net_Bas_Rgl,MP_Prem_Net_Exp_Rgl,MP_Prem_Load_Ini_Rgl,MP_Prem_Load_Coll_Rgl,MP_Prem_Load_Term_Rgl,MP_Prem_Load_Solv_Rgl,MP_Prem_Load_Disc_Rgl,MP_Prem_Net_Bas_Corr_Rgl,MP_Prem_Gross_Sgl_Sum_Paid,MP_Prem_Gross_Sgl,MP_Prem_Net_Bas_Sgl,MP_Prem_Net_Exp_Sgl,MP_Prem_Load_Ini_Sgl,MP_Prem_Load_Coll_Sgl,MP_Prem_Load_Term_Sgl,MP_Prem_Load_Solv_Sgl,MP_Premiums_Paid_Known_YN,MP_Keuzerecht_KEY,MP_Ledenvoordeel_PRC,MP_Action_Code_KEY,MP_Status_Current_DT
>  due Failed to convert from type [java.util.ArrayList<?>] to type [java.io.InputStream] for value '[!1,SPCODE,MP_Product_KEY,MP_Weight,MP_Policy_ID,MP_Policy_Number,MP_Status_KEY,MP_Policy_Start_DT,MP_Policy_End_DT,MP_Premium_End_DT,MP_Premium_FRQ,MP_Date_Of_Birth_1st_DT,MP_Gender_1st_KEY,MP_Age_Corr_1st_MNTHS,MP_Insured_1st_ID,MP_Interest_PRC,MP_Benefits_Bas_FRQ,MP_Benefits_Exp_FRQ,MP_Benefits_Bas,MP_Benefits_Exp,MP_Benefits_Prf,MP_Benefits_Terminal_Withdrawn,MP_Prem_Gross_Rgl_Sum_Paid,MP_Prem_Gross_Rgl,MP_Prem_Net_Bas_Rgl,MP_Prem_Net_Exp_Rgl,MP_Prem_Load_Ini_Rgl,MP_Prem_Load_Coll_Rgl,MP_Prem_Load_Term_Rgl,MP_Prem_Load_Solv_Rgl,MP_Prem_Load_Disc_Rgl,MP_Prem_Net_Bas_Corr_Rgl,MP_Prem_Gross_Sgl_Sum_Paid,MP_Prem_Gross_Sgl,MP_Prem_Net_Bas_Sgl,MP_Prem_Net_Exp_Sgl,MP_Prem_Load_Ini_Sgl,MP_Prem_Load_Coll_Sgl,MP_Prem_Load_Term_Sgl,MP_Prem_Load_Solv_Sgl,MP_Premiums_Paid_Known_YN,MP_Keuzerecht_KEY,MP_Ledenvoordeel_PRC,MP_Action_Code_KEY,MP_Status_Current_DT
> ]'; nested exception is org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.String] to type [java.io.InputStream]]
>  at org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:107)
>  at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:325)
>  ... 23 common frames omitted
>
>
> ________________________________
> Van: Claus Ibsen <claus.ibsen@xxxxxxxxx>
> Verzonden: vrijdag 8 juni 2018 10:45:41
> Aan: users@xxxxxxxxxxxxxxxx
> Onderwerp: Re: InvalidPayloadException when sorting lines in a csv file
>
> Hi
>
> What version of Camel do you use?
>
> And frankly Camel ought to be able to write a String body to a file,
> so that is a bit surprising. Maybe you can post your stacktrace for us
> to look at?
>
> On Fri, Jun 8, 2018 at 10:14 AM, Ivana Cace <ivana.cace@xxxxxxxxx> wrote:
>> Hi,
>>
>>
>> I'm trying to sort a csv file, but when I try to write the file i get a InvalidPayloadException due to "Error during type conversion from type: java.lang.String to the required type: java.io.InputStream".
>>
>>
>> Here is the route:
>>
>>
>> from("file:path?filterFile=${file:size} < 6000")
>>          .sort(body().tokenize("\n"), new ColumnSortComperator(1, ','))
>>          .to("file:path/sorted?fileExist=Append")
>>          .end();
>>
>> As far as i understand, the tokenizer outputs String, but to write the line to the file i need a Inputstream.
>> How do i make a Inputstream from the sorted lines?
>> Is the only way to implement a processor that stuffs the lines + line endings into the exchange body?
>>
>>
>> thanks in advance,
>>
>>
>> Ivana
>>
>>
>> ***************************************** De informatie verzonden in deze e-mail (inclusief bijlagen) kan vertrouwelijk van aard zijn en is uitsluitend bestemd voor de geadresseerde(n). Indien deze e-mail niet voor u bestemd is, verzoeken wij u vriendelijk het bericht te retourneren en vervolgens te vernietigen. Yarden doet er alles aan om ervoor te zorgen dat verzonden e-mailberichten vrij zijn van virussen, maar kan dit niet garanderen. Yarden Holding B.V., statutair gevestigd te Almere, Handelsregisternummer 34155452 Yarden Uitvaartfaciliteiten B.V., statutair gevestigd te Almere, Handelsregisternummer 30104432 Yarden Uitvaartzorg B.V., statutair gevestigd te Almere, Handelsregisternummer 04050164 Yarden Uitvaartverzekeringen N.V., statutair gevestigd te Almere, Handelsregisternummer 33244450 *****************************************
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
> ***************************************** De informatie verzonden in deze e-mail (inclusief bijlagen) kan vertrouwelijk van aard zijn en is uitsluitend bestemd voor de geadresseerde(n). Indien deze e-mail niet voor u bestemd is, verzoeken wij u vriendelijk het bericht te retourneren en vervolgens te vernietigen. Yarden doet er alles aan om ervoor te zorgen dat verzonden e-mailberichten vrij zijn van virussen, maar kan dit niet garanderen. Yarden Holding B.V., statutair gevestigd te Almere, Handelsregisternummer 34155452 Yarden Uitvaartfaciliteiten B.V., statutair gevestigd te Almere, Handelsregisternummer 30104432 Yarden Uitvaartzorg B.V., statutair gevestigd te Almere, Handelsregisternummer 04050164 Yarden Uitvaartverzekeringen N.V., statutair gevestigd te Almere, Handelsregisternummer 33244450 *****************************************



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

***************************************** De informatie verzonden in deze e-mail (inclusief bijlagen) kan vertrouwelijk van aard zijn en is uitsluitend bestemd voor de geadresseerde(n). Indien deze e-mail niet voor u bestemd is, verzoeken wij u vriendelijk het bericht te retourneren en vervolgens te vernietigen. Yarden doet er alles aan om ervoor te zorgen dat verzonden e-mailberichten vrij zijn van virussen, maar kan dit niet garanderen. Yarden Holding B.V., statutair gevestigd te Almere, Handelsregisternummer 34155452 Yarden Uitvaartfaciliteiten B.V., statutair gevestigd te Almere, Handelsregisternummer 30104432 Yarden Uitvaartzorg B.V., statutair gevestigd te Almere, Handelsregisternummer 04050164 Yarden Uitvaartverzekeringen N.V., statutair gevestigd te Almere, Handelsregisternummer 33244450 *****************************************