[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Can we allow SimpleFunction and SerializableFunction to throw Exception?

Compilation compatibility is a big part of what Beam aims to provide with its guarantees. Romain makes a good point that most users are not invoking SeralizableFunctions themselves (they are usually invoked inside of Beam classes such as MapElements), however I suspect some users do these things.

On Sun, Oct 14, 2018 at 2:38 PM Kenneth Knowles <kenn@xxxxxxxxxx> wrote:
Romain has brought up two good aspects of backwards compatibility:

 - runtime replacement vs recompile
 - consumer (covariant) vs producer (contravariant, such as interfaces a user implements)

In this case, I think the best shoice is covariant and contravariant (invariant) backwards compat including recompile compat. But we shouldn't assume there is one obvious definition of "backwards compatibility".

Does it help to introduce a new functional interface?


On Sun, Oct 14, 2018 at 6:25 AM Romain Manni-Bucau <rmannibucau@xxxxxxxxx> wrote:
Beam does not catch Exception for function usage so it will have to do it in some places.

A user does not have to execute the function so worse case it impacts tests and in any case the most important: it does not impact the user until it recompiles the code (= runtime is not impacted).

Romain Manni-Bucau
@rmannibucau |  Blog | Old BlogGithub | LinkedIn | Book

Le dim. 14 oct. 2018 à 15:19, Reuven Lax <relax@xxxxxxxxxx> a écrit :
What in Beam codebase is not ready, and how do we know that user code doesn't have the same issue?

On Sun, Oct 14, 2018 at 6:04 AM Romain Manni-Bucau <rmannibucau@xxxxxxxxx> wrote:
Hmm, tested also and it works until something in the codeflow does not respect that constraint - see com.sun.tools.javac.comp.Flow.FlowAnalyzer#errorUncaught. In other words beam codebase is not ready for that and will make it fail but it is ok cause we can fix it but user code does not rely on that so it is fine to update it normally.

Romain Manni-Bucau
@rmannibucau |  Blog | Old BlogGithub | LinkedIn | Book

Le dim. 14 oct. 2018 à 14:39, Reuven Lax <relax@xxxxxxxxxx> a écrit :
Just tried it, doesn't appear to work :(

error: unreported exception Exception; must be caught or declared to be thrown

On Sun, Oct 14, 2018 at 1:55 AM Romain Manni-Bucau <rmannibucau@xxxxxxxxx> wrote:
not with java>=8 AFAIK

Romain Manni-Bucau
@rmannibucau |  Blog | Old BlogGithub | LinkedIn | Book

Le dim. 14 oct. 2018 à 10:49, Reuven Lax <relax@xxxxxxxxxx> a écrit :
But it means that other functions that call SerializableFunctions must now declare exceptions, right? If yes, this is incompatible.

On Sun, Oct 14, 2018 at 1:37 AM Romain Manni-Bucau <rmannibucau@xxxxxxxxx> wrote:
No, only parameter types and return type is used to lookup methods.

Romain Manni-Bucau
@rmannibucau |  Blog | Old BlogGithub | LinkedIn | Book

Le dim. 14 oct. 2018 à 09:45, Reuven Lax <relax@xxxxxxxxxx> a écrit :
I've run into this problem before as well. Doesn't changing the signature involve a backwards-incompatible change though?

On Wed, Oct 3, 2018 at 5:11 PM Jeff Klukas <jklukas@xxxxxxxxxxx> wrote:
I'm working on https://issues.apache.org/jira/browse/BEAM-5638 to add exception handling options to single message transforms in the Java SDK.

MapElements' via() method is overloaded to accept either a SimpleFunction, a SerializableFunction, or a Contextful, all of which are ultimately stored as a Contextful where the mapping functionis expected to have signature:

OutputT apply(InputT element, Context c) throws Exception;

So Contextful.Fn allows throwing checked exceptions, but neither SerializableFunction nor SimpleFunction do. The user-provided function has to satisfy the more restrictive signature:

OutputT apply(InputT input);

Is there background about why we allow arbitrary checked exceptions to be thrown in one case but not the other two? Could we consider expanding SerializableFunction and SimpleFunction to the following?:

OutputT apply(InputT input) throws Exception;

This would, for example, simplify the implementation of ParseJsons and AsJsons, where we have to catch an IOException in MapElements#via only to rethrow as RuntimeException.