OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

sys.exist(1) led to standalonesession daemon closed


Hi,

I used a scala library called scallop[1] to parse my job’s arguments. When the argument didn’t 
exist in the config setting, the default behavior of scallop would call sys.exit(1).

It is not a problem when I’m using flink cli to submit job. However, when I used rest api to submit 
job, it seems that sys.exit(1) will leads to standalonesession daemon closed. Maybe the reason is 
that rest server is also in the same process as standalonesession daemon. Am I correct?

If this is the root cause, is this an expected behavior and users should be aware of not using 
sys.exit(1) in their jobs?

I tested this on 1.6.0 standalone session cluster with flip-6 mode. And here are my testing job 
and logs before and after the submission.

package com.appier.rt.rt_match
import org.apache.flink.api.scala.createTypeInformation
import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
import org.rogach.scallop.{ScallopConf, ScallopOption}
object TestMain {
  def main(args: Array[String]): Unit = {
    object Args extends ScallopConf(args) {
      val mode: ScallopOption[String] = opt[String](default = Some("development"))
      verify
    }
    val env = StreamExecutionEnvironment.getExecutionEnvironment
    env.fromElements(Args.mode()).map(a => a)
    env.execute()
  }
}

Submit by flink-cli
$ ./bin/flink run -c com.appier.rt.rt_match.TestMain -p 2 -d rt-match-assembly-4.5.1-SNAPSHOT.jar --mo xyz
Starting execution of program
[scallop] Error: Unknown option 'mo'

Submit by rest-api
2018-11-05 13:27:58,800 TRACE org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Received request /jars/.
2018-11-05 13:27:59,679 TRACE org.apache.flink.runtime.rest.FileUploadHandler               - Received request. URL:/jobs/overview Method:GET
2018-11-05 13:27:59,680 TRACE org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler  - Received request /jobs/overview.
2018-11-05 13:28:01,752 TRACE org.apache.flink.runtime.rest.FileUploadHandler               - Received request. URL:/jars/ Method:GET
2018-11-05 13:28:01,753 TRACE org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Received request /jars/.
2018-11-05 13:28:02,682 TRACE org.apache.flink.runtime.rest.FileUploadHandler               - Received request. URL:/jobs/overview Method:GET
2018-11-05 13:28:02,683 TRACE org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler  - Received request /jobs/overview.
2018-11-05 13:28:03,899 TRACE org.apache.flink.runtime.rest.FileUploadHandler               - Received request. URL:/jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry class=com.appier.rt.rt_match.TestMain&parallelism=2&program-args=--mo+xyz Method:POST
2018-11-05 13:28:03,902 TRACE org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Received request /jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry class=com.appier.rt.rt_match.TestMain&parallelism=2&program-args=--mo+xyz.
2018-11-05 13:28:04,751 TRACE org.apache.flink.runtime.rest.FileUploadHandler               - Received request. URL:/jars/ Method:GET
2018-11-05 13:28:04,752 TRACE org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Received request /jars/.
2018-11-05 13:28:04,760 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Shutting down BLOB cache
2018-11-05 13:28:04,761 INFO  org.apache.flink.runtime.blob.BlobServer                      - Stopped BLOB server at 0.0.0.0:42075

Best,
Tony Wei.