osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: docker, error NoResourceAvailableException..


Thanks Dominik, I will try that.

On Wed, Aug 15, 2018 at 3:10 AM, Dominik Wosiński <wossyn@xxxxxxxxx> wrote:

Hey,
The problem is that your command does start Job Manager container, but it does not start the Task Manager . That is why you have 0 slots. Currently, the default numberOfTaskSlots is set to the number of CPUs avaialbe on the machine.


So, You generally can to do 2 things:


1) Start Job Manager and 2 Task Managers. If you have Docker Compose available, you can paste this to your docker-compose.yml :

services:
  jobmanager:
    image:
${FLINK_DOCKER_IMAGE_NAME:-flink}
   
expose:
     
- "6123"
   
ports:
     
- "8081:8081"
   
command: jobmanager
   
environment:
     
- JOB_MANAGER_RPC_ADDRESS=jobmanager

 
taskmanager:
    image:
${FLINK_DOCKER_IMAGE_NAME:-flink}
   
expose:
     
- "6121"
     
- "6122"
   
depends_on:
     
- jobmanager
   
command: taskmanager
   
links:
     
- "jobmanager:jobmanager"
   
environment:
     
- JOB_MANAGER_RPC_ADDRESS=jobmanager

 
taskmanager1:
    image:
${FLINK_DOCKER_IMAGE_NAME:-flink}
   
expose:
     
- "6190"
     
- "6120"
   
depends_on:
     
- jobmanager
   
command: taskmanager
   
links:
     
- "jobmanager:jobmanager"
   
environment:
     
- JOB_MANAGER_RPC_ADDRESS=jobmanager

 

This will give you 1 Job Manager and 2 Task Managers with one task slot each, so 2 Task slots in general.

2) You can deploy 1 Job Manager and 1 Task Manager.Then you need to modify flink-conf.yml and set the following setting :

taskmanager.numberOfTaskSlots: 2


This will give you 2 Task Slots with only 1 Task Manager. But you will need to somehow override config in the container, possibly using : https://docs.docker.com/storage/volumes/

Regards,
Dominik.

Od: shyla deshpande
Wysłano: środa, 15 sierpnia 2018 07:23
Do: user
Temat: docker, error NoResourceAvailableException..

 

Hello all,

 

Trying to use docker as a single node flink cluster.

 

docker run --name flink_local -p 8081:8081 -t flink local

 

I submited a job to the cluster using the Web UI. The job failed. I see this error message in the docker logs.

 

org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Could not allocate all requires slots within timeout of 300000 ms. Slots required: 2, slots allocated: 0

 

The Web UI, shows 0 taskmanagers and 0 task slots on the Flink dashboard.

How do I start the docker with 2 Task slots?

 

Appreciate any help.

 

Thanks