Docker image size restritions

[saxtouri]

This issue is critical from an infrastructure perspective. Please, let me know what you think:

Some of the docker images are unexpectedly large (e.g., 7-8 GBs or more). Because of that:

1. the system fails to execute these jobs due to timeout
2. even if we override the timeout issue by pre-downloading the huge image, the cluster nodes get full very quickly, rendering the system unusable

I think we should reconsider our policy on the docker image sizes and set size restrictions. If we don't, we will experience a lot of failures and our storage resources will get consumes really fast. Also, if we run the system on a commercial cluster, the cost of the infrastructure will rise too high very quickly and it will be nearly impossible to scale.

I wonder how images get so big. What kind of software code requires so many GBs, considering that desktop OSs overloaded with heavy apps can be bundled in just 5 to 10 GBs. Is it just software or do we allow people to load images with data as well?

Putting any kind of data inside the images is a highly inefficient practice because it makes our system too expensive to run (I can explain this further, if you like).

What do you think we should do?

[dumitrescustefan]

Hi, I actually asked something like that on the forum. For example we provide a multilingual tool to perform tokenization, sentence splitting, POS and parsing on 50+ languages. Even if the neural network model itself has ~50-100MB per language, we still need vector embeddings to perform at state-of-the-art levels. For example fasttext embeddings have around 0.5-1.0 GB per language. Times 50, that makes a docker file over 50GB, say the least.

For the current version of the component we removed vector embedding completely and we loose ~1-2% accuracy, at a 10x size reduction. So the size question is very valid.

So we either provide one docker image per language (including vector embeddings should max at around 2GB), or a single image as now at about 5GB without vectors.

[greenwoodma]

The problem is that we have a policy of including all the data with the component as it is the only way to ensure reproducability. If we allowed images to download or otherwise access remote data then a) we'd need to allow possibly arbitrary internet access which is a security risk we didn't want to take and b) we would not be able to guarantee reproducability which completely negates one of the main selling points of the platform.

The current trend in TDM seems to be deep learning and that often results in huge models which need to be packaged with the components. If we want to support all forms of TDM then we need to support this, which will require either large docker images (components and data together) or the data would need to be uploaded to the registry as a separate OMTD resource and then passed to the docker image somehow. Would this second approach be better from a resource perspective @saxtouri? I would imagine it would be technically more difficult as you'd need to download the resource as well as the docker image and then make the resource available inside the running docker instance.

@dumitrescustefan in your case I would suggest one docker image per language anyway, because once as far as I know once the docker image is registered as an application you won't be able to change the language param (assuming it's a param that chooses which language to use) so you'd have to register the full image for each language anyway which would be wasteful. Instead you could register the per language images with the full embeddings and still have reasonably small images for download.

[reckart]

If the resources are downloaded from a trusted repository (e.g. a trusted Maven repository) and not just from an arbitrary URL, IMHO it is a valid approach.

[greenwoodma]

@reckart true but there are limits to how much some of those repos allow you to upload; maven central doesn't have a fixed limit but they do state they will look at any very large uploads and may remove them. After all they are a code repository not a data repository. We'd also need to ensure that any repo we did support had the same limits as maven central; i.e. once uploaded the item is fixed and cannot be changed or deleted otherwise we have the same issue about reproducability.

If we take this approach though we then get into another discussion. How is bandwidth into the cluster charged? It may not be but commercial cloud offerings would probably charge for that, and so downloading huge models, possible each time we wanted to run a component (given that the docker images are static etc.) could also run up huge costs, and could be very slow and fragile depending on the connection etc.

[reckart]

True, there are strings attached and further considerations to be taken. Mind that e.g. the NER models provided by the FREME EU project which can be used via DKPro Core are also ~1 GB per language. So we have precedence. The models are hosted on the UKP Maven repo.