CONVINCE SIT-AW-AIP

This project allows to use a VLM for anomaly identification. The deployment of the VLM is explained and the way to communicate with the model, given CONVINCE use cases. For now only UC1, vacuum cleaner, and UC2, assembly robot. The last section explains how to custom the communication.

The tests have been done only on LINUX.

Contents

• Generic Principles
• Requirements: Install docker
  • Uninstall unofficial packages
  • Set up
  • Install packages
  • Start docker services if disabled
  • Running without sudo
  • Verify installation
• VLM deployment
  • Install the deploy part
    • Optional - Clone only the corresponding folder
    • Else clone the whole project and consider only vLLM-hosting folder
  • Run and build
    • Then execute bash file
    • You may need to give execute permissions (on Linux)
    • If your are not using bash you can directly build then run the docker compose in your terminal
    • The .env file allows you to change some parameters
• Inference with VLM
  • Installations
    • Clone project
    • Build your project - Once
    • Activate virtual env - everytime you enter the project
  • Change environment variables
  • Format data - generate json (Once on a desired batch of data)
    • Variables
    • UC1
    • UC2
  • Send an identification request to the VLM server
    • Variables
• Customized UC – Goes through coding
  • First : Format your data
  • Second : Message to send to VLM
  • Third : Write your prompt
  • Fourth : Add your class