Issues about custom code and workflow

Your environment

Version : 2019.12.04 State: Updated

We are in a LAB environment with an OBD simulator.
We provide 13.5V to the device with a power supply capable of providing current of 2A.
We have 4G Internet connection on the device.

Subject of the issue

Hello AutoPi team or the other interested individuals.

We are trying to find a development workflow to execute custom code with AutoPi.
Focusing the acc_manager service for trying.

Our goal is to setup a socket connection with socketio (python-socketio), grab the realtime acceleration sensor data with a returner and emit it to clients over wireless LAN.

To achieve this we have created a hook and attached this hook to interrupt_handler in acc_manager worker, we attached our returner to the hook which we have created.

We have two main issues here.

To make the returner update we have to sequentially

  • update our code in the returner
  • sync it
  • update the hook
  • sync it
  • update the worker process
  • sync it
  • IF the user interface asks us to restart the service
  • restart
  • THEN our new code in the returner updates sometimes and doesn’t update sometimes.

We usually have to reboot the device for the minion service to restart and run the updated returner in the end.
This workflow is not sustainable.

Our question is:
Is there a way to do this faster and more efficent ?
What we have tried as an alternative is to sync the returner code to the device
then try to restart minion service by calling autopi minionutils.request_restart or autopi minionutils.restart this never worked.
We either get an API not ready or if it restarts it doesn’t run the updated returner code which we have synced before.
We already have local and remote ssh connection to the device.

The ideal situation for us would be :

It would be much more efficent for us to work mostly in terminal and update necessary files then restart necessary services and log our output for debugging.
Passing through the cloud slows us down significantly.

We normally trace the logs /var/log/salt/minion or /var/log/salt, to debug our returner we append our output to a separate file and track it. It would be much nicer for us if the returner logs could be visible in /var/log/salt/minion, how to do that ?

The second main issue is about python versions.

When we want to install a third party package as python-socketio which python environment should we have to install it. I see that there are at least 4 different python versions running in the device. Is there a documentation about it ?

Our test code until now is this,
As a whole we have never managed to run it completely.

In customcode --> returner

#!/usr/bin/env python3
import logging
import sys

#Tries to import socketio
try:
    import socketio
except Exception:
    with open("/tmp/test.txt", "a") as file:
        file.write("Can not import\n")

#Check for the environment
with open("/tmp/test.txt", "a+") as file:
    file.write(sys.version+"\n")
    file.write(sys.path+"\n")
    file.write(sys.modules+"\n")

log = logging.getLogger(__name__)
sio = socketio.Client()

def returner(result):
    with open("/tmp/test.txt", "a") as file:
        file.write("{:}\n".format(result))
    sio.emit('autopi_acc', "{:}\n".format(result))

sio.connect('http://192.168.4.2:3000')

Hi Albrecht

Yes going through the cloud is mainly for management ie. after you finish development and want to push code to multiple devices, it should be done through the cloud.
But for development workflow, there is other alternative and much faster ways to do it.

1. Development on RPI with screen+keyboard or SSH

You could instead SSH into the device and edit the files directly.
When syncing from the cloud, the modules will be put in the following directories, based on the type.

  • /opt/autopi/modules
  • /opt/autopi/returners
  • /opt/autopi/utils
  • /opt/autopi/engines
  • /opt/autopi/states

So you can simply edit/create the files directly, but make sure to avoid clashes with the cloud synced modules, to prevent losing work if you sync the module from the cloud, as that will, without asking, overwrite the file on the device, even if you have made local changes to it.

When you make changes to the files, to make sure that the minion process picks up on the change, you can either restart the salt-minion service, or run this command saltutil.refresh_modules which reloads the external modules, ie. your code.

2. Remote development on dev machine

Alternatively, another workflow can be to have a script on your development machine that copies the files to the device via SCP, that way you can make the changes in your favorite editor. (Remember to have the script also either restart the minion service or run the refresh_modules command as noted above).

Python versions

The device has python2.7 and pyton3 installed, and python2.7 is the version it the salt-minion is running, and therefore the version that all engines, modules etc is running as well.
In your example it says python3, but all the code running through the salt-minion process, must be python2.7.

I hope this helps you improve the development workflow of your project.

Remember that we also offer additional more in-depth paid support to get you started with the project if you are interested.
https://shop.autopi.io/products/development-support-voucher-16/

I hope I have covered everything - please let me know if you have any additional questions.

Best regards
/Malte

This is great Malte, thank you!
If we would like to get paid support from you.
Can we use the hours which we have bought in different days.
For example we use 1 hour of our support for one day and use the left over a week later ?
Also, could we have this support through a video chat app ?

Waiting for your quick response, because we might book it tomorrow.
All the best and thanks for your comprehensive response.

Hi

Yes it’s fine to use it over a period of time as you need it.
It can be done however you like, but keep in mind that i think it depends on the kind of support you’d need, some questions which require some preperation or research might be better suited for a longer technical response in writing.

Best regards
/Malte