Yocto-based OS Services

If there is anything wrong with Vitro Crystal, it is strongly recommended that you start by checking for its services status and logs to identify the problem.

Yocto-based OS uses systemd to manage services. Thus, we can use systemctl to examine and manage them. For example:

systemctl list-unit-files

will list all the available unit files, even the ones not loaded.

systemctl status <service>

will provide the status of said service.

In case you want to see log files, you can use journalctl:

journalctl -fu <service>

👍

The flags -f and -u are not required. -f just keep printing the logs as they are generated, while -u lets you select the logs of a specific service.

Services

There are 220 services on Yocto-based OS. Here, we will cover some of them.

Vitro

🚧

vitroio-upgrade-nodes should always be disabled. This service only starts during an OTA update and ends as soon as it finishes.

vitroio-node-gateway

It is responsible for general gateway operations. Any other task-specified services can only run if it is enabled. It can be treated as the main service in the entire system.

As an example, the following log shows that when measurements are received from Vitro Shard, it is buffered and sent to AWS:

journalctl -fu vitroio-node-gateway

(...)
Dec 17 13:21:57 vitroTV node_gateway[223]: NODE_GATEWAY_SERVICE: Buffering measurments data: jc3a4JfMVDizLeK9AMMVMm2cb+wxvluC+GaDoPTwkkQyAQEj2mp0OAB67nVb21/aHhj+ZpT04G/wJVMey0EL0/Cz6LL/ygNkOAfZA+vHwYogsijcsIDZD4jYmoSF1fFt6saWvbOWhz889NTQz2rsl5183RLHdch3W0yM+DWxAH4uarE30sUgFQEVeCN07QM= 
Dec 17 13:21:57 vitroTV node_gateway[223]: NODE_GATEWAY_SERVICE: Number of buffered samples for solution: 1 
Dec 17 13:21:58 vitroTV node_gateway[223]: NODE_GATEWAY_SERVICE: Sending buffered measurments for solutions: std::vector(mitre-poc)
Dec 17 13:21:58 vitroTV node_gateway[223]: NODE_GATEWAY_SERVICE: Publishing measurments data on topic: testCustomer/01239E11A1ACFF51EE/mitre-poc/nodeData ; msg: [ 
Dec 17 13:21:58 vitroTV node_gateway[223]: { 
Dec 17 13:21:58 vitroTV node_gateway[223]: "nodeDataValue": "jc3a4JfMVDizLeK9AMMVMm2cb+wxvluC+GaDoPTwkkQyAQEj2mp0OAB67nVb21/aHhj+ZpT04G/wJVMey0EL0/Cz6LL/yg 
(...)

vitroio-canbus-node-adaptor

It is responsible for exchanging data with Vitro Shard via CAN bus. In case there is any issue with received IoT blocks, Vitro Shard's faulty operation, among others, please check if this service is running correctly.

The following log shows CAN data exchanged between Vitro Crystal and Vitro Shard. There you can verify the specific frame ID, CAN segmentation protocol, and general CAN communication.

journalctl -fu vitroio-canbus-node-adaptor

Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Timestamp Request Handler : CAN frame with timestamp has been sent to Node
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 7ffh"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 7ffh] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  0
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  1
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  2
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  3
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  4
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  5
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  6
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  7
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  8
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  9
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  10
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  11
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  12
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  13
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  14
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  15
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  16
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  17
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  18
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  19
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  20
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS: "Received frame: node 3, parameter 134h"
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Started
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Received data segment  21
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h] : Successfuly received CAN data from Node.
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h]  COMPLETED:
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: CANBUS_NODE_ADAPTOR_SERVICE: Node data reported successfully
Dec 17 13:18:38 vitroTV canbus_node_adaptor[224]: --> TASK  Node Data Handler [Param 134h]  DESTRUCTOR

vitrio-mqtt

It is responsible for exchanging data with AWS via the MQTT protocol. It publishes IoT Blocks to a given topic. It also downloads updates when OTA is triggered.

The following log shows the MQTT message with the topic, payload, and timestamp. This same message is then sent from Vitro Crystal to AWS.

journalctl -fu vitrio-mqtt

(...)
Dec 17 13:19:08 vitroTV mqtt_proxy[216]: MQTT_PROXY: calling publish() topic:  "testCustomer/01239E11A1ACFF51EE/mitre-poc/nodeData"  message:  "[\n    {\n        \"nodeDataValue\": \"dnPQkIEY1gvD9CtwCkM5w+oI1bD5jQ6rZwUaBqwrXkc0AQEj2mp0OAB67sxa219P1Q05/OgLLTZzthctwunzBuXcsNsSg/rbHHu73eMGnzjGguRv3RBST9nOwydxToWX2nXyuzrdrvgxvE1bDWEP6YASHWyMwFWCvQa/w8faGYNaSFuxaBjAMjeR7+FPf4Y=\",\n        \"nodeId\": \"0123DA6A7438007AEE\",\n        \"timestamp\": \"1608211148\"\n    }\n]\n"
Dec 17 13:19:08 vitroTV mqtt_proxy[216]: MQTT_MOSQ_CLIENT: on_log [level= 16 ]  "Client 01239E11A1ACFF51EE sending PUBLISH (d0, q0, r0, m192, 'testCustomer/01239E11A1ACFF51EE/mitre-poc/nodeData', ... (311 bytes))"
Dec 17 13:19:08 vitroTV mqtt_proxy[216]: QMQTT_CLIENT: log [ 0x76f86410 ] : "Client 01239E11A1ACFF51EE sending PUBLISH (d0, q0, r0, m192, 'testCustomer/01239E11A1ACFF51EE/mitre-poc/nodeData', ... (311 bytes))"
Dec 17 13:19:08 vitroTV mqtt_proxy[216]: MQTT_MOSQ_CLIENT: on_publish  msgId:  192
Dec 17 13:19:08 vitroTV mqtt_proxy[216]: QMQTT_CLIENT: message published [ 0x76f86410 ] : 192
Dec 17 13:19:08 vitroTV mqtt_proxy[216]: MQTT_PROXY: published  192
(...)

Interface

Swupdate

👍

swupdate-progress tries to connect to a running instance of SWUpdate to get the status of a running update.

Other