Extracting images/metadata from binary files exported in recorder_v2 app

earlranario · September 22, 2023, 5:11pm

What package and version are you using ?
Amiga OS 2.0
Please, describe the issues you are seeing ?
Are there any existing scripts that can extract images/metadata from the .bin files saved using the Recorder_v2 application?

** Any possible information will make the live of the community members easy to reply and provide a solution.

Thanks in advance!

kylecoble · September 25, 2023, 4:00pm

The use of our scripts and examples for interacting with the pre-release Amiga OS 2.0 is not yet well documented (but is soon to come as we roll out the release!).

We have a good example for what you want to do. It’s not yet merged into GitHub - farm-ng/farm-ng-amiga at main-v2 (the farm-ng-amiga branch that corresponds to AmigaOS v2.x), so it would’ve been hard for you to find this.

To get started on the extraction of images, you can try the file_reader example that will playback your log and show the stream for a given camera stream in a cv2 window:

github.com

farm-ng/farm-ng-amiga/blob/a6c7f5051df4fda6bf62402c61fb519ee15ffccc/py/examples/file_reader/main.py

"""Example of reading events from an events file."""
# Copyright (c) farm-ng, inc.
#
# Licensed under the Amiga Development Kit License (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://github.com/farm-ng/amiga-dev-kit/blob/main/LICENSE
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations

import argparse
from pathlib import Path

import cv2

This file has been truncated. show original

Then look at the file_converter example that can convert your log either to an mp4 file or a series of jpg images for a given camera stream.

github.com

farm-ng/farm-ng-amiga/blob/a6c7f5051df4fda6bf62402c61fb519ee15ffccc/py/examples/file_converter/main.py

"""Example of how to read an events file and convert it to a video."""
# Copyright (c) farm-ng, inc.
#
# Licensed under the Amiga Development Kit License (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://github.com/farm-ng/amiga-dev-kit/blob/main/LICENSE
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations

import argparse
from collections import defaultdict
from pathlib import Path

This file has been truncated. show original

Both files have accompanying README files.

For the metadata, this depends on what you are looking for. Assuming you mean camera calibration parameters, then you can filter the events_dict in the examples for the f"/{camera_name}/calibration" topic name.

Please let me know if you have any additional questions.
–Kyle

earlranario · September 25, 2023, 5:47pm

Thank you Kyle for your quick response and useful information.

Will these scripts work for my current package versions?

farm-ng-amiga 0.0.9
farm-ng-core 0.1.4
farm-ng-package 0.1.3

I tried running the file reader script:

python file_reader.py --file-name 2023_09_14_19_06_20_494194_moats-unproved.0000.bin

And got this error:

Traceback (most recent call last):
File “/Users/earlranario/Documents/GEMINI/scripts/bin-to-images/test/convert_binary.py”, line 161, in
main(args.file_name, args.output_path, args.camera_name, args.view_name, args.disparity_scale, args.video_to_jpg)
File “/Users/earlranario/Documents/GEMINI/scripts/bin-to-images/test/convert_binary.py”, line 97, in main
sample: oak_pb2.OakFrame = event_log.read_message()
File “/Users/earlranario/miniconda3/envs/amiga/lib/python3.9/site-packages/farm_ng/core/events_file_reader.py”, line 48, in read_message
return self.reader.read_message(self)
File “/Users/earlranario/miniconda3/envs/amiga/lib/python3.9/site-packages/farm_ng/core/events_file_reader.py”, line 163, in read_message
name, package = parse_protobuf_descriptor(event_log.event.uri)
File “/Users/earlranario/miniconda3/envs/amiga/lib/python3.9/site-packages/farm_ng/core/events_file_reader.py”, line 27, in parse_protobuf_descriptor
type_split, pb_split = uri.query.split(“&”)
ValueError: too many values to unpack (expected 2)

Debugging into this line, I found 3 values rather than 2:

‘type=farm_ng.oak.proto.OakFrame&pb=farm_ng/oak/oak.proto&service_name=oak0’

Changing this:

type_split, pb_split = uri.query.split(“&”)

to this seemed to work:

type_split, pb_split, _ = uri.query.split(“&”)

kylecoble · September 26, 2023, 3:00pm

Hello @earlranario ,

These scripts will not work for your current versions of farm-ng-amiga or farm-ng-core.

Until we move the new OS from pre-release, you will need to install the packages locally on these branches:

GitHub - farm-ng/farm-ng-core: C++ and Python library for farm-ng logging format (main)
GitHub - farm-ng/farm-ng-amiga at main-v2 (main-v2)

– Kyle

earlranario · September 26, 2023, 11:12pm

Hi @kylecoble

I’m assuming I would need to install from source to install these packages on these branches?
It seems to fail when I do (using setup.py). I cloned the repository and then changed branches.

For farm-ng-amiga:

Preparing metadata (pyproject.toml) did not run successfully.

For farm-ng-core:

Building wheel for farm-ng-core (pyproject.toml) did not run successfully.

Do you recommend any other way to install them?

Side note: I was able to successfully extract the images but I also am interested in timestamps and gps information from the exported binary file.

Thank you for your continual responses.

kylecoble · September 27, 2023, 2:30pm

Hi @earlranario ,

You should be able to install farm-ng-core & farm-ng-amiga in editable mode in the following way:

# Clone the repos
git clone https://github.com/farm-ng/farm-ng-core.git
git clone https://github.com/farm-ng/farm-ng-amiga.git

# Checkout correct branches and update
cd farm-ng-core/
git checkout main
git pull
git submodule update --init --recursive
cd ../

cd farm-ng-amiga/
git checkout main-v2
git pull
cd ../

# Create and source a virtual environment
python3 -m venv venv
source venv/bin/activate

# Upgrade some deps
pip install --upgrade pip
pip install --upgrade setuptools wheel

# Install the packages
pip install -e farm-ng-core
pip install --no-build-isolation -e farm-ng-amiga

Now you should see both installed if you check with pip list. For any of the examples, there should be some basic requirements (e.g. numpy & opencv-python) in a requirements.txt file nested in the corresponding example folder. Install these in your virtual environment with e.g.,

cd farm-ng-amiga/py/examples/camera_client/
pip install -r requirements.txt

Please let me know how this goes.
– Kyle

kylecoble · September 27, 2023, 2:32pm

If you wouldn’t mind spinning each of these off into separate discourse posts (one per topic), it will help future users find the answers more easily. I’ll be happy to respond there.

– Kyle

earlranario · September 27, 2023, 5:50pm

@kylecoble

Hi Kyle, I followed your steps but could not build successfully for both farm-ng-core and farm-ng-amiga.
Here are the outputs:

farm-ng-core:

Building editable for farm-ng-core (pyproject.toml) did not run successfully.

farm-ng-amiga:

Preparing editable metadata (pyproject.toml) did not run successfully.

kylecoble · September 27, 2023, 6:09pm

@earlranario my apologies, I missed a line!

I’ve updated the instructions above. The missing magic line in farm-ngcore/ was:

git submodule update --init --recursive

Please give it a try again.

– Kyle

earlranario · September 27, 2023, 6:32pm

@kylecoble thanks Kyle, git submodule update --init --recursive ran with no error but a new error occurred when running pip install -e farm-ng-core

Building editable for farm-ng-core (pyproject.toml) did not run successfully.
…
error: command ‘/usr/local/opt/llvm/bin/clang’ failed with exit code 1
clang-15: error: unknown argument: ‘-fconcepts’
[end of output]

kylecoble · September 27, 2023, 8:00pm

Hi @earlranario ,

That is surprising, but thank you for working through this with us. Can you give me some more information on your system, including your OS, python3 version, & clang version?

You can check clang with clang --version.

If you do not have clang installed, you can run:

sudo apt update
sudo apt install clang

From your venv, you can check with python --version.

Thanks,
– Kyle

earlranario · September 27, 2023, 8:24pm

Sure!

OS: macOS Ventura 13.4.1
python: Python 3.9.18
clang:

Homebrew clang version 15.0.7
Target: x86_64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

kylecoble · September 27, 2023, 9:33pm

Thanks for the info @earlranario.

Unfortunately the python build for our new APIs has been broken for MacOS. Let me work on fixing this and I will update you very soon with new instructions or the green light to pull the latest and try the previous commands.

Thank you again for your patience,
– Kyle

earlranario · September 28, 2023, 3:47am

Thanks @kylecoble

I followed the same steps above on a different workstation and it worked without errors!

kylecoble · September 28, 2023, 6:23pm

Glad to hear it @earlranario ,

Since you are unblocked, for now we will maintain Ubuntu 20.04 + python 3.8 as the only supported version combination for running the examples. Also FYI - we are actively working on expanding the set of examples, so be sure to check back regularly.

Best,
– Kyle

kylecoble · November 17, 2023, 5:13pm

FYI @earlranario ,

We have updated the build instructions, including for MacOS at:

Python 3.8 is formally supported, but it may work on your system with Python 3.9 as well.

– Kyle

Topic		Replies	Views
File Converter Example general tutorials , announcement	0	249	April 20, 2023
Unable to use File Reader development amiga	1	268	March 9, 2023
Image Quality Problem development	4	76	July 25, 2024
Decoding camera logs *.bin has missing frames support	5	279	April 27, 2023
Exportin RGB to a MAC with USB FlashDrive general amiga-brain	5	205	May 23, 2024

Extracting images/metadata from binary files exported in recorder_v2 app

Related topics