BoSLcam

Setup

For information on how to setup the BoSLcam to work with international carriers, check out the instructions at BoSL_Board_nano#LTE_Connection. Note that The APN and MCC MNC codes can be changed on the SD card.

Programming

To flash the modem firmware and application you will need a Segger j-link debug probe and a computer with the Segger j-link software tools and Python installed.

1. Power up the BoSLcam using a battery.
2. Plug the Segger J-link into the computer.
3. For the modem firmare, follow the instructions at: https://www.bosl.com.au/wiki/NRF9160_modem_firmware to flash modem firmware version 1.3.2 available at: https://www.nordicsemi.com/Products/nRF9160/Download#infotabs
4. For the application firmware, open the J-Flash or J-Flash Lite tool if programming for development purposes ([1]). We will tailor instructions for the J-flash lite tool.
5. Select the options as per the diagram in the gallery. Click Ok.
6. Under the Data File open the firmware "BoSLcam_firmware_revx.x.x.hex" file.
7. Connect the pins on the "SWD" header to the associated pins on the J-link probe. A pinout for the J-link for SWD can be found in its instruction manual [2]. The depicted programmer probe may also be found useful for breaking out the small header pins.
8. Click the program device button in the J-flash window. If the tool indicates that an error has occurred. Check your wiring and then check it again.

• 

  step 3

• 

  helpful programmer probe

Status File Format

As of firmware revision v1.5.0 the status file format is given as:

DATE-TIME, TIME_SOURCE, CAPTURE_NO, BATTERY_mV, MCCMNC, RSRQ, RSRP

So the log:

       2024/03/22-01:44:27 UTC,NETWORK_TIME,11,4175,310260,13,45

means:

 DATE-TIME:     2024/03/22-01:44:27 UTC
 TIME_SOURCE:   NETWORK_TIME           (time obtained from the network)
 CAPTURE_NO:    11                     (number of images since last reset)
 BATTERY_mV:    4175                   (battery voltage (mV))
 MCCMNC:        310260                 (the current network operator MCCMNC code)
 RSRQ:          13                     (reference signal recieved power)
 RSRP:          45                     (reference signal recieved quality)

27th April 2022

In addition to choosing the correct camera, a lighting system will also be needed for when there is no light in the water system. Using a camera flash is one option, however this will likely need to be a pre-built flash module due to the high voltages involved. An alternate option is using LEDs with a longer exposure time. To see if this is viable we note that the sensitivity of the camera is 1.8 V/(Lux s) and the dark current is 10 mV / s.

Assuming that the led light is co-located with the the camera, then once can calculate that the needed exposure time to get a 50% grey is:

Where Texp is the needed exposure time, Vexp is the 50% grey voltage (~1 volt for our use cases), η is the reflectivity of the water surface, σ is the sensitivity of the camera (~1.8 V/(lux s) for the types of cameras we are using), Icd is the luminous intensity the leds used, θv the half the field of view of the camera, and H the height of the camera and led above the water. For a higher power white led with luminous intensity of about 33 candela and a distance to the water surface of 3 m this gives, a 20% reflection of the water surface and the OV7675 camera module this gives an exposure time of about 7 seconds. This is probably just on the edge of a feasible exposure time, so with higher power or multiple LEDs, using LEDs to illuminate the water subject should be feasible.

Ideally we would want the colour temperature of the LEDs to match that of the sun so that the difference in the illumination conditions can be minimised. Additionally a narrow angle LED would be preferred so the light isn't wasted outside the field of view of the camera something like (https://cree-led.com/media/documents/C503C-WAS-WAN-1098.pdf) would be ideal. Such LEDs tend to find application in road signs or torches when illumination at a long distance is needed.

3rd October 2022

A bit of work on the prototype hardware was completed today. A guide was found on how to connect the OV7675 directly to an Arduino Uno (https://circuitdigest.com/microcontroller-projects/how-to-use-ov7670-camera-module-with-arduino) this was followed and I was able to stream the picture data over serial to the computer. Others have had success with connecting these embedded cameras directly to an RP2040 MCU (https://www.uctronics.com/arducam-pico4ml-tinyml-dev-kit-rp2040-board-w-qvga-camera-bluetooth-module-lcd-screen-onboard-audio-more.html). The RP2040 is low cost but capable MCU (and in stock), the one disadvantage which it has is that is that its lowest power state consumes current in the milliamperes. Its SRAM is large enough to store a single complete QVGA frame from the OV7675. To trial this MCU system, the reference design the the RP2040 was extended to include a connector for the OV7675 breakout board and the camera module itself. Various jumpers have been included to aid in debugging to the board.

10th October 2022

An idea was also had to use an nRF9160 as the MCU for the camera board. This would have the following consequences for the design:

• Higher cost ($30 vs $3)
• Faster uploading
• Lower power usage
• No need to external bosl logger
• familiarity with programming the nRF9160
• no examples for camera modules with the nRF9160

For certain applications given that no BoSL would be needed, the total application cost would come out to be less. I would also suspect the uploading would be more reliable with the integrated nRF9160, as the data from camera would not need to be relayed through the MCU, BoSL MCU, and then finally to the SIM module.

22nd November 2022

The prototype development board which connects the nRF9160 to the OV7675 camera was soldered today. So far I have been unable to connect the onboard nRF9160 to the SEGGER J-link to program and work with the device. Perhaps this is a soldering issue. After a few trials this issue was determined to be a soldering issue which was solved. The solution involved using low temperature solder paste, elevating the board so that the heat was not drawn out by the desk surface, and removing the nozzle on the hot air gun so that the heat was distributed more evenly. Also using a high-ish temperature ~420 C and low airspeed seemed to work better. Once this was done the nRF9160 could be programmed by the J-link. Some improvements related to simplifying the PCB schematic were also noted, and these were implemented for the next revision of the PCB.

• 

  Hand soldered development pcb

24th November 2022

The first step to controlling the OV7675 is to get the I2C component of the SCCB bus working. It was found that in order to get responses from the OV7675 I2C a minimum of the power, ground, scl and sda (with pullups), XCKL (with clock signal), PND to gnd, and PEN to VCC are needed.

29th November 2022

I2C responses were able to be achieved on the dev board. This was achieved using the pin activations described previously. The XCLK clock was made to be an 8MHz square wave. This is somewhat difficult to achieve on the nRF9160 but once set up works reliably. Specifically, you need to use the DPPI system, which is able to connect the timer peripheral to the gpio (gpiote) through a system which uses tasks and events. Events are published to the DPPI when something happens on a peripheral, this causes all subscribed tasks on the channel to be initiated. Here the event is the timer compare register matching and the event is a toggle of the GPIO pin. The trickyness in this was ensuring that the peripherals were connected to the DPPI bus correctly via the nrfx hal code. From here the ov7075 code could be ported from the ATmega to the nRF9160, the next steps are to read in the GPIO image data. The pixel clock can be chosen to be relatively slow ~ 70 kHz. As the nRF9160 has enough memory to store image data, we can read it directly into memory. Doing this an image could be read out from the camera. There is still some work to determine the exact pixel formats, and register settings for perfect image quality, but in the mean time here is a sample image:

• 

  image from the OV7675 captured directly from an nRF9160

2nd December 2022

The camera was attempted to be connected directly to the PCB using the flat flex cable connector. This did not work however as it was found that the pinout was wrong. It was also found that for the breakout board connector pins D0 and D1 were swapped around, this may be something to note when attempting to correct the image data issues.

5th December 2022

After some tweaking of registry values and the code used to read the data from the camera, a working qvga image was able to be extracted the OV7675 onto the nRF9160. An example is given here.

• 

The one issue with this current setup is that the image is stretched vertically in a 2:1 ratio.

6th December 2022

The SD card was able to be made to talk to the nRF9160. The configuration was quite picky with which options needed to be set in order for the communication to work. As such I am uploading a zip of the project which can be build with nRF SDK v1.9.1 here for reference and backup purposes. File:Camera native-sd.zip

One important note is that the nRF9160 supports only 4 serial devices which can be either configured to be SPI,I2C, or UART. Currently we have used SPI3 for the SD card and I2C2 for the camera. UART0 is used for logging/debugging. This leaves device 1 which we will either use for another SPI device like flash memory, or another UART so the board is able to talk to other boards.

The sd can be accessed via a unix like file system. The root of the SD card is at /SD:/. From here it was simple enough to write the image buffer data to file on the SD card and view this on a computer.

7th December 2022

After some more twiddling with the settings, configurations, and libraries the LTE modem was able to be activated and register to the network. Internally it seems that the modem talks to the application through the same AT commands that are used in the serial LTE modem. Most of the common AT+ commands are there however some of the more exotic AT# or AT% commands are not. It seems these ones are related to external libraries which have their own functions to operate. FTP is one of these libraries, however all the ftp functions can be accessed through the ftp_client.h library. From here I was able to connect to the FTP server and easily put a file. Now comes the integration of all three firmware functions.

8th December 2022

An issue was identified with the FTP uploading. Images uploaded via ftp from the nrf9160 would have all bytes of 0x0d (\r) removed from the file. This is because the default mode is to send the file in FTP text mode, where as for image data FTP binary mode is needed. I am looking into how to entre ftp binary mode now. A typical FTP image is shown here. The upload link speed is about 8 kB/s, hence uploading once QVGA RBG565 photo takes about 20 seconds. Now the task is to develop a program capable of reading a config file off the SD card and begin to log images according to this config.

• 

  FTP uploaded photo.

15th December 2022

A basic program for uploading and storing the arducam images has been completed. It is now time to focus on measuring and optimising power consumption to ensure that the device is able to have a strong battery life. The first step with this was to power the board with an inline shunt resistor. The voltage drop across this resistor can be measured with an oscilloscope to measure the current via Ohm's law. In the current code state, plugging the board in it draws about 40 mA of current, which is quite high. After the sd card has been mounted sdhc_mount(), the current drops by 15.6 mA. With no SD card inserted the idle current draw is 17 mA.

With the SD card, the sim, and the OV7675 module removed, the current draw falls to 1.36 mA under a k_msleep() loop. Now with the SRAM IC (23LCV1024) CS pin pulled high, the idle current is 800 µA. This is an improvement but is still in the 100s of µA range.

16th December 2022

With all peripherals disabled the idle current consumption of 37 µA could be achieved. It was also found that the pullup resistor on the nRESET pin was not needed as it was internally pulled up to 2V2. The target idle current is about 4 µA. A 100 kΩ pull up resistor on the SD CS line may be to blame, so it was removed. This however did not have a significant impact on the sleep current. After a fair bit of troubleshooting no further improvements to the current consumption could be found. Now we need to work on how to fall and wake from sleep. In particular how to configure all the peripherals such that they are able to effectively resume from sleep. For the SD card, and SPI the build flag CONFIG_PM=y does a good job of this. However it should be noted that some SD cards have a much higher idle sleep current the san disk which I have been testing with goes up to 400 µA!. For this reason we should probably add a MOSFET to control the SD card power. For the SRAM, only the CS pin needs to be pulled high. For the LTE modem, it seems that enabling PSM will reduce the power of the modem to a few µA. The advantage of PSM is that it seems like it goes not need to reregister to the network when it wakes up from sleep.

19th December 2022

After some further investigation is is determined that the power switch on the OV7675 is not necessary as pulling the PDN pin high cuts the current to single µA. This concludes the power testing of the out of water turbidity sensor. It is estimated that the final idle sleep current will at worst 70 µA. It is now time to design the final compact PCB.

21st December 2022

The initial release of the BoSLcam/out of water turbidity probe was made. This is a miniaturised version of the dev board, in the same form factor as the BoSLnano. As always, it can be found in the design library Design_Library#BoSLcam 1.0.0. Some pictures of the front and back of the PCB ware provided.

• 
• 

11th February 2023

To use the BoSLcam with the supplied firmware the following config file needs to be placed in the root directory of the SD card. It contains all the configurable options for how the camera will take photos and upload them to the FTP server. File:Bosl cam configuration .txt A handy note is that to upload a file via FTP takes on the order of 1 minute

14th February 2023

The BoSLcam have arrived for tested. After powering the device and connecting the debugger probe the SWD debugger could connect to the nRF9160 MCU. The next step was to close the SD card power jumper to 3V3. It was found that the modem firmware needed to be flashed. This could be done by following the instructions at https://devzone.nordicsemi.com/f/nordic-q-a/52018/programming-nrf9160-modem-firmware-with-a-stand-alone-segger-j-link and flashing modem firmware version 1.3.2 available here https://www.nordicsemi.com/Products/nRF9160/Download#infotabs. This needs to be done with the segger probe.

It also appears that the test firmware supplied to the assembly house does does not allow for normal operation such as taking pictures. To do this a new firmware need to be uploaded. Specifically this one File:BoSLcam firmware.zip. Which can be uploaded with Segger J-flash or J-flash light

Here is an example image which comes out of the camera

• 

  first image from BoSLcam

20th February 2023

The aspect ratio issue of previously seen in the camera has now been corrected. This was done by switching to the https://github.com/arduino-libraries/Arduino_OV767X library to set the OV7675 registers. We also switched to the CIF image size (352 * 240). The last issue now seen is that the edges of the image data seem to be gibberish though this may be fixed by further adjusting the frame size registers. Proof of the correct aspect ratio is now seen in the below image which shows the back of a circular pen as a circle rather than an oval. From this experimenting it seems that adding a break point between initialising the camera and setting the hstart and hstop values will cause the stretching issue. odd.

• 

  CIF image from the bosl cam displaying correct aspect ratio

23rd February 2023

An interesting note is that the length of the pixel clock changes with the resolution mode chosen with QVGA and CIF having half the pixel clock compared with VGA. Horizontal line scan time does not change, nor does the time between VSYNC pulse. This implies that the number of lines are the same between all the video formats. In part, this explains why we were having issues with the stretching on the QVGA video settings, we were reading every line out of the camera assuming that there were only 240 of them, whereas there were actually 480 of them so we were only getting the first half of the frame. In short, while changing the resolution changes the horizontal resolution, it does not change the vertical resolution or the proper time taken for a frame to be transmitted. So I think that the reason CIF was working where QVGA was not was because CIF horizontal resolution of 352 does not divide the VGA horizontal resolution of 640, this likely caused something like that the MCU wasn't ready to read in the next line and so each alternate line was fixed. This is just a theory however.

6th April 2023

The BoSLcam firmware has been updated to add a secret suffix to the ftp password. This way a hacker with access to the SD card will not be able to view the password in plain text. This change required a modification to the config.txt file to change the password and a line which tells the program to add the suffix to the password. I have added to the option to turn this suffix on and off so as to maintain compatibility with ftp sites which to not take this suffix. The code has also been updated to upload the current battery voltage (in mV).

12th April 2023

rev 1.1.0 of the BoSLcam has been designed. The changes include:

• Added capacity for an LED light. A power switch and pins have been added to turn on and off an LED photo light.
• Set jumper position for SD power to be from the power management IC. This means that the jumper will no longer have to be bridged out of the gate.
• Fixed BOM such that components match the footprints used.
• Deleted battery voltage sense resistors (functionality already provided internally in the nrf9160)

Some photos of the revision 1.1.0 are provided:

• 
• 

I have also released BoSLcam firmware revision 1.0.1 to align with BoSLcam hardware rev1.0.1. This update includes battery voltage and password encryption options. One issue remains that one in every 6 or times the board is turned on, the image byte order will be reversed as displayed in the gallery. Given that the issue only occurs when the board is turned on, it suggests that it has to do with the camera initialisation. The byte order reversal is not the only difference between the two images. One can also see that the reversed image extends all the way to the edge of the frame whereas for the non-inverted image the left most 10 or so pixels appear to be bad data. Which byte order we get shouldn't be an issue so long as I can figure out a way to make the order 100% consistent.

By slowing the rate at which commands were sent to the OV7675 camera, it was found that the byte order could me made consistent and the garbage pixel data on the LHS disappeared. The goal now is to find which command needs some wait time in-between. This is implemented in firmware patch File:BoSLcam firmware rev1.0.2.hex.zip

• 
• 

21 September 2023

I have tried out adding JPEG compression to the images. The results are quite promising. It takes ~6 s for a QVGA image and results in a file and file sizes between 1/2 (quality 3) to 1/5 (quality 2) to 1/30 (quality 1). Overall I'd say that quality 2 offers the best compromise here.

9 November 2023

I have implemented the JPEG compression into a new firmware: File:BoSLcam firmware rev1.1.0.hex.zip This revision adds the new config for selecting JPEG compression. Here is a snippet of the change needed

...
//image_config_t
uint32_t auto_range_time = 1000
enum format = 1 //0 BMP, 1 JPG
//ftp_config_t
char* apn = "hologram"
...

16 November 2023

The SPI SRAM should be upgraded to have larger capacity. This would mean replacing the 1 Mbit 23LCV1024 with something closer to 8 Mbit, yet still low power.

13 February 2024

Two BoSLcam with firmware revision v1.2.1 File:BoSLcam firmware rev1.2.1.hex.zip have been placed in the lab for a long term test. One is set to upload VGA-JPG and the other QVGA-BMP. They have been logging for approximately a week at a 6 minute logging interval without major issue. Today I disconnected the antenna on wdt_qvga in a hope to observe the watchdog timer kicking in and recovery of the BoSLcam, the results of this will be reported in the coming days. For some preliminary results, below are the graphs of the logging gaps and battery voltages against number of logs for both cameras.

• 

  WDT QVGA-BMP

• 

  VGA-JPG

Of note is that over the ~1.5k logs only 10 or so were missed and the BoSLcam recovered from each of these. From the gradient of the battery voltage we can find that VGA-JPG image take ~ 0.12 mV/log and QVGA-BMP take ~ 0.07 mV/log (11200 mAh battery). A known issue with this software version is that some lines in the image miss a byte which causes inverted colours. Resolving this issue will need a close look at tightening up the timing of the image readout loop. In total, WDT QVGA-BMP recorded for a total of 7174 images and down to a battery votlage of 3023 mV.

16 February 2024

Three days later plugging in the antenna on QVGA-BMP, the BoSLcam immediately started logging to FTP again. Based on the uploaded status files (I don't want to remove the SD card yet) the BoSLcam kept taking taking and storing photos to the SD card during the period it was offline, hence the WDT didn't need to kick in. Overall, while this test did not test the WDT, it was still a good demonstration of the reliability of the BoSLcam firmware.

19 February 2024

Unlike when using QVGA when using VGA, the image quality can sometimes be affected by the focus of the lens. Thankfully the lens focus can be adjusted by rotating the lens. Clockwise screws it in which makes the focus closer and anti-clockwise moves it out which makes the focus further away. Current the best way to check the focus is via trial and error. I have found that about 1/2 to a full turn of the lens is a good initial step to adjust the focus in. Below are two images taken of the same wall where I have adjusted the focus by a about a 1/2 turn.

• 

  far focused BoSLcam

• 

  near focused BoSLcam

20 February 2024

Today I will be working on the LED function for the BoSLcam. Some very rough initial visual tests indicate that to produce a well-illuminated spot at 2 m subtending and angle of 20 deg, ~ 100 mA to 200 mA of current need to be passed through the LED.

With regard to driving the circuit, I recommend the below as an effective low-complexity option. One drawback of this design is that the LED current will vary by a factor of ~ 2 between a battery voltage of 4.2 V and 3.5 V.

• 

  LED driving circuit.

Below are images taken with the BoSLcam where the scene has been illuminated with an LED. I have compared three different LEDs: vishay VLHW5100, cree XQAAWT-02-0000-00000B4E3, osram GW CS8PM1.PM-LRLT-XX52-1. I tested the LEDs at two different currents, 30 mA and 90 mA, both of which are above the maximum forward current however, in a finalised design this current can be divided among multiple LEDs.

• 

  Cree 30mA

• 

  Cree 90mA

• 

  Osram 30mA

• 

  Osram 90mA

• 

  Vishay 30mA

• 

  Vishay 90mA

Any of the LEDs produce an acceptable image however, I prefer the cree or osram devices as they are SMD and they have a much wider illumination beam which gives them a more uniform image illumination. I think the 30 mA is just a bit too dim and so it will be good to design around a 90 mA current target.

Now it is time to design a PCB for this.

18 March 2024

A PCB was designed, with three CREE XQAAWT-02-000-00000B4E3 LEDs each with their own 49 Ω ballast resistor. With this design, the LEDs draw between 15 to 30 mA when the battery voltage is between 3.7 and 4.2 V. A DMN2310U N-channel mosfet allows the LEDs to be switched on and off from any of the GPIO on the BoSLcam.

I have also incorporated these components directly onto rev1.2.0 of the BoSLcam. Some care was taken to place the LEDs a few mm away and behind the camera so as to minimise the lens flare.

22 April 2024

As of firmware revision v1.4.0 the status file format is given as:

DATE-TIME, TIME_SOURCE, CAPTURE_NO, BATTERY_mV, MCCMNC, RSRQ, RSRP

So the log:

      2024/03/22-01:44:27 UTC,NETWORK_TIME,11,4175,310260,13,45

means:

DATE-TIME:     2024/03/22-01:44:27 UTC
TIME_SOURCE:   NETWORK_TIME           (time obtained from the network)
CAPTURE_NO:    11                     (number of images since last reset)
BATTERY_mV:    4175                   (battery voltage (mV))
MCCMNC:        310260                 (the current network operator MCCMNC code)
RSRQ:          13                     (reference signal recieved power)
RSRP:          45                     (reference signal recieved quality)

2 May 2024

The BoSLcam rev1.2.0 have arrived. They have been preprogramed with modem firmware v1.3.2 and application firmware v1.4.0, as such they just need a SD card, sim card, and ov7675. No soldering is required anymore!

15 May 2024

As a further test of the inbuilt LED flash function, a BoSLcam was set up in the den. The picture in the gallery below was taken with the full VGA image resolution and was solely illuminated by the builtin LED flash. The wall was 2 m at its closest point to the camera and 3 m at its furthest. A comparison image is also provided with the LEDs turned off. It is fairly clear that the LEDs improve the ability of the BoSLcam to take images without an ambient light source. It seems that during this test the BoSLcam died of low-signal strength, however it was able to recover a day later via the WDT. A good test of the feature!

• 

  Image taken with the BoSLcam rev1.2.0 using only the builtin led flash for illumination.

• 

  Comparison image taken with the LEDs turned off.

3 June 2024

Here is my ICUD2024 presentation on the BoSLcam!