Difference between revisions of "In Water Vel Sensor"

From BoSL Wiki
Jump to navigation Jump to search
 
(85 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
<b> This section documented our journey to develop a low-cost water velocity sensor based on doppler effect!</b>
 
<b> This section documented our journey to develop a low-cost water velocity sensor based on doppler effect!</b>
 +
 +
Check out the full paper!: [https://doi.org/10.3390/s22197451 https://doi.org/10.3390/s22197451]
  
 
For information on specific velocity sensors see: [[BoSL Velocity Database]]
 
For information on specific velocity sensors see: [[BoSL Velocity Database]]
Line 1,144: Line 1,146:
 
File:Velocity 8 plot.png|Week 4 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)
 
File:Velocity 8 plot.png|Week 4 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)
 
</gallery>
 
</gallery>
 +
 +
The plots show that this algorithm can be used quite effectively to measure the flow velocity. Particularity also useful is that the sporadically high measurements appear as having a much lower signal strength, meaning that they can be effectively filtered out.
 +
 +
== 27th August 2020 ==
 +
 +
The refinement of the 3d printed case from 8th August was tested. The initial indications were positive. Much less leakage of the Gell-Gum occurred as had with previous designs. The design looks water tight, however particularly with the lack of cable grommet, this needs to be tested further, and for a longer duration.
 +
 +
== 8th September 2020 ==
 +
 +
New data was received from the velocity sensor. This was compared against the hach probe. The data was less robust than usual, as the velocities which are greater than 200 mm/s generally had given stronger readings, however this was not the case for this data set for some of the rainfall events.
 +
 +
The below plots are the week 4 and 5 data with the algorithm tweaked slightly to increase threshold and introduced a filtering on low amplitude data.
 +
<gallery>
 +
File:Vel filt algo 1.png | Week 4 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)
 +
File:Vel filt algo 2.png | Week 5 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)
 +
File:Vel filt algo corr.png|Week 5 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)
 +
</gallery>
 +
The changes can be seen to minimally impact the data from week 4 while addressing some issues with the week 5 data. The main issue seen to be caused by the algorithm changes are the of the minimum measurement velocity of about 150 mm/s. This change was not unintentional as it was found by looking at the FFT of the data that below this noise was seen in very great amounts and made it so that no real velocity data was present here.
 +
 +
This is shown below, with the old algorithm the peak at the low velocities would have been taken as the predominant peak for the velocity. This low velocity peak however is commonly seen in some FFT's regardless of the water velocity, so it was assumed that this should be not taken as part of the water velocity calculation.
 +
This was especially problematic as if the low bin peak dropped below the threshold before the the true peak then the true peak would have been excluded from the velocity calulation.
 +
 +
To combat this three changes were made, the first was to check that if the peak was detected in the low velocity bins if there was another and perhaps slightly smaller peak at a higher velocity. If so this high velocity peak was chosen instead. This worked to alleviate most of the problem. However the minimum bin to check was also changed from bin 2 to bin 4 (causing the minimum measurable velocity to jump to 150 mm/s). Finally, the threshold was adjusted and optimized in light of these changes.
 +
 +
<gallery>
 +
File:3384.0.png|FFT with noise in low velocity bins (black: hach) (red: new algorithm)
 +
</gallery>
 +
 +
 +
== 9th September 2020 ==
 +
 +
7 velocity sensors were build up. 6 of them had no issues however one of them presented a constant spike at about 3.5 m/s. This is similar to the issue seen in the field, however it is present without any transducers connecting. The issue look like a hardware/construction issue, as the PCB looks identical to all the others which did not have this issue. I think it would be good to write some testing code to screen for this at the manufacturing.
 +
 +
The velocity database has been updated [[BoSL_Velocity_Database]]. 2 PCB remain however I have run out of cable to make these into sensors.
 +
 +
== 10th September 2020 ==
 +
 +
Week 6 data was analysed today, the usual plots are shown below. The sensor is showing worse performance with the than in previous weeks. Notably many more points where spontaneous high readings are measured. This can be seen from about after August 31. It could be possible that something happened to the sensor at this time to change its characteristics or it could be that after this date new flow regimes were encountered that were not before. 
 +
 +
<gallery>
 +
File:Wk6 corr.png|Week 6 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)
 +
File:Wk6 plt.png|Week 6 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)
 +
</gallery>
 +
 +
== 11th September 2020 ==
 +
 +
A production versions of the velocity sensor code with the above algorithm for detecting velocity was made.
 +
 +
It is a available here: [[File:Velocity Firmware rev 0.1.3.zip]]
 +
 +
== 16th September 2020==
 +
 +
The code on the velocity sensor here [[In_Water_Vel_Sensor_FFT_SD_Logging_Test]] was updated to [[File:Velocity Firmware rev 0.1.3.zip]] so the velocity reading could be taken directly from the automatic logging.
 +
 +
The data from the past week of logging was processed and the standard graphs are shown below:
 +
 +
<gallery>
 +
File:Wk7 corr.png|Week 7 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)
 +
File:Wk7 plt.png|Week 7 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)
 +
File:Wk7 corr alt.png|Week 7 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak) however the threshold for blue saturation is higher
 +
</gallery>
 +
 +
As can be seen the plot shows a similar trend to the past weeks, the sensor performs well at high velocities, and less low at lower velocities generally tracking similarly to the hach probe. Notice that on the late end of the 2020-09-13 velocity event the BoSL probe declines in velocity measurement much faster than the hach. The hach always seems to have a similar smooth trailing off from rainfall events so perhaps this is an artifact of the some internal averaging method used.
 +
 +
The correlation plots show now much more data in the 500mm/s range. This data follows the linear trend-line indicating that the two sensors are measuring similar values. There is more data which occurs at low hach velocity but registers a high BoSL velocity. This appears to have a strong intensity (deep blue). The third plot plots the correlation again but with a higher threshold that is that points need a stronger intensity to be coloured deep blue. Here we see that these points at low hach velocity become much more lightly coloured that is that they can be distinguished from the points on the 1:1 line. This higher threshold does make the points at lower velocity very faint so this calls for a intensity threshold dependent measured velocity for effective filtering without throwing out good data unnecessarily.
 +
 +
== 17th September 2020 ==
 +
 +
A seperate PSD was measured for each velocity sensor. This was done to assess the variance in the PSD's between the sensors.
 +
 +
To evaluate the PSD the velocity sensor was placed in a bucket of still water. The depth of the water was about 20 cm the sensor was placed facing downwards at a 5 cm depth.
 +
The sensor was then set to print its RAW fft output to serial. This was recorded for about 250 samples and then the RMS of each bin was taken to find the PSD.
 +
 +
It was found that the positioning of the sensor in the bucket did not play a large role in determining the shape of the PSD for a particular sensor.
 +
The sensors did vary quite considerably in their PSD, so it seem that getting a PSD for each individual sensor is necessary. I am thinking of uploading the PSD into the velocity sensor database so that it can be easily retried whenever needed.
 +
 +
== 4th October 2020 ==
 +
 +
A new velocity FFT analysis algorithm was tested. The method used in https://ieeexplore.ieee.org/document/9013073 was used, both uses boxcar averaging on the initial velocity spectrum and estimates the spectrum by find the FWHM.
 +
A plot of the data under this algorithm is shown below.
 +
<gallery>
 +
File:Roman.png| Implementation of the algorithm as in described in the paper
 +
File:Current.png| Current algorithm
 +
File:Currentmaxamp.png|Current algorithm with new signal strength algorithm
 +
File:Currentboxcar.png|Current algorithm with new signal strength algorithm and boxcar averaging
 +
</gallery>
 +
It is seen that the algorithm described in the paper produces worse velocity readings with higher deviation from the 1:1 correlation line when compared to the current algorithm.
 +
3rd image displays the current algorithm where the signal strength is taken as the the highest single bin. It is seen that this performs better is distinguishing the velocity points close to the 1:1 correlation line with a higher signal strength then those far away when compared to the current algorithm. The 4th images takes the algorithm and adds boxcar averaging as described in the paper. This performs similarly to the 3rd image however does have more high intensity points at 0 velocity and fewer near the 1:1 correlation.
 +
It is hence chosen that the new signal strength algorithm will be incorporated into new version of the velocity sensor firmware.
 +
 +
 +
For the velocity paper some statistical analysis is needed to determine the accuracy of the sensor. I think it makes most sense to do this analysis not over the entire range of the sensor but calculate an error for the ranges of the sensors operation, perhaps 20 mm/s intervals.
 +
 +
The fractional error of the i-th velocity measurement is given by,<br>
 +
[[File:Math.png|frameless]]<br>
 +
Where v is the velocity as measured by the BoSL, and u the actual velocity.
 +
The actual velocity determined by measurement of the flow with a hach probe. This comes with its own uncertainty Δu.
 +
Thus, the uncertainty in η is,<br>
 +
[[File:Math1.png|frameless]]<br>
 +
We want to find the RMS fractional error over a number of points at similar velocities, hence taking the RMS of the fractional errors,<br>
 +
[[File:Math2.png|frameless]]<br>
 +
To find the uncertainty in the RMS fractional error we have that $R$ is a function of η<sub>i</sub>, so the uncertainty is,<br>
 +
[[File:Math3.png|frameless]]<br>
 +
Note, <br>
 +
[[File:Math4.png|frameless]]<br>
 +
Thus substituting, <br>
 +
[[File:Math8.jpg|frameless]]<br>
 +
An expression for the uncertainty in the error.
 +
 +
For the Δu, the uncertainty in the hach velocity I have had some difficulty coming up with a reliable figure. The only source so far is [https://static1.squarespace.com/static/56fb56208a65e2ecc1316d3b/t/570f0e7fb6aa60320a39d6d1/1460604554173/AV9000ds.pdf Hach datasheet] which quotes a 1% error for the spectrum analyser and 2% for the doppler sensor. This is almost certainly a underestimate of the actual uncertainty as the figure is quoted for a uniform flow field. In the mean time the figure of a 3% uncertainty will be used until a better one can be found.
 +
Running the above uncertainty calculation for the velocity data set of [[In_Water_Vel_Sensor_FFT_SD_Logging_Test]] from week 3 to week 8, with 200 mm/s bin sizes for the steps in hach velocity we get that
 +
{| class="wikitable"
 +
|-
 +
! RMS (%) !! Mean erro (%) !! N !! u (mm/s)
 +
|-
 +
| 420 || 268 || 21973|| 0 - 200
 +
|-
 +
| 141|| 40 || 4591 || 200 - 400
 +
|-
 +
| 51 || -3 || 1758 || 400 - 600
 +
|-
 +
| 34 || -14 || 1596 || 600 - 800
 +
|-
 +
| 34 || -21 || 710 || 800 - 1000
 +
|-
 +
| 37 || -29 || 48 || 1000 - 1200
 +
|-
 +
| 69|| -62 || 12 || 1200 - 1400
 +
|-
 +
| 62 || -55 || 6 || 1400 - 1600
 +
|-
 +
| 81||  -79 || 7 || 1600 - 1800
 +
|-
 +
| 73 || -69 || 6 || 1800 - 2000
 +
|}
 +
The mean error was calculated by finding averaging the difference between the sensor measurement and the HACH measurement, then dividing the average by the center of the velocity range to get a relative figure.
 +
 +
 +
It should be noted that the sensor is not designed to perform below 200 mm/s hence the high error, in practice these results could be filtered out using the point signal strength or the depth of the water. Above 1200 mm/s the error increases substantially again, however making definite conclusions about the error is difficulty as there are fewer than 10 data points. Additionally the hach probe occasionally read sporadically high, hence this may also be contributing to the high uncertainty.
 +
 +
== 10th October 2020 ==
 +
 +
This weeks data is plot below:
 +
 +
<gallery>
 +
File:Plotoct5.png | note hach velocity is now blue an the bosl organge, the darker the point the higher the amplitude
 +
</gallery>
 +
The sensor was cleaned during the battery replacement on the 29th of September 
 +
It is see that cleaning the sensor did not seem to help with the noise issues present, as erroneous high velocities are still being measured when water depths are low. Also at high velocities the sensor appears to consistently measure below the hach value. Other hypothesis include that the sensor's casing is not entirely waterproof and so there is some internal damage to the sensor. This would be easiest to test by destructively investigating the sensor.
 +
 +
We can also now plot the data for the ID008 sensor.
 +
<gallery>
 +
File:Plotoct5008.png|Plot of velocity readings for ID008 sensor
 +
</gallery>
 +
This seemed to have an issue of cutting out occasionally for multiple hours. The data however looks much cleaner than the other sensor, with many fewer sporadically high readings and greater similarity with the hach at higher lower velocities.
 +
 +
Deterioration of the sensor is seeming to play a significant role on the error of the sensor. If the RMS error calculation is run only on the week 3 data then, the results below are obtained, significantly less error than the entire dataset
 +
{| class="wikitable"
 +
|-
 +
! RMS (%) !! ΔR (%) !! N !! u (mm/s)
 +
|-
 +
| 166 || || 4756 || 0 - 200
 +
|-
 +
| 23 ||  || 235 || 200 - 400
 +
|-
 +
| 29 || || 44 || 400 - 600
 +
|-
 +
| 16 || || 394|| 600 - 800
 +
|-
 +
| 17 ||  || 194|| 800 - 1000
 +
|-
 +
| 34 ||  || 14 || 1000 - 1200
 +
|-
 +
| || || 0 || 1200 - 1400
 +
|-
 +
| ||  || 0 || 1400 - 1600
 +
|-
 +
| ||  || 0 || 1600 - 1800
 +
|-
 +
| ||  || 0 || 1800 - 2000
 +
|}
 +
 +
== 11th October 2020 ==
 +
 +
Some issues were being had with the ID008 sensor in the field. It was not logging every minute and sometimes would go hours between logs. The SD card would not log either. I have reduced the BAUD rate on the BoSL- velocity line as hopefully this help in making this transmission more reliable.
 +
 +
Another idea has to see if the poor velocity readings of the sensor in field was to see if a re calibration could be conducted with the received data. To do this the PSD was taken from the FFT's September 16 to September 21. Here the velocity was low, the closest condition to still water in the data. This PSD was then used to rerun the velocity detection algorithm.
 +
The results are below:
 +
 +
<gallery>
 +
File:Sept recal.png
 +
</gallery>
 +
The sensor now appears to track the peaks better and generally perform better during flow events. The number of sporadically high velocity readings has decreased slightly however they still are common in the data. Additionally the sensor seems to not detect low flow events anymore that is that it rarely detects a point at low velocity. This is to be expected in hindsight as when we calibrate at this velocity, it causes the signal at that velocity to be interpreted as a null signal. This test does suggest that some re-calibration of the sensor would be useful.
 +
 +
== 18th October 2020 ==
 +
 +
It was investigated to see whether the dirtiness of the sensor had anything to do with the the poor performance of the velocity sensor. Starting week 9 of [[In_Water_Vel_Sensor_FFT_SD_Logging_Test]] the 5K6 velocity sensor was cleaned and then placed back in the water.
 +
 +
The below are the usual time series graphs for before and after cleaning.
 +
 +
<gallery>
 +
File:Old.png|before cleaning
 +
File:Clean.png|after cleaning
 +
</gallery>
 +
The graphs do not tell have many significant differences. And it is difficult to tell if there has been an change in the quality of the sensor readings. This likely shows that the affect of the dirtiness of the sensor is a small factor in its overall performance.
 +
 +
For reference a photo of the sensor is provided below (its the 10K sensor but these were installed side by side for the same period of time):
 +
 +
<gallery>
 +
File:Sensor clean.jpg
 +
</gallery>
 +
 +
 +
A test was going to be conducted on weather the sporadically high points were consecutive or sparely and uniformly distributed. As this would inform whether running measurement a couple of times until one of suitable amplitude was received is an effective technique to reduce this.
 +
However current all the data is sporadically high, so this test will not tell much. However it can be said that the same sporadically high value is not achieved every measurement and this figure jumps around a lot.
 +
 +
Another issue diagnosed today was the velocity meters in the field logging only intermittently.
 +
This seems to be an issue isolated to the hardware rev: 0.1.3. Whilst the exact issue was not able to be replicated, one which behaved similarly was.
 +
The suspected cause was that the hardware rev 0.1.3 turns on a number of analog components when it takes a measurement. This causes a large current draw and over a long cable with considerable resistance, a voltage drop. This caused the velocity sensor to reset.
 +
The issue was solved by modifying the firmware. Instead of turning on all components at once each component is turned on with a 20 ms delay. This spreads out the peak of the voltage drop and causes the velocity sensor to reset less.
 +
It was also found that a large (470 μF) electrolytic capacitor place at the 3V3 - 5V converter help fix this problem.
 +
 +
 +
== 29th October 2020 ==
 +
 +
The 5K6 sensor used in the velocity testing is now back in the lab. It is currently undergoing tests, before a final deconstruction and investigation. The first test will be a PSD calibration in air, when the sensor is still dry.
 +
The PSD which this resulted in did not show significant improvement when it was used to analyse the data. That is that many sporadic points still existed in it.
 +
The sensor was then placed in water and the PSD calibration was done again.
 +
When this PSD data was used, it was seen that the data did change, most of the sporadic points moved to higher velocitites
 +
 +
== 20th November 2020 ==
 +
 +
Some changes were made the velocity algorithm and now it is working quite well. The falling edge detection algorithm was decided to be used as it produced the cleanest results. In addition this method was improved significantly by running the calculation on the natural log of the data. Some other smaller changes were made. The first was that the amplitude was calculated based on the height difference between the top and bottom of the peaks. The second was a method of reducing spontaneously high points. It was noticed that the edge detection algorithm would latch on to any edge of sufficient height even if it was more a 'bump' than a true edge of the transition between the high and low velocities. To attempt to filter these out, the regions before and after the 'bump' were compared in height if the height difference was not great enough then that edge would be ignored. The actual calculation of this was more technicality involved as it needed to be optimized to run on an ATmega328p.
 +
This algorithm was seen to work well with the velocity distinguished and erroneous points having a small amplitude so they can effectively be filtered out. The algorithm did not work as well for smaller velocities below about 200 - 250 mm/s however it proved challenging to find a good way to extract this data, as it was often drowned in the 1/f noise of the of the low velocity bins. It was found the velocity measured by the falling edge algorithm was about 1.5 times higher than the HACH velocity however, it is hypothesized that this is due to the difference between the mean flow and the maximum speed of dispersed particles in the flow.
 +
 +
 +
<gallery>
 +
File:Velocity Edge Time.png|Velocity Edge Time plot, dark points have high amplitudes
 +
File:Velocity Edge Correlation.png|Velocity Edge Correlation plot, dark points have high amplitude
 +
</gallery>
 +
 +
Below is the data again with a crude filter in place. The filter is set to only filter the data with a low amplitude when the velocity is above 300 mm/s. This is because at velocities below 300 mm/s some of the data with a low amplitude is not noise but the actual flow measurement.
 +
<gallery>
 +
File:Velocity Edge Filter.png|Velocity Edge Filter, darker points have higher velocity
 +
</gallery>
 +
 +
=== Velocity Algorithm Description ===
 +
Below is a python psudo-code implementation of the velocity algorithm used for the results above
 +
 +
            indx_low = 3 #start of fft bin range used
 +
            indx_high = 64 #end of fft bin range used<br>
 +
            max_bin = 0 #stores velocity bin of interest<br>
 +
            #whiten noise
 +
            fftpsd = [binn/weight for binn,weight in zip(fft, PSD)]<br>
 +
            #take natural log
 +
            fftpsd = [math.log(x) for x in fftpsd]<br>
 +
            #gausian blur velocity bins
 +
            fftpsd = gf(fftpsd, sigma = 2)<br>   
 +
            #sobel derivative
 +
            sob = [-1, 0 ,1]
 +
            dfft = [] #array to store the derivative of the FFT array <br>
 +
            #calculate derivative of array
 +
            for i in range(len(fftpsd)):
 +
                try:
 +
                    val = sum([x*y for x,y in zip(sob,fftpsd[i-1:i+2])])
 +
                except IndexError:
 +
                    val = 0
 +
                dfft.append(val)<br>
 +
            ctlow = -0.35 #threshold for edge gradient
 +
            ctinc = -1    #threshold for edge height
 +
            peak_can = False #peak candidate flag
 +
            first_zero = True #peak candidate zero cross flag
 +
            lst_zero = indx_high #bin index of last zero positive zero crossing
 +
            peak_int = 0 #integral of peak ie edge height<br>
 +
            #the following code find the bin with the sharpest edge based on two criterion.
 +
            #the first is that the edge should have a minimum gradient of ctlow
 +
            #the second this that the edge height should be at least 1 (arb). Note that the height of the edge is calculated as the height difference between the first point where the edge the bottom of the edge defined (where the gradient is zero, to the top of the edge where the gradient again crosses zero for the third time. 
 +
            for i,val in reversed(list(enumerate(dfft))[indx_low:indx_high]):
 +
                if peak_can == False:
 +
                    if val > 0:
 +
                        lst_zero = i
 +
                        peak_int = 0
 +
                    else:
 +
                        peak_int += val                     
 +
                    if val > ctlow:
 +
                        max_bin = i
 +
                    elif val < dfft[i-1]:
 +
                        max_bin = i
 +
                        peak_can = True
 +
                    else:
 +
                        pass
 +
                else:
 +
                    peak_int += val
 +
                    if first_zero:
 +
                        if val > 0:
 +
                            first_zero = False
 +
                    else:
 +
                        if val < 0:
 +
                            if peak_int < ctinc:
 +
                                break
 +
                            else:
 +
                                peak_can = False
 +
                                first_zero = True
 +
                                lst_zero = i
 +
                                peak_int = 0<br>
 +
            max_val = dfft[max_bin]  #set threshold parameter as the edge bin of interest found above     
 +
            mean_indx = 0 #variable to store index of velocity <br>
 +
            #remove values greater than 0.2 of the peak of the edge, this allows us to isolate the edge
 +
            dfft = [(0 if (x > 0.2*max_val) else x) for x in dfft]
 +
            def getelt(i):
 +
                try:
 +
                  return dfft[i]
 +
                except IndexError:
 +
                    return 0
 +
            #isolate peak  on the right and left sides by setting everything after the first zero to zero
 +
            is_peak = True
 +
            for i,x in enumerate(dfft):
 +
                if i < max_bin:
 +
                        pass
 +
                else:
 +
                        if (is_peak and dfft[i] < 0):
 +
                                pass
 +
                        else:
 +
                                is_peak = False
 +
                                dfft[i] = 0 
 +
            is_peak = True
 +
            for i,x in reversed(list(enumerate(dfft))):             
 +
                if i > max_bin:
 +
                        pass
 +
                else:
 +
                        if (is_peak and dfft[i] < 0):
 +
                                pass
 +
                        else:
 +
                                is_peak = False
 +
                                dfft[i] = 0<br>
 +
            #calculate the amplitude as the height difference of the edge
 +
            amp = -sum(dfft[indx_low:indx_high])
 +
            #find the center of the edge as an estimate to the velocity
 +
            try:
 +
                mean_indx = np.average(list(range(indx_low,indx_high)), weights = dfft[indx_low:indx_high])
 +
            except ZeroDivisionError:
 +
                mean_indx = 0<br>
 +
            #multiply found index by factor to convert to mm/s         
 +
            mean_indx *= SdPoint.binconv * 0.6#0.6 falling edge mean velocity fudge factor <br>
 +
            #return velocity and amplitude
 +
            return mean_indx, amp
 +
 +
The flowing is a natural language description/explanation of the algorithm:<br>
 +
 +
The raw data received from the sensor is a time series of samples from the ADC which is connected to the mixer circuit. The FFT of the data is then taken. This FFT is then RMS averaged with 7 others. The algorithm described operates on this RMS averaged FFT.
 +
 +
1. The FFT is first whitened against the relevant PSD for the sensor by dividing each bin with the corresponding PSD entry.<br>
 +
2. The log of the whitened FFT is taken.<br>
 +
3. A convolution is applied to the log of the FFT, this convolution filter applies a Gaussian blue of σ = 2, and a derivative operator. Thus the output is an array corresponding to the smoothed derivative of the data.<br>
 +
4. A peak detection algorithm is run. To find prominent edges in the FFT. To do this a threshold is set (see above for specifics) and the derivative array is scanned from highest bin to lowest bin until a bin with at least this derivative is found. The array is scanned from highest bin to lowest bin because all the FFT tend to have a very large spike at the low bins which throws off measurements.<br>
 +
5. Now the 'bump' detection is applied. To do this, starting from the smallest bin above the bin found in 4. which is negative, and descending, the derivative values are summed until the just before the bin where the derivative becomes negative again. This is approximately the next height change over the edge or 'bump', if the net height change is greater than a threshold it is determined that the bin from 4. does indeed correspond to and edge rather than a 'bump' which would return to the same height. If a 'bump' is detected 4. is repeated starting from the bin below the found 'bump'.<br>
 +
6. Now that a bin in the edge of interest is found it needs to be isolated from the data, to do this any bins after the first zero on the right and left of the edge are set to zero. This creates an array where the only data is the derivatives at the edge of interest.<br>
 +
7. The amplitude of the point is now calculated as the area under the derivative peak, or the height of the edge.<br>
 +
8. The velocity is calculated as the weighted mean bin of the derivative peak. This is then multiplied by a conversion factor of get to mm/s, a factor of 0.6 is also included in this which is thought to represent the difference between the mean flow and the fastest moving particles in the flow.
 +
 +
== 23rd November 2020 ==
 +
 +
The RMS error can be recomputed using the new algorithm. The results are below, the errors are all less than the previous algorithm.
 +
{| class="wikitable"
 +
|-
 +
! RMS (%) !! Mean error (%) !! N !! u (mm/s)
 +
|-
 +
| 53 || -11 || 26275|| 0 - 250
 +
|-
 +
| 44 || -22 || 4966 || 250 - 500
 +
|-
 +
| 26 || 4 || 3059 || 500 - 750
 +
|-
 +
| 18 || 0 || 1764 || 750 - 1000
 +
|-
 +
| 42 ||  -27 || 83 || 1000 - 1250
 +
|-
 +
| 84 ||  -82 || 5 || 1250 - 1500
 +
|-
 +
| 93 || -93 || 10 || 1500 - 1750
 +
|}
 +
 +
== 5th April 2021 ==
 +
 +
An explanation was found for the factor of 0.6 in the velocity algorithm first discussed [[In_Water_Vel_Sensor#Velocity_Algorithm_Description | here]]. This was not in-fact a constant of proportionality between mean flows and fastest moving flows or anything like that but rather just a calibration error. The calibration in the velocity code the 69.7 hertz per bin corresponds to a sample rate of 17.8 kHz. The real sample rate of the sensor was measured to be closer to 11.1 kHz. Taking their ratio we get 0.62, the factor which was multiplied for in the code.
 +
 +
== 16 April 2021 ==
 +
 +
Some testing was done to measure the current of the sensor. Sensor ID015 was used. At first the sensors couldn't be communicated with. It was found that the cable was too long for the serial connection so the cable was shortened a bit. The firmware revision used was 0.1.4.
 +
 +
{| class="wikitable"
 +
|-
 +
! Power State !! current (mA)
 +
|-
 +
| Measurement || 27.0
 +
|-
 +
| Active|| 12.8
 +
|-
 +
| Sleep|| 0.109
 +
|}

Latest revision as of 00:08, 22 December 2024

This section documented our journey to develop a low-cost water velocity sensor based on doppler effect!

Check out the full paper!: https://doi.org/10.3390/s22197451

For information on specific velocity sensors see: BoSL Velocity Database

Contents

20th November 2019

So the first task we are going to try is to use Arduino to generate 40 kHz square wave and emit that signal by using a ultrasonic transducer.

By facing another same transducer to the one emitting the ultrasonic signal, we can successfully detect that signal.


Experiment design.png


Figures below show the generated square wave from the emitting transducer (measured by an oscilloscope), and the detected wave signal from the receiver transducer (by using AnalogRead).

Siliscop 40khz square wave.jpg Arduino detetcted wave.png


The figure here shows how we wire up the transducers with Arduino to conduct this quick test. The code we used is also attached here to show you how to use Arduino to generate a certain frequency.

Schematic diagram.png


Code.png


21th November 2019

What are we going to try today is to place the two transducers next to each other and facing the same direction.

One emit the signal to a metal wall, and the receiver to detect the signal bounce back.

Multiple op-amps will be used to strength the signal to make it strong enough for Arduino to detect.

Bouncing back.png

25th November 2019

For the arduino to effectively read the low amplitude high frequency signals it is going to need some assistance via external circuitry.

We tested an non-inverting op-amp design based on the LM324. This proved to have challenges reproducing the high frequency signals and after some diagnosis, we determined that the slew rate of the LM324 was too low.

LM324N Waveform in Slew Limited Domain

The image above shows the limited slew rate experienced from the LM324N. In this domain the waveform appear as linear as the OpAmp attempts to match the proper signal. It demonstrated a slew rate of about 0.5 V / μS. To overcome this we will need to replace the LM324N with a better OpAmp.

By default the Arduino ADC sample frequency is rather slow at 45 kHz. For our application we will need much faster sampling rates. Fortunately, the clock divider which the ADC runs on (default 32x) is user settable. In testing various values for this, a sample rate of 615 kHz was able to be achieved. <1>

This was accomplished via the following code:

Fast ADC Code.png

26th November 2019

We tested out some LM883N and NE5532AP OpAmps to see if they performed better than the LM324 (they have a slew rating of ~5 V / μS so they should). Both worked much better than the LM324 being able to reproduce the waveform however, we found that the NE5532AP was better at higher gains.

It was found that the circuit below performed well at amplifying the AC signal of the receiver resonator.

Amplification Circuit for Ultrasonic Receiver

The gain can be adjusted through resistors Ra and Rb via the formula: Gain = 1 + Ra/Rb

Issues were faced where very high noise was present on the output of the op-amp. Some investigation revealed that this was due to the same Arduino driving the 40 kHz squarewave as powering the op-amp. This limited the ability to receive low amplitude signals and made it difficult to detect the reflected ultrasonic waves. A temporary solution to this was to use separate Arduinos for the transmitted and receiver.

A noise profile was obtained by having the ultrasonic receiver covered and placed such that no ultrasonic frequencies would be measured by it. The op-amp output was then measured. Below are the measurements for the two-Arduino setup vs the one-Arduino setup.

The noise is improved in the two-Arduino setup however there is still a significant ammount, it may be necessary to make a PCB with better noise isolation for test which require higher sensitivity.

  • One-Arduino Noise Profile

  • Two-Arduino Noise Profile

Arduino Sample Frequency

<1> From doing more test it appears that not all Arduinos are able to sample at the 615 kHz rate, a more reliable max appears to be 215 kHz.

Testing the values of the clock divider registers ADPS2, ADPS1, ADPS0, the following table was made indicating about the various sample frequency on a 16 MHz 5 V ATmega382P (Arduino Uno).

ADPS2 ADPS1 ADPS0 Sample Frequency (kHz) Working
0 0 0 325 No
0 0 1 325 No
0 1 0 215 Yes
0 1 1 125 Yes
1 0 0 66 Yes
1 0 1 35.6 Yes
1 1 0 17.6 Yes
1 1 1 8.9 Yes

Arduino Frequency Measurement

To measure frequency an DFT was performed on a 128 size sample at 215 kHz of the amplified ultrasonic receiver signal.The sample frequency was chosen as it was the highest stable, and the sample size as this was the limit of the ATmega382p's memory.

Arduino FFT of Ultrasonic Receiver Signal

The y-axis height represents strength of that component frequency, and the x-axis the frequency bins, where eight bins corresponds to 1.7 kHz

Concerningly the peak height is at about 90 kHz, and the predominant frequency when measured with an oscilloscope is 40 kHz.






27th November 2019

Some thought was conducted into the feasibility of direct frequency measurement using the ATmega328p and extracting the Doppler shift digitally.

The velocity is given by the equation: v = c/2 (fd/f) cos(θ), where c is the speed of sound, fd is the Doppler shift, f is the emitted frequency, and θ is the angle between the flow and sensor.

Let us assume that the sensor is parallel to the flow (θ = 0). By Nyquist-Shannon sampling theory, to detect a frequency at f we need the ATmega328p to sample at 2f. Applying a DFT to this sample gives a frequency resolution Δf of 2f/N, where N is the number of samples. Therefore, making the substitutions fd = 2f/N, θ = 0, we get: Δv = c/2 (2f/(f*N)) = c/N

For water (c = 1500 m/s) and a velocity resolution of Δv = 0.1 m/s this implies that 15 000 samples would need to be taken. This would require an amount of memory well beyond the 2 kB of the Arduino (which can realistically only store 128 samples).

This implies that direct measurement of the Doppler shift is not feasible and that some signal pre-processing is necessary. The idea is to extract the Doppler shift so that we can have a easier time sampling the lower frequency Doppler shift signal.

There are two approaches which we are going to trial, amplitude demodulation, and a heterodyne converter.

Amplitude Demodulation
AM Demodulation Block Diagram of Doppler Shift Signal

The premise of the AM demodulator is that the sum of the two signals produces a beating frequency (f' - f) which is conveniently also the Doppler shift fd. The rectifier eliminates the negative peaks of the resultant sum, and the low pass filter blocks the high frequency carrier wave of the signal, leaving only the low frequency Doppler shift.

The draw back of this method is that the signal is very weak, and it is difficult to scale to higher frequencies.

A pathfinder version of this circuit was constructed as indicated in the diagram. The inputs were provided by two Arduino Unos with one generating a 40 kHz 5Vpp Square wave, and the other a 43.4 kHz square wave. The output as measured by an oscilloscope was good, displaying an about 3.4 kHz wave which was the difference in the two frequencies. The output was however, a bit noisy, and low in amplitude (~500 mV). Building this on a proper PCB should help eliminate some of the noise and allow the output to be amplified.

  • AM Demodulation Circuit Schematic

  • AM Demodulation Circuit Test Setup

  • AM Demodulation Circuit Output

The 43.3 kHz Arduino was then disconnected from the circuit and the ultrasonic transducers were wired into the circuit, (with some appropriate changes to the resistor values). The transducers were then placed so that they were facing each other and then the receiver was moved back and forward at a steady rate. The output of the demodulator was able to detect the Doppler shift and thus this motion, however much noise was present and the signal was weak.


  • Transducer Arrangement

  • AM demodulator circuit output with ultrasonic transducer inputs

Heterodyne Pre-Processing
Heterodyne Pre-Processing Block Diagram of Doppler Shift Signal

The Heterodyne is more standard for high frequency Doppler shift applications, the mixer multiplies the two signals. For two sine waves with frequencies fa and fb, their product will be the sum of cosines with frequencies |fa-fb| and |fa + fb|. For fa = f', and fb = f, the lower frequency signal is the Doppler frequency, which can then be isolated from the higher frequency using a low pass filter. This method provides the advantage of it being proven to work at a wide range of frequencies, in excess of 500 MHz.

28th November 2019

Following on from yesterday, I tested to see if the Arduino could analyse the output of the AM demodulator circuit. The output of the circuit was connected analog pin A1, and the below code was run.

The code, generates a 40 kHz 5 Vpp square wave on pin 3, then samples at 100 Hz on pin A1 for 128 samples. This data is then passed on the arduinoFFT library when does a DFT on the data to convert it into the frequency domain. The frequency domain data is then plotted to serial.

Running this code while moving the ultrasonic sensor at about 4 cm/s. The FFT in the gallery was the output.

  • FFT Ultrasonic Analysis Code

  • FFT Ultrasonic Output

The peak is seen at about bin 10. The bin frequency is given by fb = fs/N, where fs is the sampling rate (100 Hz in this case) and N the number of samples (128). This gives a bin frequency in our case of 0.78 Hz. As we are interested in bin 10, our Doppler frequency is 7.8 Hz.

Using the formula: v = c/2 (fd/f) => v = 340/2 (7.8/40 000) => v = 3.3 cm/s

Which is within our margin of error for the velocity, and indicates that the principle behind the sensor is as observed.

AM Demodulator PCB

PCB drawn up to take the AM demodulator off the breadboard and hopefully reduce some noise. Off to the Voltera!

  • AM Demodulator Schematic

  • AM Demodulator PCB Render

Note: C1 was changed to from 1 μF to 100 nF as it had better performance.

The circuit works well, much the much better noise floor creates greatly improves the signal to noise ratio. The signal is still a bit weak however some more amplification may help this out. An oscilloscope is able to detect the Doppler shift when the transducers are facing each other and one is moved from about 1 meters distance, and from about half a meter when the signal is bounced off a wall. It is hoped that optimization of the values of the components can improve on this.

Additionally, it still needs to be tested if the Arduino can detect the signal from this.

When the traducers are facing each other, the Arduino is able to read the voltage and perform the analysis on it. Below is an FFT plot from the Arduino in this arrangement.

  • AM Demodulator PCB

  • Arduino FFT Plot of Ultrasonics Using AM Demodulator PCB

Wrote some code to use the frequency data to determine the speed. Initial tests indicate its working well, although a proper trial with calibrated speed references will be necessary to be sure if its working properly.

29th November 2019

Some tuning was done to the AM demodulator circuit to improve performance. One issue found when probing with the oscilloscope is that the output of the opamp was being clipped as the amplification was too large, and so the resistor R6 was changed to 5 kΩ to prevent this.

3rd December 2019

Some testing was done into if it would be possible for an Arduino to produce a 1 MHz frequency, which will be needed if we want to switch to a 1 MHz transducer. It was able to generate the waveform, however, significant ringing did occur on the rising and falling edge. This may be able to be alleviated with a capacitor if it becomes an issue.

Heterodyne Circuit

In order to switch to using higher frequencies the heterodyne circuit topology is going to be needed. A circuit based on the SA612AD is going to be build according to this schematic.

Building this circuit, it seems to perform effectively, and with a output signal much cleaner and stronger than the AM demodulator. The waveform in the gallery was taken with the two 40 KHz ultrasonic sensors pointing towards a wall about 30-50 cm away. The amplitude of the signal being about 80 mV should be able to be detected by the Arduino, however if not the output signal is SNR is low enough that amplification could be done. Furthermore, it seems as though the low pass filter has a much easier time of isolating the low frequency component of interest when compared to the AM modulator, another benefit of this design.

For two 40 KHz ultrasonic sensors facing each other at about a half a meters distance, the Arduino was used to record the output of the Heterodyne Circuit. The FFT of the signal is shown below. It is seen that the signal produces a much more distinct peak at the Doppler frequency, which will help the extraction of the velocity from this output.

  • SA612AD Ultrasonic Application Circuit.

  • Arduino Generated 1 MHz Square Wave

  • Output waveform for Heterodyne Circuit

  • Heterodyne Circuit On Breadboard

  • FFT of Heterodyne Circuit with 40 KHz Ultrasonics

In investigating the SA612AD, the two RF inputs IN_A and IN_B can be used to form a ballanced line. Connecting either side of the Ultrasonic transducer to a these pins through a 100 nF series capacitor, reduces the noise which is introduced into the amplifier, and so improves the sensitivity of the circuit.

1.7 MHz Tranducers

Having success in measuring the Doppler shift at the lower frequencies, it is time to test out the higher frequency ultrasonic transducers.

The specific piezoelectric disk which we going to be testing is: https://au.mouser.com/ProductDetail/PUI-Audio/UB161M7?qs=pFWzdYhrkoc8QGl1JZ231g%3D%3D

It has a rated resonant frequency of 1.7 MHz, however the Arduino is only able to produce frequencies an integer division of its clock frequency (16 MHz). It was found that out of the frequencies which the Arduino could generate (16 MHz / 9 = 1.78 MHz) produced the largest resonance in the ultrasonic transducers.

These disk do not come with any wires soldered to them and so this needed to be done manually. Something which was quite difficult as the solder contacts ideally should be quite small.

In modifying the above Heterodyne circuit for use with the 1.7 MHz tranducers, I found that the grounding on the breadboard is not good enough for a low noise output. I might have to make a PCB for it, or add better grounding.

The noise was not able to be solved easily, and so a PCB is going to be made.

4th December 2019

Little success was had with the PCB, the higher frequencies of the 1.7 MHz tranducers results in many more complications in the circuit design. Additionally, with the multiple problems present it is difficult to isolate and diagnose how each part of the circuit affects the overall performance.

PCB of Heterodyne Circuit

In testing the circuit works well at 40 KHz, better heterodyne circuit built on the breadboard. As the IC is rated for operations up to hundreds of MHz, this narrows down the source of the issue to be either, the ultrasonic tranducers, the signal integrity, or the values of some of the components being unsuitable for MHz work.

Transducer Resonant Frequency

I measured the resonant frequency of the two 1.7 MHz traducers using the oscilloscope. The values were quite a bit off from 1.7 MHz with one being 1.64 MHz and the other 1.63 MHz. It is likely that this shift is because of the wires which needed to be soldered on to the traducers. This shift could also be one of the reasons that the circuit is working.

In re-reading the datasheet and conducting some testing on the tranducers, they are a type of transducer known as a bender. This type is more suited towards high voltage application 10s of Volts, and is not suited to detecting small pressures as are those found in ultrasonic air waves. These tranducers do not have the characteristics necessary for our application, and so hopefully the ones in the post are better.

1 MHz Transducer

I had a bit of a test with the 1 MHz ultrasonic tranducer: https://www.digikey.com/product-detail/en/unictron-technologies-corporation/H2KLPY11000600/2047-H2KLPY11000600-ND/9921487, to see if it was able to detect a reflected signal.

The setup was to send 20 1 MHz pulses to the transducer and then use an oscilloscope to monitor if any reflected signal was present.

In air this was not able to be done. Varying the distance didn't seem to help either no reflected signal ever being detected in the osciloscope.

In water however, the results were quite successful. The waveform attached is of the transducer facing down into a cylinder of water about 15 cm high, the surface of the transducer was touching the water. As can be seen the reflected waveform produces a distinct peak, some time after the driving pulse is stopped.

The reason that the trial in water worked, whereas the one in air didn't is probably to do with impedance matching. That is at higher frequencies, much less of the vibrational energy gets transferred from the tranducer to the low density and slow speed of sound air than compared to the more dense and higher speed of sound water. This is something which we will need to pay close attention to as we wish to maximize the amount of energy that gets transferred into the medium, so we can have an easier time detecting the reflections.

Reflected Ultrasonic Signal In Water


5th December 2019

We found some new 1 MHz ultrasonic transducers, and they seems to be more suited towards transducer application: (https://www.steminc.com/PZT/en/piezo-ceramic-disc-10x2mm-r-215-khz-wire-leads-smd10t2r111wl).

I tested the resonant frequency and found it to be the quoted 1 MHz.

Good news! The transducers are transduceing, that is to say, using the oscilloscope you can detect the oscillation of the driven transducer with the receiver transducer. It is important to note here that these tests were done in water as in air the coupling is too weak.

The tranducers also seem to have a resonant frequency at just undre 250 kHz, about 243 kHz. This one actually seems to be stronger, with more signal being transduced at this frequency.

The reflected pulse off the bottom of a cylinder of water was not able to be measured by this transducer, as with the previous 1 MHz transducer.

In doing some preliminary testing, it is looking like all of the MHz tests are going to need to be conducted in water. Additionally, it appears as though the container shape matter as the ultrasonic waves tend to reflect off the walls and create resonances in the cylinder, this interferes with the signal.

With some adjustment to the capacitor C? from 68 pF to 170 pF, I was able to get a signal out of the Heterodyne circuit when both of the transducers were facing each other. The signal was weak and still had a significant 250 kHz component, but it was able to be measured with an oscilloscope.

Doppler Shift using 250 kHz transducers in water

9th December 2019

Some new ultrasonic transducers arrived. They are already enclosed and wired up so they look more promising than the other ones.

I used the function gen and oscilloscope to get a precise reading on the resonant frequency in air and it is about 1.018 MHz. This value was obtained by finding the local minimum in amplitude across the traducer, with a series 330 Ω resistor, when sweeping through the frequencies using a function gen.

In water the resonant frequency was 0.999 MHz, which is within the measurement error for the manufacturers quoted 1 MHz

I conducted another test in which I applied a 1 MHz frequency to one transducer in water and measured the voltage across another transducer with an oscilloscope. A few good things to note from this was that more voltage across the input transducer produces a stronger signal on the output transducer. The relationship is not exactly linear but it is fairly proportional, with diminishing returns occurring at larger voltages. Also, the arrangement of the transducers is very important, they need to be facing each other or have a good path through a reflection off a wall. Otherwise the signal is almost undetectable.

I hooked up the transducers to the Heterodyne circuit with the Signal Gen providing the 1 MHz input wave. Using an oscilloscope I am able to detect the motion of the transducers. The signal is very weak and still superimposed with the 1 MHz input wave so I am unable to upload a meaningful image of it yet.

Passing the signal through a low pass filter (f = 4.5 kHz), the doppler shift becomes much more visible on the oscilloscope. In this setup the transducers were placed such that the wave reflected off the bottom of a cylinder of water. The input wave did have to be increase to 9 Vpp, however this can be reduced later once the circuit is tuned correctly. Attached are two images one of the output when the receiver is still and one when it is being moved very slowly, (> 1 cm/s). The Doppler frequency is clearly visible when comparing these two, I suspect that the low frequency wave in the no motion waveform is an artifact of the oscilloscope.

  • No Motion

  • Motion

At this point, the circuit-transducer combo is quite well tested, and it is time to look at getting a proper PCB manufactured and encasing the transducers.

12th December 2019

A housing was printed for the ultrasonic sensors to keep them well aligned and as a prototype for the final packaging of the sensor.

Velocity Sensor Housing

16th December 2019

A schematic for the PCB was drawn up. It features an onboard 1 MHz clock generator to provide the stable 1 MHz signal needed for the Tranducers, and it outputs the low frequency Doppler shift.

A rendering of the PCB is also shown in the gallery.

  • Heterodyne PCB Schemati

  • Heterodyne Render

19th December 2019

The heterodyne circuit and the 1 MHz tranducers were tested in the 3D printed casing in a water sink. The output was very good for movement of the sensor, (photo in gallery), with the signal being many times the noise level. Granted, this was with a input voltage of 9 V, but this would be able to be reduced in the future.

Motion of the water, however, proved to be more difficult, it was generally undetectable, expect for very fast velocities. The signal to noise amplitude ratio never exceeded one. This domain is the one we are interested in to measure the fluid velocity, so much more optimisation of the design will be needed if the signal is to become Arduino readable. It is noted that tap water was used to fill the sink, which lacks the particles necessary to get a good reflected ultrasonic signal. It is presumed that in storm-water their will be more particles present so this will help in making the signal stronger.

The water velocity signal was too weak to get a useful screen grab from the oscilloscope.


  • Oscilloscope output for motion of sensor

  • Sink used for velocity testing

22nd January 2020

The new PCB designed to reduce noise and enable a higher sensitivity reading was soldered and tested today. The noise floor for a 5 V 1 MHz Arduino generated square wave was about 20 mV. When tested in water the Doppler shift off a hard surface such as the sink wall was able to be clearly detected, however no signal was had with just the flowing water. The J4 output was much worse performing than the JP1 voltage point. This is likely due to the ground noise making the low pass filter (C8, R11) ineffective. The balanced line amplifier has not been tested yet so it is still to be seen if this method produces a cleaner output. Grounding the EMF shield of the Doppler transceivers had little effect on the noise performance of the device.

  • Custom PCB Heterodyne Circuit

Switching from a driving signal generated from the Arduino to one generated from the integrated oscillator, the noise floor dropped to below 5 mV, a substantial improvement. However after the traducer was plugged in the noise increased substantially. Reflections were still able to be detected off hard surfaces. Tests as to whether it could detect flowing water were inconclusive. This time the (C8, R11) low pass filter did help improve the signal quality. At this point I am thinking that the much increased noise level has to do with the much lower impedance of the transducer when compared to the open circuit, thus requiring larger current draws from the oscillator and creating more interference. One possible method of trying to overcome this is to not use the direct signal from the 1 MHz square wave oscillator to drive the transducer but instead to take the first pass it through the low pass filter system which produces a sine wave for pin 3 of the mixer and then amplify that using the unused channel of the op amp. This should work better as the sine wave does not have any higher than 1 MHz harmonics like the square wave does and so the amplification of this signal should produce less interference.

After testing amplifying the sine wave works to reduce the interference of once the transducer is plugged in. However the problem is now that the op-amp selected does not have a high enough gain at the 1 MHz signal. A gain of around 25 would be desired. The NE5532 used can only achieve a gain of 4 or 5 at this frequency. The two best options to overcome this would be to look for an op-amp which has better characteristics at this frequency or to implement a two stage amplifier in which the output of one op amp is fed into the next.

Testing this setup in water it seems that the 1 V signal from the single NE5532 is still strong enough for the returned signal to be measurable. Due to the low noise the signal is now very clean and easily readable, so it is worth considering whether the driving voltage for the transducer needs to be amplified more or whether it is good as is. Water velocity flow is still unable to be detected. The signal off the reflection from water is at least several times weaker than hard surfaces, and it is hard to get a consistent flow of water in the sink I was using so it was difficult to see whether the signal was due to the flow of water or just random noise in the circuit. Having a large steady flow of water in some test apparatus would help with this, as would using the balanced line amplifier on the circuit to get a better resolve the small signals.

3rd February 2020

Analog characterisation of the heterodyne circuit

The heterodyne circuit was tested for its analog performance using the following circuit

  • Analog Characterisation Circuit

The circuit was powered on and then the function generator was used to imitate the return signal from the receiving ultrasonic traducer. First the frequency of the function generator outputting a 100 mVpp sine wave was tuned such that the frequency matched that of the in-circuit oscillator. This frequency was found to be about 1.000 32 MHz +- 1.000 01 MHz. From here frequency was shifted up to 1.000 350 00 MHz to produce a signal on the output of the circuit, a frequency which roughly corresponds to a flow velocity of about 2 cm/s. The amplitude of this signal was measured with the oscilloscope. This method produced the graphed relation between input signal amplitude and output signal amplitude.

  • Graphed data of input vs output signal amplitude

From the graph it is seen that the noise floor is about 20 mV however voltages as small as 5 mV are still able to produce a distinguishable signal. The device has good sensitivity to small input signals and amplifies the input with a gain of about 3. The voltage produced by a ultrasonic receiver in water has not yet been tested, however it is likely to be quite small, so any performance improvements achievable here would be welcome. To feasibly measure the signal with the Arduino the voltage amplitude needs to be on the order of 10s of millivolts ideally greater than 50. To accomplish this the on-board amplifier will now be activated and the device will be retested.

The in-circuit amplifier turn out to be difficult to get working. Issues with the design of the amplifier lead to it being very unstable with positive feedback loops oscillating the signal between both supply rails. An amplifier more suited towards amplifying the balanced line output of the SA612AD and should be implemented on the next revision of the PCB. The new amplifier only has 20 dB/decade roll-off however this should be sufficient to filter out the 1 MHz carrier from the 100s Hz signal.

Other design ideas which could be implemented to help out with the signal acquisition could be to use the Aref pin of the Arduino to feed in a 100 mV signal, thus making the range of the ADC smaller and the measurements more precise. Another idea is to add an independent I2C ADC onto the velocity circuit board. This has the advantage of using fewer wires in the cable and also dose not need to analog signal to travel the long distance up the cable where the signal quality would be degraded due to interference.

The signal was connected to an Arduino and an FFT was taken on the data (N = 128, fs = 1000 Hz). The arduino was easily able to detect the frequency shift of a 1 mV signal input into the receiver ultrasonic transducer. The memory limitations of the Arduino create range restrictions on the data. At a range which would include the flow velocity of 2 m/s the resolution is 0.03 m/s. To overcome this traditionally would require more samples to run the FFT on however as mentioned the Arduino does not have the memory for this. Another more resource efficient solution is to take two reading, one at a higher sample frequency and one at a lower sample frequency. This way the lower sample frequency will have the resolution to distinguish slow flow velocities, and the high sample rate fast velocities.

4th February 2020

The velocity sensor was tested in the flume to check to see if it is able to receive a signal in the water to any extent. The velocity sensor was attached to the flume as shown in the photograph below.

  • Velocity Sensor Attached in Flume.

  • Velocity Sensor Submerged in Water

The sensor was originally attached through a 2 m Cat-5e Ethernet cable however this was found to diminish already small signal too much, so the Arduino was connected directly to the Heterodyne circuit. Once integrated and installed in-situ this would be unfeasible and so it is going to be necessary to include a ADC converter on the circuit.

In this arrangement the FFT was able to distinguish a peak at the Doppler frequency of the flow. The peak was very weak and not apparent on all sample readings, however it was present and observed to shift left and right with the changing speed of the water in the flume. When the maximum frequency (converted to water velocity) was printed to serial and compared to a proper water probe the reads showed some correlation with the BoSL sensor being within 50% of the proper probe. The current signal analysis is not good for velocities above 330 mm/s or below 50 mm/s so dynamic sample rate adjustment will need to be implemented to solve this

  • Slow Water Velocity

  • Medium Water Velocity

  • Fast Water Velocity

The data was taken with the BoSL probe and a FloMate probe in the flume at a range of water velocities, the graphed data is below, the series of orange points is the 1:1 curve.

  • Calibration curve of flow velocity measurement from FloMate probe vs BoSL probe

The graph shows reasonable correlation between the FloMate probe and the BoSL probe, with it measuring consistently less at higher velocities, however being better in the region below 0.1m/s flows.

7th February 2020

Sample Intervals

The Arduino FFT code was tested to see how reliably it measures frequency. The function generator was used to generate a 100 mVpp sine was inputted directly into analog pin A1 on the Arduino. The Arduino sampled this wave at 2000 μs, 1000 μs, 500 μs and 250 μs intervals for 128 points and then an FFT was performed on the data. The maximum frequency of this FFT was then recorded as the measured frequency. A plot of a range of frequencies is shown below.

  • Plot of Measured Frequency vs Input Frequency for a Range of Sample Intervals

The linearity of the frequency measurements is good with all sample intervals being roughly concordant with each other and the input frequency up until their max frequency threshold. At this point, the measured frequency decreases linearly with increases in the input frequency until the measured frequency is zero again, it then goes back to increasing linearly until max frequency threshold, with this cycle repeating with some drifting occurring in later after several 'reflections'. This phenomena provides an interesting way to extract more resolution out of high frequency at a lower memory cost than increasing the number of sample points. The small sample interval FFT could be used to get a rough idea of the frequency and then this could be refined by checking where the large sample interval is between its minimum and maximum sample frequency.

Circuit Supply Capacitor

The heterodyne circuit currently uses a 330 μF capacitor on the power input between +5V and GND. This is unideal as everytime the device is to be switched on this capacitor needs to charge up, requiring a large amount of current, and so using up the battery quicker. The capacitor is necessary to smooth instantaneous surges in current draw from the circuit, reducing the noise, however the capacitance of the capacitor is likely able to be reduced without significantly affecting performance. To test this the FFT plot produced by the Arudino with no input signal on the heterodyne circuit (but the ultrasonic tranducer still plugged in) was recorded for input capacitance of 0 μF, 3.3 μF and 330 μF with plots included below.

  • 0 μF Noise Profile

  • 3.3 μF Noise Profile

  • 330 μF Noise Profile

The four regions of signal are the four FFT at sample intervals: 2000 μs, 1000 μs, 500 μS, and 250 μs. From these plots it is seen that having no capacitance increase the noise by about 30% - 50%, however having a 330 μF capacitor does not significantly decrease the noise over a 3.3 μF capacitor, and a 3.3 μF capacitor will be used going forward.

Downstream Flow Testing

The velocity sensor was positioned to face downstream in the flume, and the reading was compared to the FloMate to produce the following graph.

  • Plot of Upstream BoSL Measured Velocity vs FloMate Measured Velocity.

The data is very non-linear demonstrating that the BoSL velocity sensor does not work when facing downstream. One possible explanation to be investigated is that the probe has a flat rectangular shape and so the flow behind the probe is severely disrupted, having a more turbulent flow. Making the upstream facing profile of the sensor better maintain flow could be helpful in mitigating this.

Upstream Flow Testing

The velocity sensor was then turned around to face downstream and the performance of the multiple sampling intervals was tested. For each flow velocity the average reading over one minute is plotted.

  • Upstream Velocity Testing: BoSL Measured Velocity at Varying Sample Intervals vs FloMate Measured Velocity

The error bars represent the quantisation range of the FFT, thus giving some reading of the error in the value. The dashed line shows the linear correlation of an ideal sensor. The sensor shows good agreement with the FloMate for at least one of the sample intervals per point. The slower sample intervals perform better for the slower velocities and the faster, the higher the 250 μs sampling interval has the unideal behavior of measuring far off form the actual value when the flow velocities drop below about 80 mm/s, this should be carefully considered when converting between the four given velocities and a final velocity figure, as it could be mistaken for a faster flow.

10th February 2020

An ADC is to be implemented on the heterodyne circuit board to reduce signal quality losses in the analog transmission of the signal along a wire. A MAX11609 was first tested to see how it performed at sampling quickly. The ADC is an I2C device and so a small code loop was written to test the sample frequency. A frequency gen was connected to the ADC and the amount of sine waves which could be sampled in a given number of samples was counted to find the sampling frequency.

  • I2C ADC Code Loop

The sample rate was found to be about 2.7 ksps

Enabling External Clock Mode the sample rate was able to be boosted to about 4.9 ksps

The MAX11645EUA+T ADC's arrived, featuring an extra 2 bits of resolution. In external clock mode these ADC's can reach sampling rates of 5.29 ksps. Differential probing was also able to be used meaning it is ready for integration onto the heterodyne PCB.

The heterodyne PCB was modified to use the MAX1645EUA+T ADC instead of the analog output. In order to accomplish this the unbalanced low-pass filter was disconnected and two 330 Ω resistors and a 33 nF capacitor were used to make a balanced low pass filter between the two outputs of the mixer. The outputs of this filter were then directly connected to the inputs of the ADC. The ADC was secured to the circuit using a breadboard with also served to connect the digital I2C bus to the Arduino. In the current configuration the ADC is running of a 3.3 V supply whereas the rest of the circuit is running of a 5 V supply. The main concern with switching the whole board over to 3.3 V is that it is uncertain how the reduced output of the ultrasonic transmitter, due to its lower driving voltage, will affect the strength of the signal returning in the ultrasonic receiver.

  • Heterodyne PCB Front

  • Heterodyne ADC Back

11th February 2020

The new ADC displays much improved FFT amplitude resolution and is able to distinguish a small frequency peak much better than the Arduino as able to. The FFT below is of a 1 mV signal, it can be seen that the SNR is now well above one hopefully implying that the Doppler peak in the water is much easier to resolve.


  • FFT of a 1 mV Sine Wave with New ADC


With the new ADC the sample rates change due to having a different code sequence, and a new calibration of the FFT need to be conducted. To do this, a frequency gen was connected to the input of the ADC and a 60 mV sine wave was applied at various frequencies. The FFT bin over 4 averages which this fell into was then recorded the results were graphed.

  • FFT Calibration Curves

The graphed region for each sample interval is when they are below their threshold frequency. As seen the output for all of the sample intervals is quite linear and the gradients can be used to convert from the FFT bin into an actual frequency measurement for the Doppler velocity calculations. The sample intervals could be changed to get a better spread over the input space if required.

The sensor was then tested in the flume again however the water velocity was not able to be measured this time as there were prominent spikes in the FFT not related to the water signal. The

  • ADC Velocity FFT With Spikes

These spikes were able to be reproduced by running the cables for the ultrasonic receiver and emitter next to each other, and so it is likely that this is the source of the interference. Connecting the shielding of the transducers to ground eliminated this issue in the out-of-flume testing.

12th February 2020

The circuit was switched over to 3.3 V and on first inspection it seems to be working well. The input into OSC_B on the mixer was found to drop to just about 200 mV when the oscillator was powered using 3.3 V, above the minimum threshold.

A re-calibration of the FFT frequency bins will need to be completed to get a better spread over the potential range of velocities.


The length of the cable was tested and it was found that the sampling speed changed slightly with a longer cable, this is problematic and it is likely that code changes will need to be made to make the timing more resilient. The cable length of 2 meters was able to be used on the 100 kHz speed but smaller pull up resistors (1 kΩ) were needed to be able to communicate with this sensor in fast mode (400 kHz). The downside of this is that it will draw more power and so might need some circuitry to disconnect these pull-up resistors when not in use.

This value of pull-up resistor did not work for a cable of 10 m length.

Peak Finding Algorithm

The previous algorithm used to determine the maximum frequency was to find the FFT bin with the highest amplitude. This worked alright however it suffered from the limitations of not accounting for the width of the peak and being rather jittery. The need to account for the width of the peak is important as for variable flows the Doppler frequency changes slightly and so the peak is spread over multiple bins, reducing the maximum amplitude. The area of the peak, however is more consistent in this case less susceptible to noise.

The peak finding algorithm implemented now works by first isolating the predominant peak and then finding the weighted average of this peak. Specifically, it first makes the mean of the FFT data set zero, following this it subtracts 3 and makes all negative values equal to zero. This works well to get rid of most of the baseline noise and leave only peaks. Following this the highest point is found and then the all values which are not after the first zero bin on each side of this peak are set to zero. This ensure that only one peak is in the data set if there happened to be another due to interference or excessive noise. The weighted average center of this peak is then found and this value is used as the FFT bin for the Doppler shift. The sum of all the bins in the peak is also taken and if this value is less than 10 a flag is raised indicating that the peak was small and might be due to noise rather than actual signal.

13th February 2020

Due to the wire length limitations found yesterday it is being investigated whether having a micro-controller on the velocity sensor itself is a viable option. One though with this is to use the 1 MHz oscillator already on the board as the clock for the micro controller. It is uncertain however if the I2C interface will be able to run on a micro-controller with such a low clock so this is going to be tested.

In testing on an Arduino Uno and changing the clock frequency with the avr/power.h function clock_prescale_set() the clock was able to be reduced to 4 MHz at the I2C bus speed of 200 kHz before it was no longer able to communicate. This means that an external oscillator will be required for the Arduino.

3.3V Calibration Probe Calibration

The sensor was again tested in the Flume against the FloMate with the measurements plotted below.

  • 3.3V Velocity Probe Calibration *0 - *4 are the different sampling rates with *4 being the fastest sampling rate

The dashed line represents the 1:1 response curve. The BoSL probe tracks fairly well with the data there are outliers however particularly with the high sample frequency at low velocities and the low sample frequencies at high velocities. One method to determine which measurement is correct is to look at the amplitude of the peak in the data, it is the high sample frequencies have a low peak significance at how velocities and so adding a test for this could help with determining which value to finally record.

Another method is to use the concordance and spacing between results. Most of the time when a number of sampling frequencies cluster together they cluster close to the FloMate's measured value, also when the values are more different from each other it tends to be further from the FloMate's value. The only potential issue with this is if the oversampling of the lower frequencies at higher velocities coincidentally causes them to cluster together at a velocity which is not the true value. This maybe able to be solved by changing the sampling frequencies slightly.

Sample Frequency Adjustment

Currently the sample frequencies roughly form a geometric sequence with r = 2 and the frequency plots have large regions of overlap between the sample frequencies. This is bad as it will cause results to cluster at velocities which aren't the actual velocity when in the under-sampling domain. A plot below shows this effect, the x axis is the Doppler frequency and the y-axis is the measured frequency for each of the sample frequencies. As seen when the frequency is about 1.9 - 2.6 kHz the four sample frequencies converge together making it appear that the Doppler shift is lower than it actually is and that the high sample frequency point is an outlier. This can be mitigated if the sampling frequencies share fewer common multiples.

  • Velocity Response Curve r=2

14th February 2020

To overcome the issues with the I2C bus cable being too long to communicate at the high data rates and the memory issues related to the FFT an ATmega328P will be implemented on the board to act as a controller for the velocity sensor. This provides more flexibility and also enables new features, such as the ability to turn components of the circuit on or off to reduce idle power consumption, or dynamically change the behavior of the board.

The new circuit schematic is shown below.

  • Heterodyne Circuit with ATmega328P

The final PCB measures 35.5 mm by 19 mm and is four layers.

  • BoSL Velocity Front

  • BoSL Velocity Back

28th February 2020

A laser cutting at the ANU Maker space is going to be used to prototype the PCB, the laser cutter is only able to print 2 layer boards, and has some limitation on minimum track width of about 0.5 mm so modification will need to be made to the board to accommodate this. These modifications shouldn't change performance of the heterodyne circuit too much, the main issue is that errors may have been made in making these changes, so if this circuit works it is not an exact guarantee that the commercially fabricated PCB's will be. It will, however, allow the testing of the integrated ATmega328P.

  • PCB Copper Front

  • PCB Copper Back

The image below shows a sample PCB which the ANU Maker space laser cutter is able to produce. As seen, on some of the laser etches the copper has not fully been removed and so care will need to be taken that the PCB produced does have all of the copper between tracks removed so that they are not electrically shorted together.

  • Laser PCB Test

The next step is to etch the PCB blank on a double sided copper clad fiberglass blank.


A 3D printed case was also designed for the velocity sensor. Care was taken to minimize the the sharp edges which could disrupt flow while also considering miniaturisation of the device The case seals with an o-ring joint along its perimeter, and the two halves are held together with screw inserts. The cable is to come out the back of the device with the sealing method yet to be decided. The two transducers at the front of the device stand out at 30 degrees to the vertical in order to give a better reading of the velocity profile of the flow rather than only at the bottom of the pipe. The exact way this changes the reading, such as the approximate reading range of the sensor needs to still be tested. The PCB will fit in the cavity behind the tranducers. Enough space is provided so that it should easily be able to be inserted during manufacturing. The bounding dimensions of the case are 85 mm long by 32 mm tall by 52 mm wide.

  • Velocity Case

  • Velocity Case Front

  • Velocity Case Back

6th March 2020

The laser cutter was used to test etch the front side of the heterodyne PCB. The laser cutter has many settings which need to be adjusted per material, in this case for copper clad etching. The actual etching process took a small amount of time, about 15 minutes for 4 four etching passes required to remove all the copper on a single side. The detail level turned out to be quite high with 0.2 mm traces being able to be resolved. The final PCB loosed very well made and cut. Tests will now need to be conduced to see if the traces have proper and correct electrical connectivity.

After connectivity tests with a multimeter, the PCB was found to have a few issues with tracks being connected which ideally shouldn't be. This isn't too much a road block though, as the connectivity is due to the laser cutter not cutting deep enough to remove all the copper rather than only the pitch being too fine. More passes can be made with the laser to cut deeper into the copper and remove all of it.

  • Laser Cutter Cutting PCB

  • Final Cut PCB using Laser Cutter. PCB dimensions are 35.5 mm by 19 mm


Overall this method of PCB manufacturing seems to have better performance over others tested as well as being relatively quick once the per material adjustments have been made to etch the copper clad PCB properly.

For another PCB prototyping method check out PCB Prototyping

9th March 2020

I have begun working on optimizing the FFT algorithm. The current implementation uses floats, which are 4 bytes long. It should be possible to implement an FFT algorithm with just int16_t (2 bytes), and so reduce the memory requirement by a half. This would enable a higher number of points per sample, thus increasing the resolution of the velocity sensor.

This algorithm was then tested to see how closely it compared to the floating point algorithm, it is expected that there would be some differences between the two due to successive rounding errors at each step in the int16_t algorithm. Below is a figure which plots the percentage difference between the two implementations at a given frequency for a low amplitude (int -4 to 4) random noise input.

  • Percent difference between amplitudes of float and int16_t fft algorithms vs frequency bin


As can be seen the difference is always below 40% but usually around 15%. This may seem significant, however it only affects the amplitude of the peak and not the central frequency, which gives the Doppler shift. Additionally, as the peak relative peak height is more important in determining the significance of the peak over the absolute height, a consistent proportional offset between the algorithms does not matter. What is more important is the spread of this offset, as this gives insight into the error of the relative peak height. From the data the standard deviation of offsets at each frequency bin for the two FFT algorithms is about 8%.

16th March 2020

Some new ultrasonic transducers arrived. [insert image] They are smaller than the type currently used, and so if they have similar or improved performance over the current ones then they are worth switching to as they will aid in making the velocity sensor smaller and less intrustive.

First the resonant frequency was tested. This was done by applying a sine wave to the traducer in series with a 1 kΩ resistor. The oscilloscope was used to measure the amplitude of the voltage signal across the transducer only. The resonant frequency was measured at the frequency in which the amplitude across the transducer was at a minimum. This test was done in air.

The frequency generator used wasn't able to reach high enough frequencies to find the resonant frequency. A more basic test of sticking the transducer and watching the vibrations seemed to indicated that the resonant frequency was on the order of 2 - 3 MHz.

A more careful scan of the frequency range revealed that the transducer does have a resonant frequency at about 0.98 MHz - 0.99 MHz, in air, this minimum in amplitude occurs only for a very narrow frequency range and so was not seen at first when the frequencies were scanned coarsely.

The next test was to see how the transducer signal was able to be received with another transducer. This test is ideally done in water, however a good container was not at hand and so the test was done with the transducers in contact. Water normally offers good coupling with is better than or comparable to direct contact of the transducers. Hence, the results obtained here should give a rough idea of the performance in water. A 1 MHz sine with peak to peak voltage 1 V was used to drive the emitter ultrasonic transducer. The receiver ultrasonic transducer was placed in contact, face-to-face, with the emitter, and the two outputs were directly connected to an oscilloscope. The receiver produced a signal of in excess of 550 mV, at a very specific frequency around 1.00 MHz, the voltage amplitude was significantly less with some frequency deviation from the maximum. This gives a coupling ratio of 0.55 which is very good, and definitely fits what is needed for the heterodyne circuit.

NOTE: the metal casing of these transducers are connected to ground, this is an important design consideration given the metal face of both the traducers will be in contact with the water. Hence a way of insulating them might need to be found.

These transducers lack shielding around the cable. This proved important in reducing the signal to noise ratio for the current transducer setup and so a test needs to be conducted to see if this lack of shielding will cause a significant issue, once the transducers are implemented in the circuit.

24th March 2020

Some minor design changes were made to the the velocity sensor casing. This was done to ensure a better fit with the component. The casing will now be printed to test. An image below is included of the new design.

  • Altered Velocity Casing

A case for the smaller traducers was also designed. An image is included below. These traducers allow significant space savings for the in the design, specifically giving it a smaller cross section which will help with minimizing its disruption to the flow profile.

  • Velocity Casing for Small Transducers

5th April 2020

The velocity PCBs have arrived an I have begun to attempt to boot load them. They seem to encounter intermittent issues when boot loading, as program data is able to be written to the ATmega328P but the process halts at different points throughout the verfication sequence. I suspect this is due to some electrical connection issue, so I am going to check the PCB and the wiring connections.

To ensure that the issue is not with the crystal oscillator it was removed and replaced with a 16 MHz crystal. The boot loader was able to be burn with the 16 MHz crystal. Note that as 16 MHz at 3.3 V is considered out of spec for the ATmega328P the voltage had to be increased to 4.2 V before the bootloading process could work. This indicates that the problem lies with the crystal. I do not have another 8 MHz crystal on hand, I am going to try boot loading on another velocity board to see if it was an issue specific to this board's oscillator, or with the oscillator in general.

In trying another board it is able to have the bootloader burnt at 8 MHz, but only when the voltage is above 3.9V, else it returns an error.

After boot loading, uploading code using TX, RX works fine at 3.3V 8MHz. I'm not going to diagnose this issue further at the moment, as there is a reasonable workaround and it only affects the boot loading. This will be investigated before the next revision is finalized.

Power consumption test: The fully on idling of the velocity PCB is 5.4 mA.

The code was modified to power on the oscillator and the op-amp under the new power saving features. Transducers where then connected to see whether the circuit was able to detect the signal. Unfortunately, it was not able to. With the Arduino giving the same reading back every time, regardless of whether the transducers were connected or not. The issue seems to be related to the I2C address of the ADC, the ones the code were written with were 0x36, but the ones on the velocity board have address 0x34. This difference is likely due to slightly different variants of the ADC being purchased and is something to be aware of when ordering from different suppliers.

The ADC started measuring values at this point, however it was unclear if they were related to the signal from the water velocity. Other buggy, and unstable behavior also occurred, and so this will need to be ironed out. In order to get it working properly.

9th April 2020

The issues with code instability were fixed. The issue related to the 16 bit integers being casted to 8 bit integers and overflowing in the I2C library. After changes were made the velocity code appears to be working as it was on the prototype circuit.

The casing was changed to increase the size of the cutouts for the transducers, and the internal geometries were optimized to minimize the supports needed when printing. The aim is to be able to print a casing without needs for supports hence, sharp internal overhangs were reduced or tapered.

An image of the modified case is included below. The transducer fit in well, after a few taps, and the screws tapped into the plastic securely, keeping the two halves together. Adding a o-ring or similar into the split between the two halves is hoped to make the casing watertight.

  • Velocity Casing

11th April 2020

A setup to test the velocity sensor in a water flow was put together. The setup is not perfect but should suffice for some limited testing.

The new pcb and code was then tested to see if they could be used to detect a water flow. Unfortunately, it was unable to. The FFT output had no signal or peak at a frequency corresponding to a water velocity. In investigation, the data coming in from the ADC had a range of plus minus one or two. This is much less than was found on the previous prototype. I'm investigating now the cause of this decrease.

Applying a DC signal to the input of the ADC, on the new PCB, output a correct digital signal, swapping the polarity of the signal was also reflected in the digital output. The overall DC sensitivity was 0.5 mV/step. The AC characteristics of the amplifier still need to be tested as it remains a possibility that the issue lies in the more rapid sampling.

Applying a 100 mV 300 Hz signal to the input of the ADC, it was able to successfully sample this, however issues were had when the signal amplitude was reduced down closer to 20 mV. The FFT, was not able to resolve this peak well compared to the background noise. I suspect that this is due to the different inference characteristic of the new PCB.

12th April 2020

In using the power pins to turn off the Oscillator and Op-amp it seems that noise interference is not an issue. The no signal-reading did not change significantly after these were turned off. What did have an effect on the noise present in the recorded data how the goodness of the connection between the traducers and the PCB. Currently this is done with header pins however this does not provide the best connection, and can come loose occasionally. The testing setup may also be a cause of the reduced effectiveness of the velocity meter. The water flow may not be stable enough or deep enough to give a proper reading. One other test to perform is to see if the op-amp traducer signal output is of the same amplitude as the prototype. If a weaker signal is being output, this may explain the difference in the amplitude of the readings.

13th April 2020

The issues with the velocity sensor not reading were solved. The ADC voltage reference was changed from the 2.048 V internal reference to the voltage divider 1.1 V in the REF pin. This effectively increased the sensitivity of the device by a factor of 2 and, in doing it became able to measure the Doppler shift in clear water. The data out from the ADC has still only about 1 bit of dynamic range, which is less than ideal, appears to be sufficient to produce pronounced peaks in the FFT.

In the circuit schematic, for future revisions, the voltage divider for the ADC's input voltage was changed from VPP to a digital Arduino pin, being able to turn this off during sleep should save about 1 mA.

14th April 2020

The smaller ultrasonic transducers were tested with the Velocity PCB, using a similar setup to yesterday. The results were positive, with it able to detect velocity peaks similar to that of the the other traducers. In looking at the raw output from the ADC, the amplitude was even a bit higher from the new traducers, with the the signal having a range of about 1 but it is unclear hoe significant this added range is on the performance of the device.

With some algorithm tweaks and bug fixes for the new int16 FFT, all 5 ranges are concordant on their velocity reading for >100 mm/s measured, and the first four are concordant for >50 mm/s this is a significant improvement over the float algorithm where the readings would often differ between the sampling speeds.

The case was test fitted with the small transducers and PCB. It fit well, and an image is included below. Some changes will need to be made including adding a flange for mounting, and making the cable hole smaller, as it is currently quite large.

  • Small Transducer Casing

17th April 2020

Some work was done to develop a protocol to communicate to the velocity sensor from a BoSL board as a proof of concept. The a UART serial interface was used at 9600 bps. The setup was shown to work over small length of cable and also longer lengths with 15 m working without an issue.

The casing for the velocity sensor was waterproofed. Some geometries which needed to be changed were the hole size for the cable gland, the clearance for the screws, and the size of the back of the velocity sensor. Waterproofing was achieved by first using a hot glue gun to close secure wires and components in place, and then GumGel waterproofing compound was poured into the back half of the case. The front half of the case only contains the transducers and so needs less waterproofing.

It remains to be tested how effective this waterproofing is.

22nd April 2020

The velocity sensor survived in 300 mm of water depth over 2 days. The sensor was working both in the water, and after it was taken out.

An issue has been found between the BoSL board and the velocity sensor, when the velocity sensor goes it sleep it causes the BoSL board to reset. The cause of this issue is unknown and under investigation. After some testing this issue was solved and found to be caused by the character buffer being too small for the amount of returned characters from the velocity sensor.

It was found that the current consumption of the velocity sensor is 10 mA when active, and 1.7 mA when sleeping. This high sleep current is mostly due to the ADC voltage divider being permanently active.

24th April 2020

The sensors have been sent out for field testing (Thanks Luke!), an image of the final sensor design is shown below:

  • Velocity Sensor

27th April 2020

We have begun testing one sensor at David's house. We uploaded this code to to program use the sensor via the BoSL board: File:BoSLVelcocity.ino

We connected the wires as follows: Green (3.3V), Half Green (GND), Orange (Digital Pin 7), Blue/Brown (Digital Pin 8,9). When building the sensors we accidentally reversed the blue and brown wires for one of them, so if it doesn't work initially just swap them around.

9th May 2020

Graph of peak amplitude vs velocity reading. Blue: sample speed 1 (slow), Orange: sample speed 2, Green: sample speed 3 (fast)

A velocity sensor was installed in Dandenong creek for testing (Thank Luke!). So far the results returned look like they have some promise, however there is still much work to be done before they are usable and reliable. Part of this unreliability is caused by the code. Upon reexamining i, it takes the average of three velocity readings by only records the last amplitude, hence if the first two velocity readings have low amplitude and are bad, but the last has a high amplitude and is good, it throws off the average velocity but the amplitude reading still indicates its a good data point.

The graph shows that readings with an amplitude of less than 1000 are mostly noise, and above this meaningful data is obtained. However it also shows that the vast majority of points have low amplitudes and few are 'successful' readings. It is going to be important to find a way to increase the number of data points which have a high amplitude, so that the sensor reports a reliable velocity more often.

After thinking about the possible causes of a low velocity reading. It is likely that the strength of the signal out of the heterodyne is the issue, as the ADC often only read a signal range of -1 to 1. Putting an amplifier here may work in boosting the signal, and so having more resolution in the digital output of the ADC.

11th May 2020

The an error was found in the velocity sensor, R5 and R6 were soldered to each others places. ie the R5 resistor was soldered in R6's place. Because of this the voltage reference is double what it should be, leading to half the resolution.

It was also found in the MAX11613 datasheet that the 1V vref limit was only due to the noise floor of the device. In our case a noisy signal can be tolerated so, it was decided to test how the device performed on a 0.3 V voltage reference.

When this was done the readings ranged from about -4 to 4 of digital code, the component of the signal from the traducer could not not be determined any better.

It when tested again it was found that the circuit performed significantly better at 5V, than at 3.3V. Running it at 5V cleared many issues had with the ADC not detecting any signal from the heterodyne output.

12th May 2020

Graphs of FFT averaging, top sample input, bottom FFT average output, Blue: 1 sample, Orange: 2 samples, Green 3 samples, Red: 4 samples

Some analysis was done to see if it was possible to get a better reading from the velocity sensor through improvement to the peak finding algorithm. The effect of averaging multiple FFT outputs was simulated in Python. A sin wave was signal was generated and then random noise was added such that SNR = 4. ie the amplitude of the noise was 4 times that of the sin wave. This was to simulate a worst case scenario, in reality the heterodyne returns a signal a bigger than 1/4 the noise floor. The effect of ADC quantisation was introduced by rounding the values to 2 to 3 bits of resolution. The fft of this signal was calculated. The output of multiple fft, from multiple inputs were summed, and then plotted (see below). It was found that this method is able to significantly increase the ability to resolve a single frequency amongst noise. The average increase the final SNR in the fft from about 1 - 2 with only 1 average compared to about 4 - 6 with 4 samples, a substantial improvement.

This averaging was then implemented on the velocity sensor to see if it had a real world effect. The results were difficult to discern exactly but it was found that the changes did have an improvement (vref = 0.3 V, vpp = 3.3 V). 3 averages were used for the testing of the velocity sensor application. Peak preeminences were seen to increase and the proportion of readings what were noise decreased. A proper test to see if this had any substantial effect will still need to be conducted. It was very difficult to tell if any of the readings produced by the velocity sensor had any accuracy. This was because the method used to generate the flow was moving the sensor by hand in a still body of water, something which produces an inherently unstable motion.

16th May 2020

Plot of velocity reading vs time (Blue, Orange, Green, Red, Purple): various sample frequencies of the BoSL velocity code, Brown: Hack probe

The results from the in field velocity sensor where graphed against a hack probe installed at the same site. (below) Unfortunately the results to not show that the sensor is measuring velocity. The readings from the velocity sensor do not follow those from the hack probe with it giving readings in seemingly unrelated locations. There are likely many reasons for this, one of them however is that the averaging used for the data was not ideal (as described above), the code this has been partially fixed in the current firmware installed on the velocity sensor so it waits to be seen if next weeks data shows this to have solved some of the error in the measurement.

25th May 2020

Some new Velocity PCB's arrived, These had the changes of adding a wake line to the sensor, so it could be woken from sleep mode. The voltage divider which powers R_ref of the ADC was also switched to be power by a digital pin, so it could be switched off when not in use, conversing current.

Modification did need to be made to the PCB, the resistors which form the voltage divider for R_ref had the incorrect values. The were replaced with a 1k and 10k resistor to form a 0.3 V voltage divider. This follows changes that were found to make the velocity sensor more sensitive to flow measurement.

26th May 2020

An oscilloscope was used to probe various test points on the circuit. It is interesting to note that the the old yellow Heterodyne circuit had a transducer emitter voltage of about 0.5 V. The new pcb's have it at 100 mV. This is unexpected as they use the same components in the same circuit. More, investigation to come.

I've decided to compare the wave forms at every point in the 1MHz pipeline between the old prototype circuit and the new pcbs, tabulated results are below:

Sample Point Prototype PCB Notes
Immediate output of 1 MHz oscillator 3.3 V Square wave 3.3 V Square wave Waveforms very comparable
Driver_Wave 300 mV 1 MHz sine wave 300 mV 1 MHz sine wave PCB slightly noisier
Transducer Out 600 mV Sine-distorted wave No Signal Example

From this we can see the issue is isolate to the op-amp.

To verify this I connected the oscilloscope up to the pcb sensor op-amp output. I then used a function generator to apply a 200 mV sine wave to the input of the op-amp amplifier. The a frequency sweep was conducted. Up to 200 kHz the op-amp worked fine, beyond this the amplitude of the output started to decrease rapidly, until it vanished at about 800 kHz.

This indicates that the op-amp cannot amplify the 1 MHz signal. This is somewhat unexpected as the exact same NE5532 opamp on the prototype PCB is able to. The major difference is the package the prototype uses a DIP package whereas the new PCB board uses a SOIC package.

Another observation found was when the amplitude of the injected signal was tuned to a fine voltage around 270 mA at a frequency of 1 MHz, then the opamp would indeed amplify the signal. The voltage needed was very specific and seems to change with time.

I suspect that the issue is being caused by the biasing of the input voltage is incorrect. I am going to test this by soldering components to the circuit to adjust the biasing.

This did not work as expected, it was very difficult to get the biasing correct with different resistors. Additionally, even connecting the oscillator directly to the output of the op amp did not work. It seems as though the input impedance of the op-amp is lower at high frequencies. That might not be exactly the case, but the outcomes are similar, and it is acting like it is.

27th May 2020

From configuring the NE5532 in a voltage follower configuration and using the function generator to apply a 3.3V 1 MHz sine wave, I have determined that it is not a very good op-amp for the application. When the transducer is connected to the output the the op-amp waveform collapses, indicating that it is well suited to driving low impedance loads at high frequency. Modification will need to be done on to the driver circuit to have it best be able to drive the output transducer.

1 June 2020

Testing was done on the electrical performance of the velocity PCB. Specifically receiver transducer and sampling section of the velocity PCB.

The first test done was to find the voltage resolution of the ADC. To do this a 50 mV amplitude 1 MHz sine wave injected into where the receiver transducer soldered to. An oscilloscope was used to measure the voltage in the inputs to the differential ADC. This was also about a 50 mV sine wave. The ADC gave an amplitude of about 600 divisions. Dividing these two gives a voltage resolution of the ADC of 80 μV per division.

The test was repeated however with the velocity board powered with 5V DC. A similar method got a voltage resolution of 120 μV per division. Of note however was that the SA612AD mixer had a much greater gain. An injected 50 mV sine wave produced a output 400 mV sine wave. This gain of 4 is significant and so we note that running the SA612AD on 3.3V while it still functions as a mixer reduces its performance substantially. Due to the performance issues of the velocity sensor running it off 5V may be an worthwhile compromise to gain the additional performance needed to have the device work suitably.


The second test done was on the FFT output of the transducer input. A similar setup was used with inputting a signal transducer input, however the FFT output of the ADC was monitored. 5 Volts were used to power the velocity PCB. The effective Doppler velocity shift of the input signal was 200 Hz. Only a direct scan was used to compute the FFT, ie no multiple scans and summing over frequency bins.

In conducting this test it was observed that the slower the sample frequency the broader the spectral distribution, that is that the slow sample frequencies with high resolution frequency bins tended to spread out the frequency peak over a wider number of bins. The slowest sample frequency being as wide as about a quarter of the frequency bins, however the fastest sample frequency was much sharper only having the peak span 4 or so bins.

This is important as the wider the peak is the less tall it is and so it makes detecting it more difficult amoung the noise.


A third test done was to see the lowest voltage on the transducer input whose frequency would be reliably measured by the FFT. To do this a signal generated was used to generate a 1 MHz sine wave. The FFT output was monitored as the amplitude of the function generator was tuned down. This was able to be done all the way to 1 mVpp with the signal still remaining detectable. To reduce the voltage further it is known that the function generator has a 50 Ω output impedance, so putting in a parallel resistor from a voltage divider. with a 12 Ω resistor the signal was able to be reduced to 200 μVpp this was still easily detectable on the FFT ouput with an SNR of > 10. Again these test were done at 5V and with one sampling.

The takeaway from today's test is that the input side of the velocity circuit is performing very well when run on 5V. It is best to focus performance improvements on the emitter transducer side of the circuit.

Knowing the minimum detectable voltage is important as it gives insight into the maximum power before the circuit is unable to measure the returned signal. The power of a transducer is proportional to its voltage squared. If the input transducer requires a minimum of V_i = 200 μV, and the output transducer is driven with a V_o = 600 mV signal, then the maximum power attenuation is, (600/0.2)^2 ~ 1E10. This gives us a budget to work in. Some power loss will be required due to the inefficiencies of the transducers and the geometry of how the Doppler signal reflects off water. What is variable with changes to the PCB design are changes to the voltages V_o and V_i. Decreasing the already small V_i would be challenging, and so work will be done to see if increasing V_o improves the sensitivity of the circuit.

2 June 2020

The new opamps for testing arrived. Work was done to evaluated these. The NE5532 opamp was replaced with a OPA2830 opamp, it was a drop-in replacement so it could be directly soldered to the pads of the old one. Immediately the output waveform was much nicer, with less ringing and it looking more sinusoidal.

Some modifications were made to the circuit to see if better amplification could be achieved. With this modification a 1 MHz 4 Vpp sine wave was produced to the transmitter output. This was a great improvement on the 600 mVpp triangle-ish wave produced before.

The following circuit was used:

  • 1 MHz 4Vpp sine wave generator

To main issues were had with the amplification, the first being that the impedance of the source was high, to overcome this the signal was first input into the unity gain op-amp follower circuit. This produces a unity gain low impedance version of the source, the signal was then passed into the input of the non-inverting amplifier. It is crucial to operation that the DC bias of the incoming signal was correct. To achieve this a 68 kΩ resistor was used to pull the signal closer to ground. This provided enough of a change to the DC voltage of the input signal to have it be the same as the bias of the inverting terminal of the opamp. This method is unideal as it required selection of resistors to match the DC bias of the circuit. Ideally a method of only amplifying the AC component of the signal is wanted. This is possible it just requires components not available at the time of construction. The feedback resistor of the non-inverting amplifier was also adjusted from 39 kΩ to 5.6 kΩ this was done as it reduced the gain such that the amplification of the signal was not clipped by the supply rails of the op-amp. As the NE5532 was a slow slew rate opamp, the larger gain combined with the slew rate caused the output of the opamp to not be clipped as the slew rate was slow enough that the opamp did not have time in a cycle to reach a supply rail.

Changing the opamp to the OPA2830 and making these associated changes is a step to improving the performance of the velocity sensor.

3 June 2020

The tuning of the resistors was not liked in yesterday's circuit so the one below was used. It did not require any resistor tuning however it did suffer from clipping the top volt off the signal. This is due the opamp internal offset voltage, a better AC coupling will be needed to overcome this.

  • 1 MHz 4Vpp sine wave generator

The circuit below with the better AC coupling mentioned was built up and tested. It performed much better, with the 4Vpp sine wave being output and the output not clipping at all.

  • 1 MHz 4Vpp sine wave generator

A quick check was done to see how this circuit performs in-water. No results were obtained, however it was noted that the 50Hz signal was very strongly present in the FFT, dominating all over frequencies. Perhaps this is due to interference with the mains wiring. More investigation will need to be done.

The circuit was tested with the older larger transducers. The 50 Hz interference was not noticeable then. Additionally, inspection of the ADC waveform showed significant returned signals from the transducer. The frequency of which seemed to correlate with the water velocity, but the testing method was crude and not designed to verify this. Furthermore, this occurred both on 5V and 3.3V, though the signal on 3.3V was significantly weaker.

I will verify again how the circuit performs with the the smaller transducers, as using them is much preferred due to their smaller size.

7 June 2020

This new circuit was tested with water flowing through a PVC pipe. The speed of the water in the pipe could not accurately be controlled but filling the pipe with water, and then raising it so the water flowed out faster proved to work well as a means to get a range of water velocities. Doing this is was possible to see how the readings of the velocity sensor changed with changes to water velocity.

The good news is that the velocity sensor seemed to work well to detect the Doppler shift. Both on 3.3 V and 5V and both with the smaller and larger traducers. It should be noted that the readings were substantially better on 5 V so going forward it will probably be best to move towards this.

Two of the circuits with the modification were made. They are going to be tested in the near future.

26 June 2020

Data came back from the test of a velocity sensor with the aforementioned modifications to the transducer amplification circuit (5V). The velocity sensor was installed next to a hach probe on Dandenong Creek, and the measured values were plot below. Only the fastest sample frequency is plot as the rest were found to have no better resolution and poorer velocity reading range. It is likely that these will not need to be measured in the future and only the fastest sample speed will need to be used by the sensor.

  • BoSL Velocity Sensor (orange), Hach probe (blue)

It is seen that the two sensors compare favorably most of the time. The BoSL sensor appears to have two main issues. The first being that is seems to not always read the true velocity but rather has measurements distributed between the true velocity and 0. The second is that when velocities are low the reading gives ~3.5 m/s.

Thoughts about how to diagnose these issues are that perhaps more samples should be taken. This will allow averaging of the velocity FFT spectrum over a longer period of time and hopefully mitigate the effects of random electrical or water current induced noise to provide a more stable reading. Another idea is the calibrate the sensor with whitening of the noise in the velocity FFT spectrum to nullify any electrical noise present at a particular frequency.

29 June 2020

Work was done on developing an interconnect which will allow the 5V velocity sensor to operator off a 3.3V BoSL. The device uses a boost converter to generate the 5V line from the BoSL's battery. Level converters are use to let the 5V velocity sensor logic talk to the 3.3V logic on the BoSL. Three lines up convert from 3.3V to 5V and one down converts from 5V to 3.3V. It should be possible to use the converter to link any 5V sensor to the BoSL.

The boost converter used converts the 4.2V battery into 5V. The TPS61222 converter was used, as was used on the velocity sensor V2. Using it in this design should provide data on whether it is viable to install directly on future revisions of the BoSL board.

Some solder jumpers are present in the design. JP1 selects whether the boost converter output should be controlled via the EN pin, or connected to VPP, and always be enabled. JP2, JP3 activate a voltage divider that may produce a 3.3V reference. It is preferred that the 3.3V line from the BoSL is connected however.

  • BoSL Interconnect Schematic

  • BoSL Interconnect

A small change was made to the casing. PCB rails were added to the sides to keep the PCB aligned when assembling. This should make assembly easier.

  • Velocity Case Back With PCB Rails

2 July 2020

It is being hypothesized that the variation in the velocity reading is being caused by different particles moving at different velocities in the water. To test this the FFT output is being logged onto an SD card. This will allow us to see if the peak has a board spectrum, or a defined peak that occurs at random points. The first being evidence for our hypothesis.

Getting the SD card to work is a bit of work. It requires tight memory management. Wrapping strings in F() stores them in flash and helps with reducing ram usage. Other optimizations such as reusing character buffers may also be needed.

We will once again only test on the highest velocity. Collect 3 samples for the FFT and power the sensor off 5V.

3 July 2020

The June 26 graph was transformed to plot the measurement recorded by the BoSL prove against the measurement recorded by the Hach probe. The data was taken from the period 2020/06/20 to 2020/06/25. The graph shows that the BoSL sensor does not often read higher than the Hach probe, however has its measurements distributed fairly uniformly below it. At low Hach velocities there is also a set of points from the BoSL probe with a very high velocity ~3.5 m/s the cause of this is not yet identified.

  • Correlation between BoSL and Hack probe, 1:1 line in red

The power spectrum density (PSD) of the velocity sensor FFT was measured. This was done to examine if there was significant interference of the velocity spectrum from external sources. The PSD was estimated both in air and in still water. The raw FFT from the 5V sensor was plot to serial and then logged. The RMS of each bin was taken to get an estimate for the power spectral density of each bin. About 1500 samples were taken for each bin. The resultant graphs are plot below.

  • PSD Air

  • PSD Water

The results indicate that the PSD is mostly flat in both air and water above bin 50. Water however has much slower decrease from the peak. Significant noise is present at the lower frequencies this should be taken into account when developing an algorithm to find the peak of the velocity. A way of mitigating this might be to whiten the noise so that the PSD becomes more flat, and thus the height of the bin better correlates with signal rather than random noise.

This is one possible explanation for the results seen where the velocity is consistently lower than the hach probe. If the water PSD has significantly more noise at lower frequencies then they are more likely to randomly be higher than the actual signal. Thus leading to the recording of them as the water velocity instead of the true peak.


6 July 2020

A test will be conducted gather the raw FFT data from the velocity sensor before the algorithm is used to find the peak.

Find details here

8 July 2020

The some sensors were finished production today, ready to be deployed. Of the 4 sensors made 3 turned out to work and 1 did not.

A database was set up here to track the growing number of sensors.

The test in the previous post was begun (Thanks Luke!) updates can be seen on the test page.

12th July 2020

Video update for the BoSL Velocity Sensor. Take a look at the progress we have made for the velocity sensor. We are getting closer to a final prototype.

15th July 2020

The data from this test was begun to be analysed today.

Plots where drawn up of each FFT, spectrum was plot with the hach velocity for comparison.

Specifically, the x axis plots velocity and the y axis plots amplitude in arbitrary units. The FFT spectrum is plot on blue. The black vertical line is the hach measured velocity. The green vertical line is the bosl calculated velocity. The orange region is the bosl calculated standard deviation. The red region is the first 2 bin, and is ignored by the bosl velocity calculations due to excessive noise.

There is a lot of data which hasn't been properly analysed however, below I will provide specific examples of graphs which I think are noteworthy.

  • 4535.0.png

This plot is one of the sporadically high data points ~3.3 m/s. it is seen that the FFT is uncharacteristically clean around the spike. It suggests that this is a hardware issue, as a signal from a transducer would also have larger values for FFT bins below 3 m/s. Additioanlly, a very similar issue was had with another BoSL velocity sensor, which displayed a peak very similar to this even out of water. For reference the sensor is: 003. Perhaps one of the 1 MHz driver traces is interference with the receiver circuit.


  • 2869.0.png

This plot is one where the BoSL and Hach compare favorably, you can see the FFT peak is very defined in this sampling.


  • 1050.0.png

This plot is typical of the sensor, where the BoSL reads lower than the Hach. It is seen that his may be due to the impact of the large peak in the first few bins. This does not support the hypothesis that the variability is due to the different speeds of particles but rather hardware noise. Interestingly, you can see a small version of the peak that causes the ~3.5 m/s noise.


  • 3219.0.png

In this plot the FFT stays reasonably high and constant until the Hach velocity, then it sharply decreases. This does support the hypothesis of the different speed of particles causing the variability.

From these graphs we can say that there isn't one reason that the velocity sensor is isn't performing as repeatably as desired. It is likely multiple reasons from hardware issues and the dynamics of the water and particle flow.


It is noted that at slow velocity readings above 100 000 rarely occur outside of the initial peak, and then only occur at fast water velocities ~1500 mm/s. Noise at low frequencies was found in the when the PSD was computed recently. So it suggests that these initial readings are much more influenced by noise rather than signal.

16th July 2020

Now the task is to look how best to process the signal to get a final velocity reading. The first step was to whiten the FFT spectrum. This was done using the data PSD spectrum calculated on the 3rd of July 2020.

The following plots are plot with BoSL velocity in blue, Hach velocity in Orange, and the scale on the y-axis is mm/s.

The algorithm implemented on the Arduino gives the following plot.

  • Norm.png

When the FFT bins are divided by the PSD bins calulated in air, and then the algorithm is run:

  • PSDair70.png

When the FFT bins are divided by the PSD bins calulated in water, and then the algorithm is run:

  • PSDwtr70.png

It can be seen that dividing by the PSD eliminates the variability seen in the points being below the Hach measurement. Thus noise whitening is seen as a critical step to processing the FFT. Difference between the water and air PSD is slight however it is seen that the water PSD follows the hach curve more closely, whereas the difference when using the air PSD is more nonlinear with faster velocities. It is seen that using the PSD increases the number of points at the ~3.5 m/s lever, however I am quite confident now that this is due to a hardware issue and not with the measurement methodology.


We can simulate more samples during measurement by averaging the FFT spectrum of neighboring points. This isn't quite exactly the same as the neighboring samples are taking 2 minutes apart, however it does give a good approximation. Doing this for a 2 point average we generate the following graph. We have also restricted the peak finding algorithm to bin 64. This eliminates the issue of detecting the ~3.5 m/s points.

  • 2ave64.png

The larger number of points at zero is because the algorithm returns zero instead of nan when either one of the previous points had a velocity value of nan.

Doing this we see that the spread of the data decreases. This suggests that the sensor using more samples would be beneficial for improved performance of the sensor.

17th July 2020

Today we will investigate other algorithms for edge detection. We will implement the 'Falling Edge' Idea. That is determine the velocity based on the falling edge of the velocity profile. The first algorithm to try is the canny edge detector.

We didn't implement a canny filter exactly, however many of the ideas were used. Specifically, the first the noise was whitened using the air PSD, then a Gaussian filter was applied to the FFT bins. The numeric derivative was taken next. Finally, the bin which had a derivative greater than a certain threshold value, and the surrounding bins with had a derivative greater than 0.3 of the max bin were averaged (weighted proportional to their derivative) to get the velocity. Only bins below bin 64 were calculated to avoid the ~3.5m/s points.

Doing this retrieved the following results:

  • Cannyrawair.png

You can see that the BoSL now measures consistently above the Hach. There are also a few more isolated points of sporadic velocity. The velocity measurements tend to stay clustered in a line except at lower velocities where the issue of the velocity measurement being distributed from a 0 m/s to the top of the curve reappears a bit.

Otherwise the method seem to provide good performance while still being easy enough to implement. It is going to be tested if it can be implemented on the velocity sensor with ease.

Another test was done, here the amplitude of the derivative was used to color the points. High amplitudes were given a blue colour and low amplitudes a pale colour. It is fortunately seen that many of the undesirable points from above have a low amplitude. Hence, the amplitude may be an additionally method of screening points for quality.

  • Cannycolour.png

20th July 2020

The noise whitening was implemented into velocity sensor firmware: File:Velocity Firmware rev 0.1.1.zip. The number of scans was also changed to 8, this was done to decrease the SNR.


A video was made of all the FFTs from the data points from: In Water Vel Sensor FFT SD Logging Test. The FFT output has been whitened to the air PSD, and had a Gaussian blur of sigma = 1 applied. The black line indicates the hach measured velocity.

25th July 2020

The second week of data from In Water Vel Sensor FFT SD Logging Test was plot using the whitening noise version of the current algorithm (16th July Plot 2). It is seen that the sensor continues to perform well when the water velocity is above a ~300 mm/s, it never seems to measure below a mean of about ~200 mm/s thus when the hach is measuring lower the bosl velocity remains 'stuck' higher. It is however seen that the signal strength of these points is significantly less than when the water velocity is high giving a method to distinguish these points. From the correlation plot we see that the trend is linear however the gradient is slightly below 1 . This implies that the BoSL tends to measure a lower velocity than the Hach when the velocity is high. The lack of large rainfall events producing fast flows means that the data here is sparse, and hence difficult to tell with high certainty how the sensor performs at water velocities in excess of 500 mm/s

  • 2 Weeks of data velocity calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)

  • 2 weeks of data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)

27th July 2020

A thought was had that the high frequency spike seen in the FFT at about ~3.5 m/s is aliasing of a higher frequency and that perhaps a low-pass filter would filter it out. There is currently a low pass filter installed however it cuts out frequencies above 11 kHz whereas the nyquist frequency of sampling is about 5 kHz. This leaves a 5 kHz band where interference might be able to occur. A test will be conducted on this.

31st July 2020

The algorithm implemented into the firmware File:Velocity Firmware rev 0.1.1.zip was tested. The FFT was used to run the indented implementation of the algorithm on my computer. This was then compared to the result returned by the sensor. The implementation was not perfect, as the threshold parameter was not identical, however once this was change the two results were identical, indicating that the noise whitening and associated processing was able to be successfully accomplished on-board the Velocity sensor.

It was found after plotting FFTs that the PSD used for 3 scans did not work well for 8, in that the FFT was weighted much heavier towards the low velocities (example below). The PSD is going to be regenerated to amend this.

  • 8 scan FFT plot showing weighting towards low frequencies

The PSD was regenerated in air an it was seen to be different to the 3 scan PSD. The data currently available of the 8 scan FFT is only of one water velocity so the re-calibration of the other parameters in the algorithm is not possible at this time.

4th August 2020

The BoSL Velocity rev 0.1.3 and velocity converter arrived today, images below, and are to be tested.

  • BoSL Velocity Interconnect PCB

  • BoSL Velocity rev 0.1.3 PCB Front

  • BoSL Velocity rev 0.1.3 PCB Back

The interconnect works as expected generating a 5V line and allowing the BoSL and velocity sensor to communicate. The setup is simple, connect the wires as directed on the silkscreen with the enable pin set to high whenever the 5V line is needed. One caveat was that is was expected that the 5V line would drop to ground once the enable pin was low, instead it appears that the line goes 'low' to 3V3, this will cause excess current consumption as it was designed with the assumption that this would go to zero. Detailed current draw analysis is to follow.

Keeping the boost converter on constantly, find that when taking a measurement the converter and velocity sensor (rev 0.1.2) draw a total of 40 mA, when in sleep but the converter is left on, the current draw is 11 mA.

When turning the converter off when the velocity sensor is not in use, it idles at 5mA. Still higher than ideal. Part of this is due to the velocity sensor rev 0.1.2 not being the most power optimised and the other half is due to the interconnect design issues, such as the boost converter not turning off when the enable pin goes low.

The following changes were implemented in the rev 0.2 version of the BoSL Velocity Interconnect BoSL Velocity Interconnect 0.2.0.

  • Replaced costly screw terminals with 0.1" sockets
  • Removed 3V3 input, 3V3 to be generated from 5V line (reduces cables required)
  • Added MIC94090 load switch for bringing the 5V line to a high impedance state when EN low (reduces quiescent current consumption substantially)
  • Smaller profile 21 mm * 22 mm.

The current velocity interconnects are still usable with the aforementioned power consumption caveats to be aware of.

6th August 2020

Testing was done on the new rev 0.1.3 PCB's to see if they perform as expected.

Mainly power testing was done first. This revision has made the heterodyne powered from the Arduino digital pin, hence being able to be turned off when not in use. Incorporating this feature into the code the sleep current is now 100 μA on 5V, a quite small amount. It still needs to be tested if the powering of the heterodyne through the Arduino causes issues with the sensitivity of the sensor, this will need to be done once the sensor is in the field.

-- On the issue if checking weather aliasing is causing the ~3.5 m/s velocity spike, one sensor will have its low passed filter capacitor C4 changed to 100 nF. This gives the low pass filter a cut-off frequency of about the Nyquist frequency (~9 kHz) as opposed to the current cut off of ~40 kHz.

8th August 2020

A refinement of the 3d printed casing was modeled. Improves included removing the cable grommet in hopes to replace this with 3d printed version directly integrated to the case, relocation of the PCB in the case to have easier routing of cables, and the addition of groove and channel on the joint between the front and back halves of the case to help prevent the Gel-gum from leaking during manufacturing. Various geometries were also optimized to use less plastic and gel-gum, mostly by reducing unused space in the from the inside of the casing.

Externally the case looks similar to previous version


The data from last week was also analysed, as this data had flow events it was able to be used to re-calibrate the algorithms parameters for 8-scans. The calculated velocities are listed below.

  • Week 4 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)

  • Week 4 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)

The plots show that this algorithm can be used quite effectively to measure the flow velocity. Particularity also useful is that the sporadically high measurements appear as having a much lower signal strength, meaning that they can be effectively filtered out.

27th August 2020

The refinement of the 3d printed case from 8th August was tested. The initial indications were positive. Much less leakage of the Gell-Gum occurred as had with previous designs. The design looks water tight, however particularly with the lack of cable grommet, this needs to be tested further, and for a longer duration.

8th September 2020

New data was received from the velocity sensor. This was compared against the hach probe. The data was less robust than usual, as the velocities which are greater than 200 mm/s generally had given stronger readings, however this was not the case for this data set for some of the rainfall events.

The below plots are the week 4 and 5 data with the algorithm tweaked slightly to increase threshold and introduced a filtering on low amplitude data.

  • Week 4 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)

  • Week 5 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)

  • Week 5 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)

The changes can be seen to minimally impact the data from week 4 while addressing some issues with the week 5 data. The main issue seen to be caused by the algorithm changes are the of the minimum measurement velocity of about 150 mm/s. This change was not unintentional as it was found by looking at the FFT of the data that below this noise was seen in very great amounts and made it so that no real velocity data was present here.

This is shown below, with the old algorithm the peak at the low velocities would have been taken as the predominant peak for the velocity. This low velocity peak however is commonly seen in some FFT's regardless of the water velocity, so it was assumed that this should be not taken as part of the water velocity calculation. This was especially problematic as if the low bin peak dropped below the threshold before the the true peak then the true peak would have been excluded from the velocity calulation.

To combat this three changes were made, the first was to check that if the peak was detected in the low velocity bins if there was another and perhaps slightly smaller peak at a higher velocity. If so this high velocity peak was chosen instead. This worked to alleviate most of the problem. However the minimum bin to check was also changed from bin 2 to bin 4 (causing the minimum measurable velocity to jump to 150 mm/s). Finally, the threshold was adjusted and optimized in light of these changes.

  • FFT with noise in low velocity bins (black: hach) (red: new algorithm)


9th September 2020

7 velocity sensors were build up. 6 of them had no issues however one of them presented a constant spike at about 3.5 m/s. This is similar to the issue seen in the field, however it is present without any transducers connecting. The issue look like a hardware/construction issue, as the PCB looks identical to all the others which did not have this issue. I think it would be good to write some testing code to screen for this at the manufacturing.

The velocity database has been updated BoSL_Velocity_Database. 2 PCB remain however I have run out of cable to make these into sensors.

10th September 2020

Week 6 data was analysed today, the usual plots are shown below. The sensor is showing worse performance with the than in previous weeks. Notably many more points where spontaneous high readings are measured. This can be seen from about after August 31. It could be possible that something happened to the sensor at this time to change its characteristics or it could be that after this date new flow regimes were encountered that were not before.

  • Week 6 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)

  • Week 6 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)

11th September 2020

A production versions of the velocity sensor code with the above algorithm for detecting velocity was made.

It is a available here: File:Velocity Firmware rev 0.1.3.zip

16th September 2020

The code on the velocity sensor here In_Water_Vel_Sensor_FFT_SD_Logging_Test was updated to File:Velocity Firmware rev 0.1.3.zip so the velocity reading could be taken directly from the automatic logging.

The data from the past week of logging was processed and the standard graphs are shown below:

  • Week 7 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak)

  • Week 7 velocity data velocity vs time plot, y-axis velocity (mm/s), orange hach, blue bosl, colour of bosl indicates signal strength of point (blue strong- white weak)

  • Week 7 velocity data calibraiton, y axis hach (mm/s), x axis bosl (mm/s) red line 1:1, colour of indicates signal strength of point (blue strong- white weak) however the threshold for blue saturation is higher

As can be seen the plot shows a similar trend to the past weeks, the sensor performs well at high velocities, and less low at lower velocities generally tracking similarly to the hach probe. Notice that on the late end of the 2020-09-13 velocity event the BoSL probe declines in velocity measurement much faster than the hach. The hach always seems to have a similar smooth trailing off from rainfall events so perhaps this is an artifact of the some internal averaging method used.

The correlation plots show now much more data in the 500mm/s range. This data follows the linear trend-line indicating that the two sensors are measuring similar values. There is more data which occurs at low hach velocity but registers a high BoSL velocity. This appears to have a strong intensity (deep blue). The third plot plots the correlation again but with a higher threshold that is that points need a stronger intensity to be coloured deep blue. Here we see that these points at low hach velocity become much more lightly coloured that is that they can be distinguished from the points on the 1:1 line. This higher threshold does make the points at lower velocity very faint so this calls for a intensity threshold dependent measured velocity for effective filtering without throwing out good data unnecessarily.

17th September 2020

A seperate PSD was measured for each velocity sensor. This was done to assess the variance in the PSD's between the sensors.

To evaluate the PSD the velocity sensor was placed in a bucket of still water. The depth of the water was about 20 cm the sensor was placed facing downwards at a 5 cm depth. The sensor was then set to print its RAW fft output to serial. This was recorded for about 250 samples and then the RMS of each bin was taken to find the PSD.

It was found that the positioning of the sensor in the bucket did not play a large role in determining the shape of the PSD for a particular sensor. The sensors did vary quite considerably in their PSD, so it seem that getting a PSD for each individual sensor is necessary. I am thinking of uploading the PSD into the velocity sensor database so that it can be easily retried whenever needed.

4th October 2020

A new velocity FFT analysis algorithm was tested. The method used in https://ieeexplore.ieee.org/document/9013073 was used, both uses boxcar averaging on the initial velocity spectrum and estimates the spectrum by find the FWHM. A plot of the data under this algorithm is shown below.

  • Implementation of the algorithm as in described in the paper

  • Current algorithm

  • Current algorithm with new signal strength algorithm

  • Current algorithm with new signal strength algorithm and boxcar averaging

It is seen that the algorithm described in the paper produces worse velocity readings with higher deviation from the 1:1 correlation line when compared to the current algorithm. 3rd image displays the current algorithm where the signal strength is taken as the the highest single bin. It is seen that this performs better is distinguishing the velocity points close to the 1:1 correlation line with a higher signal strength then those far away when compared to the current algorithm. The 4th images takes the algorithm and adds boxcar averaging as described in the paper. This performs similarly to the 3rd image however does have more high intensity points at 0 velocity and fewer near the 1:1 correlation. It is hence chosen that the new signal strength algorithm will be incorporated into new version of the velocity sensor firmware.


For the velocity paper some statistical analysis is needed to determine the accuracy of the sensor. I think it makes most sense to do this analysis not over the entire range of the sensor but calculate an error for the ranges of the sensors operation, perhaps 20 mm/s intervals.

The fractional error of the i-th velocity measurement is given by,
Math.png
Where v is the velocity as measured by the BoSL, and u the actual velocity. The actual velocity determined by measurement of the flow with a hach probe. This comes with its own uncertainty Δu. Thus, the uncertainty in η is,
Math1.png
We want to find the RMS fractional error over a number of points at similar velocities, hence taking the RMS of the fractional errors,
Math2.png
To find the uncertainty in the RMS fractional error we have that $R$ is a function of ηi, so the uncertainty is,
Math3.png
Note,
Math4.png
Thus substituting,
Math8.jpg
An expression for the uncertainty in the error.

For the Δu, the uncertainty in the hach velocity I have had some difficulty coming up with a reliable figure. The only source so far is Hach datasheet which quotes a 1% error for the spectrum analyser and 2% for the doppler sensor. This is almost certainly a underestimate of the actual uncertainty as the figure is quoted for a uniform flow field. In the mean time the figure of a 3% uncertainty will be used until a better one can be found. Running the above uncertainty calculation for the velocity data set of In_Water_Vel_Sensor_FFT_SD_Logging_Test from week 3 to week 8, with 200 mm/s bin sizes for the steps in hach velocity we get that

RMS (%) Mean erro (%) N u (mm/s)
420 268 21973 0 - 200
141 40 4591 200 - 400
51 -3 1758 400 - 600
34 -14 1596 600 - 800
34 -21 710 800 - 1000
37 -29 48 1000 - 1200
69 -62 12 1200 - 1400
62 -55 6 1400 - 1600
81 -79 7 1600 - 1800
73 -69 6 1800 - 2000

The mean error was calculated by finding averaging the difference between the sensor measurement and the HACH measurement, then dividing the average by the center of the velocity range to get a relative figure.


It should be noted that the sensor is not designed to perform below 200 mm/s hence the high error, in practice these results could be filtered out using the point signal strength or the depth of the water. Above 1200 mm/s the error increases substantially again, however making definite conclusions about the error is difficulty as there are fewer than 10 data points. Additionally the hach probe occasionally read sporadically high, hence this may also be contributing to the high uncertainty.

10th October 2020

This weeks data is plot below:

  • note hach velocity is now blue an the bosl organge, the darker the point the higher the amplitude

The sensor was cleaned during the battery replacement on the 29th of September It is see that cleaning the sensor did not seem to help with the noise issues present, as erroneous high velocities are still being measured when water depths are low. Also at high velocities the sensor appears to consistently measure below the hach value. Other hypothesis include that the sensor's casing is not entirely waterproof and so there is some internal damage to the sensor. This would be easiest to test by destructively investigating the sensor.

We can also now plot the data for the ID008 sensor.

  • Plot of velocity readings for ID008 sensor

This seemed to have an issue of cutting out occasionally for multiple hours. The data however looks much cleaner than the other sensor, with many fewer sporadically high readings and greater similarity with the hach at higher lower velocities.

Deterioration of the sensor is seeming to play a significant role on the error of the sensor. If the RMS error calculation is run only on the week 3 data then, the results below are obtained, significantly less error than the entire dataset

RMS (%) ΔR (%) N u (mm/s)
166 4756 0 - 200
23 235 200 - 400
29 44 400 - 600
16 394 600 - 800
17 194 800 - 1000
34 14 1000 - 1200
0 1200 - 1400
0 1400 - 1600
0 1600 - 1800
0 1800 - 2000

11th October 2020

Some issues were being had with the ID008 sensor in the field. It was not logging every minute and sometimes would go hours between logs. The SD card would not log either. I have reduced the BAUD rate on the BoSL- velocity line as hopefully this help in making this transmission more reliable.

Another idea has to see if the poor velocity readings of the sensor in field was to see if a re calibration could be conducted with the received data. To do this the PSD was taken from the FFT's September 16 to September 21. Here the velocity was low, the closest condition to still water in the data. This PSD was then used to rerun the velocity detection algorithm. The results are below:

  • Sept recal.png

The sensor now appears to track the peaks better and generally perform better during flow events. The number of sporadically high velocity readings has decreased slightly however they still are common in the data. Additionally the sensor seems to not detect low flow events anymore that is that it rarely detects a point at low velocity. This is to be expected in hindsight as when we calibrate at this velocity, it causes the signal at that velocity to be interpreted as a null signal. This test does suggest that some re-calibration of the sensor would be useful.

18th October 2020

It was investigated to see whether the dirtiness of the sensor had anything to do with the the poor performance of the velocity sensor. Starting week 9 of In_Water_Vel_Sensor_FFT_SD_Logging_Test the 5K6 velocity sensor was cleaned and then placed back in the water.

The below are the usual time series graphs for before and after cleaning.

  • before cleaning

  • after cleaning

The graphs do not tell have many significant differences. And it is difficult to tell if there has been an change in the quality of the sensor readings. This likely shows that the affect of the dirtiness of the sensor is a small factor in its overall performance.

For reference a photo of the sensor is provided below (its the 10K sensor but these were installed side by side for the same period of time):

  • Sensor clean.jpg


A test was going to be conducted on weather the sporadically high points were consecutive or sparely and uniformly distributed. As this would inform whether running measurement a couple of times until one of suitable amplitude was received is an effective technique to reduce this. However current all the data is sporadically high, so this test will not tell much. However it can be said that the same sporadically high value is not achieved every measurement and this figure jumps around a lot.

Another issue diagnosed today was the velocity meters in the field logging only intermittently. This seems to be an issue isolated to the hardware rev: 0.1.3. Whilst the exact issue was not able to be replicated, one which behaved similarly was. The suspected cause was that the hardware rev 0.1.3 turns on a number of analog components when it takes a measurement. This causes a large current draw and over a long cable with considerable resistance, a voltage drop. This caused the velocity sensor to reset. The issue was solved by modifying the firmware. Instead of turning on all components at once each component is turned on with a 20 ms delay. This spreads out the peak of the voltage drop and causes the velocity sensor to reset less. It was also found that a large (470 μF) electrolytic capacitor place at the 3V3 - 5V converter help fix this problem.


29th October 2020

The 5K6 sensor used in the velocity testing is now back in the lab. It is currently undergoing tests, before a final deconstruction and investigation. The first test will be a PSD calibration in air, when the sensor is still dry. The PSD which this resulted in did not show significant improvement when it was used to analyse the data. That is that many sporadic points still existed in it. The sensor was then placed in water and the PSD calibration was done again. When this PSD data was used, it was seen that the data did change, most of the sporadic points moved to higher velocitites

20th November 2020

Some changes were made the velocity algorithm and now it is working quite well. The falling edge detection algorithm was decided to be used as it produced the cleanest results. In addition this method was improved significantly by running the calculation on the natural log of the data. Some other smaller changes were made. The first was that the amplitude was calculated based on the height difference between the top and bottom of the peaks. The second was a method of reducing spontaneously high points. It was noticed that the edge detection algorithm would latch on to any edge of sufficient height even if it was more a 'bump' than a true edge of the transition between the high and low velocities. To attempt to filter these out, the regions before and after the 'bump' were compared in height if the height difference was not great enough then that edge would be ignored. The actual calculation of this was more technicality involved as it needed to be optimized to run on an ATmega328p. This algorithm was seen to work well with the velocity distinguished and erroneous points having a small amplitude so they can effectively be filtered out. The algorithm did not work as well for smaller velocities below about 200 - 250 mm/s however it proved challenging to find a good way to extract this data, as it was often drowned in the 1/f noise of the of the low velocity bins. It was found the velocity measured by the falling edge algorithm was about 1.5 times higher than the HACH velocity however, it is hypothesized that this is due to the difference between the mean flow and the maximum speed of dispersed particles in the flow.


  • Velocity Edge Time plot, dark points have high amplitudes

  • Velocity Edge Correlation plot, dark points have high amplitude

Below is the data again with a crude filter in place. The filter is set to only filter the data with a low amplitude when the velocity is above 300 mm/s. This is because at velocities below 300 mm/s some of the data with a low amplitude is not noise but the actual flow measurement.

  • Velocity Edge Filter, darker points have higher velocity

Velocity Algorithm Description

Below is a python psudo-code implementation of the velocity algorithm used for the results above 

           indx_low = 3 #start of fft bin range used
           indx_high = 64 #end of fft bin range used

           max_bin = 0 #stores velocity bin of interest

           #whiten noise
           fftpsd = [binn/weight for binn,weight in zip(fft, PSD)]

           #take natural log
           fftpsd = [math.log(x) for x in fftpsd]

           #gausian blur velocity bins
           fftpsd = gf(fftpsd, sigma = 2)
    
           #sobel derivative
           sob = [-1, 0 ,1]
           dfft = [] #array to store the derivative of the FFT array 

           #calculate derivative of array
           for i in range(len(fftpsd)):
               try:
                   val = sum([x*y for x,y in zip(sob,fftpsd[i-1:i+2])])
               except IndexError:
                   val = 0
               dfft.append(val)

           ctlow = -0.35 #threshold for edge gradient
           ctinc = -1    #threshold for edge height
           peak_can = False #peak candidate flag
           first_zero = True #peak candidate zero cross flag
           lst_zero = indx_high #bin index of last zero positive zero crossing
           peak_int = 0 #integral of peak ie edge height

           #the following code find the bin with the sharpest edge based on two criterion.
           #the first is that the edge should have a minimum gradient of ctlow
           #the second this that the edge height should be at least 1 (arb). Note that the height of the edge is calculated as the height difference between the first point where the edge the bottom of the edge defined (where the gradient is zero, to the top of the edge where the gradient again crosses zero for the third time.  
           for i,val in reversed(list(enumerate(dfft))[indx_low:indx_high]):
               if peak_can == False:
                   if val > 0:
                       lst_zero = i
                       peak_int = 0
                   else:
                       peak_int += val                       
                   if val > ctlow:
                       max_bin = i
                   elif val < dfft[i-1]:
                       max_bin = i
                       peak_can = True
                   else:
                       pass
               else:
                   peak_int += val
                   if first_zero:
                       if val > 0:
                           first_zero = False
                   else:
                       if val < 0:
                           if peak_int < ctinc:
                               break
                           else:
                               peak_can = False
                               first_zero = True
                               lst_zero = i
                               peak_int = 0

           max_val = dfft[max_bin]  #set threshold parameter as the edge bin of interest found above      
           mean_indx = 0 #variable to store index of velocity 

           #remove values greater than 0.2 of the peak of the edge, this allows us to isolate the edge
           dfft = [(0 if (x > 0.2*max_val) else x) for x in dfft]
           def getelt(i):
               try:
                  return dfft[i]
               except IndexError:
                   return 0
           #isolate peak  on the right and left sides by setting everything after the first zero to zero
           is_peak = True
           for i,x in enumerate(dfft):
               if i < max_bin:
                       pass
               else:
                       if (is_peak and dfft[i] < 0):
                               pass
                       else:
                               is_peak = False
                               dfft[i] = 0  
           is_peak = True
           for i,x in reversed(list(enumerate(dfft))):              
               if i > max_bin:
                       pass
               else:
                       if (is_peak and dfft[i] < 0):
                               pass
                       else:
                               is_peak = False
                               dfft[i] = 0

           #calculate the amplitude as the height difference of the edge
           amp = -sum(dfft[indx_low:indx_high])
           #find the center of the edge as an estimate to the velocity
           try:
               mean_indx = np.average(list(range(indx_low,indx_high)), weights = dfft[indx_low:indx_high])
           except ZeroDivisionError:
               mean_indx = 0

           #multiply found index by factor to convert to mm/s           
           mean_indx *= SdPoint.binconv * 0.6#0.6 falling edge mean velocity fudge factor 

           #return velocity and amplitude
           return mean_indx, amp

The flowing is a natural language description/explanation of the algorithm:

The raw data received from the sensor is a time series of samples from the ADC which is connected to the mixer circuit. The FFT of the data is then taken. This FFT is then RMS averaged with 7 others. The algorithm described operates on this RMS averaged FFT.

1. The FFT is first whitened against the relevant PSD for the sensor by dividing each bin with the corresponding PSD entry.
2. The log of the whitened FFT is taken.
3. A convolution is applied to the log of the FFT, this convolution filter applies a Gaussian blue of σ = 2, and a derivative operator. Thus the output is an array corresponding to the smoothed derivative of the data.
4. A peak detection algorithm is run. To find prominent edges in the FFT. To do this a threshold is set (see above for specifics) and the derivative array is scanned from highest bin to lowest bin until a bin with at least this derivative is found. The array is scanned from highest bin to lowest bin because all the FFT tend to have a very large spike at the low bins which throws off measurements.
5. Now the 'bump' detection is applied. To do this, starting from the smallest bin above the bin found in 4. which is negative, and descending, the derivative values are summed until the just before the bin where the derivative becomes negative again. This is approximately the next height change over the edge or 'bump', if the net height change is greater than a threshold it is determined that the bin from 4. does indeed correspond to and edge rather than a 'bump' which would return to the same height. If a 'bump' is detected 4. is repeated starting from the bin below the found 'bump'.
6. Now that a bin in the edge of interest is found it needs to be isolated from the data, to do this any bins after the first zero on the right and left of the edge are set to zero. This creates an array where the only data is the derivatives at the edge of interest.
7. The amplitude of the point is now calculated as the area under the derivative peak, or the height of the edge.
8. The velocity is calculated as the weighted mean bin of the derivative peak. This is then multiplied by a conversion factor of get to mm/s, a factor of 0.6 is also included in this which is thought to represent the difference between the mean flow and the fastest moving particles in the flow.

23rd November 2020

The RMS error can be recomputed using the new algorithm. The results are below, the errors are all less than the previous algorithm.

RMS (%) Mean error (%) N u (mm/s)
53 -11 26275 0 - 250
44 -22 4966 250 - 500
26 4 3059 500 - 750
18 0 1764 750 - 1000
42 -27 83 1000 - 1250
84 -82 5 1250 - 1500
93 -93 10 1500 - 1750

5th April 2021

An explanation was found for the factor of 0.6 in the velocity algorithm first discussed here. This was not in-fact a constant of proportionality between mean flows and fastest moving flows or anything like that but rather just a calibration error. The calibration in the velocity code the 69.7 hertz per bin corresponds to a sample rate of 17.8 kHz. The real sample rate of the sensor was measured to be closer to 11.1 kHz. Taking their ratio we get 0.62, the factor which was multiplied for in the code.

16 April 2021

Some testing was done to measure the current of the sensor. Sensor ID015 was used. At first the sensors couldn't be communicated with. It was found that the cable was too long for the serial connection so the cable was shortened a bit. The firmware revision used was 0.1.4.

Power State current (mA)
Measurement 27.0
Active 12.8
Sleep 0.109
Retrieved from "https://www.bosl.com.au/wiki/index.php?title=In_Water_Vel_Sensor&oldid=23954"