Break away from Motion?

Hi,
provided that I don't fully understand the details and therefore I might be misunderstanding the complexity of doing this - but wouldn't it make sense to bake motion detection inside raspimjpeg?
While I'm grateful to have motion detection and for all the effort put into the Motion software, it seems to me not to be very effective at its job: based on my tests it takes a lot of cpu to do motion detection at really low framerate (2fps) due to its implementation relying on the CPU. There's a -mmal branch that performs better, but is not mainstream and still doesn't seem to be able to push more than 10fps (which is plenty for most applications, mind you).
Looking at picamera and some python implementations, it seems that it's possible to make this work relatively easily by using the motion vectors provided by the h264 encoder. Here's a walkthrough of the process:
http://bits.citrusbyte.com/motion-detection-with-raspberry-pi/ (look at Method 2, mid page)
Here's an original article from the foundation talking about the motion vectors:
https://www.raspberrypi.org/vectors-from-coarse-motion-estimation/
I'm still pretty much a noob around C++ and MMAL and can't quite send a patch, but thought I'd bring this up anyway for consideration.
thanks,
Spike
PS: could you guys share any quick way to change code/compile/test on a pi? recompiling the userland each time on the pi is really time consuming. Any article you could point me to to make the testing cycle faster would be great for me to try things out.

and just to comment on the benefits/pov: this would save a lot of CPU making it possible to run raspimjpeg on an A+, not to mention simpler code/leaner dependencies (since motion does a ton of other things we don't use) not to mention that an implementation using the GPU can work at higher framerates and help with things like license plate detection for example.

Sounds pretty interesting to me because I don't really like motion for motion detection, I think it's very slow and sometimes doesn't react at all. And as you said, the CPU-ussage and more important, the higher framerate would allow a much better detection.

I'm going to do some changes to show the motion-vectors from the h264-encoder on the motion-page on the webinterface, this way we can easily decide whether or not we want to replace motion with this one. I let you know as soon as it works.

This sounds exciting both being more effective, lowering load and simplifying the overall scheme.
I think the key parameters that would be nice to replicate here would be : vector Value change (like Noise), Number of vectors (like threshold), Mask area of image to include, number of changed frames to trigger start, number of still frames to trigger stop (Gap)

on top of what has been discussed so far, I've been doing a lot of tests at night and it seems to me motion isn't really stable in those conditions, or at least something communicating with it isn't: I've had two cases where the itnerface got stuck "recording", two different pis, same config.

I made some tests yesterday with the motion vectors and could extract them from the h264-encoder, I'm going to try to analyze the data now. My idea is to provide a live-preview of the motion vectors on the web interface, this way it would be very easy to calibrate the parameters and so on. Additonaly, it would look pretty cool ;)

After a few more tests I have come to the conclusion that I need to process the data on the GPU itself. I think the best would be to convert the motion-vectors into yuv-pixeldata and send to the preview-imagedecoder. On the webinterface could be an option to change between image-preview and motion-preview. I tried to send the vectors to the browser directly and show with html canvas and other things, but the browser is to slow to convert the stream into a live-picture.

Unfortunately no success yet after hours of debugging. I try to convert the motion data into YUV-Frames and send them to the image converter for the live-preview, so it would be easy for the users to check what the motion detection reqognizes.
I added a new brach to the userland named MotionDetection, so if anybody has an idea what could be wrong, I would be very happy.

@silvanmelchior I don't see any branch here (or the circular buffer for that matter):
https://github.com/silvanmelchior/userland
where are you pushing this stuff to?

To roberts fork of the userland, my repo is out-of-date
On 17 Apr 2015 22:10, "spikedrba"<EMAIL_ADDRESS>wrote:

@silvanmelchior https://github.com/silvanmelchior I don't see any
branch here (or the circular buffer for that matter):
https://github.com/silvanmelchior/userland
where are you pushing this stuff to?
—
Reply to this email directly or view it on GitHub
https://github.com/silvanmelchior/RPi_Cam_Web_Interface/issues/56#issuecomment-94064680
.

FINALLY, a first success. I was able to convert the motion data in YUV420P-Format and then the GPU converts it into JPG for the live-preview. A little demonstration is here:
https://www.youtube.com/watch?v=5Yw94-f3GnY
If you want to try it, here is the binary: http://grustu.ch/various/raspimjpeg
You need to have the buffer enabled and the resolution needs to be set to standard at the moment.
Now there is one big question: How is it possible to detect motion without buffering/recording. Because this is important for the night preview and so on. One idea would be to split the preview-output of the cam and then simultaneously convert it to jpg for the preview and h264 for the motion detection part. What do you guys think?

woohoo :+1:
this might end up being a design choice more than a "how do I do this". To me the whole idea that the preview is not the same as the recording isn't helpful, in fact the opposite, it's misleading. As a user I expect first and foremost consistency. In that sense to me recording/buffering all the time and showing that rather than the brighter preview would be the correct behavior. At that point this problem no longer exists. Again being in mind my use case, I'm working to ward a security camera system, thinking I have clear recording and then when something happens I go watch the footage and it's too dark would potentially costs me a lot. Literally.
fwiw in picamera this is commonly done by using a different splitter port and recording while throwing the recording away and just keeping the motion vectors. This also has the benefit of being able to use a resized/sampled down stream. See this for an implementation I was prototyping:
https://gist.github.com/spikedrba/aecb82d8b51e991dbd01
This works very well for me. Before deciding to use raspimjpeg I was going to add another start_recording, throw away the video and just pass a sampled down version to a custom output to spit our a jpeg to the ramdisk like raspimjpeg does.

Great news
However, I'm not able to run it successfully at the moment plus few questions to make sure I understand this so I can make some sensible input.

Tried this out both from the your binary plus one I built. (They are slightly different sizes but give similar results). I have default 1920x1080 video settings. If buffering is on I get the live preview motion vector image for a few seconds then it dies with a Error: Could not get buffer for motion data. Sometimes it dies straight-away but always within 10 seconds. If I set buffering to 0 then it doesn't crash but starting video recording isn't updating cam.jpg like I would expect.

As I understand it the h264 call back is getting the motion data. This can be written to a file if motion_file is configured but is not used in the demo. The test flag sets it up to encode the motion data to cam.jpg so that the result is visible in normal preview. Does that sound right?

In terms of integrating this in, in the short term into the current environment, I can see a few possibilities. We could ignore the preview compensation issue for the moment as it is also out of alignment with the actual video recording as spikedrba says. So we then add commands to give user a choice between normal view and motion (mainly just for fun / set up), and we add in code to raspimjpeg to process the motion data into start stop detection, If that is built in then I assume we don't need the motion_file export? We would then need some extra commands to pass in appropriate detection threshold parameters like I mentioned earlier.

For initial integration we could just pipe out the detection start stop so they go through the same process as today. This is not maybe a clean as one would like but avoids making too many changes at once. On the web side the motion set up would need rework and we could just abandon that page and put the internal motion set up parameters and command calls on the main page under a new accordion.

I added a bit of tentative motion support material to the branch. I have not integrated in or touched RaspiMCam.c There is a new module RaspiMMotion.c intended to  contain the motion detection material. The actual algorithms processing the vectors are not there just initial parameter and command support to mimic the existing motion set up and frame processing, although I realise we may want to change and enhance that. The citrusbyte method mentioned earlier seemed to be similar in using x^2 + y^2 as a noise measure and then using a threshold count similar to motion.
I am not sure about the vector buffer structure. I know it is 4 bytes per block but some references seem to imply sad + x + y and others x + y then sad.
I also have an index page with these parameters / commands under a new accordion below camera settings and the old motion settings button gone.
I am still getting crashes (Could not get buffer for motion data) even with larger buffer. It seems to last about 3 or 4 minutes but I don't see any pattern.

Strange, I runned mine for 30 minutes without any problems, I'm going to do more tests. I'm not at home right now, but I should have some time this evening.

Yes thats true

The problem is that if we force the recording while motion detection is active, we disable the option to just take images in the night, triggered by motion detection. And the motion_file option could be useful for further projects I thought.

Sounds good

Well, my first goal is to switch between live-preview and motion-vectors on the webinterface, even if there is no capturing/buffering active. I try to add a splitter to the preview output and connect the h264-encoder either to the splitter or to the capturing outputof the camera, depending on the state. Or do you have a better idea?
Of course you can start with the processing of the vectors as you think it would make sense.

I think we can pursue both. My idea was to get a replacement for motion in place quickly even if we then wanted to add more sophistication later. That way we can put this out and remove motion altogether. Even just using the noise / threshold counting parameters I think this is likely to do a much better job than motion itself as it should be less sensitive to light changes.
Most of the infrastructure for the motion detect is in place although obviously untested. It is going to need calls to setup_motiondetect to init or when major parameters change, plus calls to set_motion_state (0 / 1) to enable disable. It will then need the analyse_vectors to be called when new vector buffers arrive. I was planning just to start with the x^2 + y^2 calculation for noise counting to start with.
It would be nice to get the sizing of the vector buffer dynamic but I think from your TODO's you are planning on that.
I have temporarily parked a replacement index.php (named as indexmb.php under the www-src) rather than create a separate branch at the moment. That has the new accordion for the motion parameters, although I left the existing motion set up in as well. I have been running with that and it seems OK.
There is an added cmd vm intended to be used for switching between a vector display and normal but it is just a cmd at the moment and does nothing.

Thanks a lot. I moved it to a new branch, so we can easily check and merge at the end.

I updated raspimjpeg:

choose live-preview or motion-vectors from web-interface
bigger preview-picture for vectors
support different resolutions than 1080p
I added a counter to check if the "Could not get buffer for motion data"-error happens just sometimes or all the time. If it happens just sometimes, it could be that the image-encoder is to slow to show every single frame, so with the counter it just droppes a few frames and gives an error if this happens to often. Could you check if you still get the error with the update?

My next step would be to make motion detection without capturing/buffering possible, I'm pretty busy next week, but I'm working on it as much as possible

Those changes sound good, but I am struggling to make them work at the moment.
In motion vector preview off mode then it is working normally. If I switch it on then preview image freezes. raspimjpeg is still running and I can set the motion vector preview back to off and the normal image resumes.
When the motion vector preview is on the the shm/mjpeg folder has a frozen cam.jpg plus a file called cam.jpg.part of length 0 and also frozen.
Logging also shows that when I switch to motion vector on it starts logging 'image' for every frame. Setting it back to off then this logging of 'image' continues even though the display is back to normal.

Hmmm, sounds strange, my rpi is working for 3 hours now in motion vector
preview-mode without a problem. Does your config-file contain all new
options?

Yes. To start with I just copied in the raspimjpeg.1 into /etc and used your binary. I get similar results with a binary I have built.
The 'image' logging is weird because there is no printLog or error in raspimjpeg with that but then I noticed it is coming from web logging not raspimjpeg but only when Motion preview is turned on. I am continuing to look into that.
I'll also try a full clone rather than patching my existing set up just in case there is something I missed. I had updated raspimjpeg, its config, index.php and config.php

I fixed the logging issue was some local debug I had that was getting triggered. Rest of the behaviour remains the same, however, including the cam.jpg.part.
Update: I did a full upgrade rpi-update reboot etc. Still didn't seem to be working but then I upped the buffer from the 2000 to 10000 and it sprung into life. So maybe there is something still dependent on the buffer size.

Great, I try to find out why small buffers don't work.
One question: What is the idea behind the vector_mode-parameter you introduced?

It was a placeholder for switching to vector preview so is now redundant. vector_preview is a better name.

I updated raspimjpeg. Attention, nearly nothing works right now, but I share it with you guys because I don't have time to continue my work today, so maybe you find some problems.
In the new version is a splitter behind the preview-port of the camera. The jpeg-encoder for the live-preview is connected to this splitter so generate the preview as normal. Additionally is the h264-encoder connected to the splitter if no buffering/capturing is happening. THe h264-buffer get reconnected to the video-output of the camera if needed. Some points for the new version:

I recognized a higher latency for the live-preview. This could be because with the splitter is an additional component between the camera and the image encoder. Do you think this delay is a problem?
If you want to try the motion-vectors, you can change your config files to vector_preview 1 and no buffer, then start raspimjpeg. The switching on the website doesn't work at the moment. There are also a lot of errors with the buffer, but as I said, I can't fix them right now.
Capturing/buffering doesn't work at the moment, I need to fix this.

So if somebody of you finds a bug, please report ;). Otherwise I try to continue tomorrow.

Yes. Definitely having some problems trying to use this one.
I get a  mmal: mmal_vc_port_info_set: failed to set port info (3:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x1302f90 (EINVAL) on start up
raspimjpeg is still running and updating cam.jpg
I was also getting please restart raspimjpeg from web site even though raspimjpeg was running. That seems to have sorted itself out. I guess the status file had got out of sync somehow.

Hi there, sorry for the absence over the last couple days, had to deal with some stuff.
Just tried out the branch and getting same errors as @roberttidey
RaspiMJPEG Version 5.2.0
Error: Could not get buffer for motion data
Error: Could not get buffer for motion data
Error: Could not get buffer for motion data
Error: Could not get buffer for motion data
Error: Could not get buffer for motion data
mmal: mmal_vc_port_info_set: failed to set port info (3:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x9c1a70 (EINVAL)
Could not set bitrate
mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x9c1750 (EINVAL)
mmal: mmal_connection_create: format not set on input port
mmal: mmal_connection_destroy_internal: connection vc.ril.camera:out:1/vc.ril.video_encode:in:0 could not be cleared
Error: Could not create connection camera -> video converter
MJPEG streaming, ready to receive commands

with motion detection disabled the cam works just fine.
fyi: I normally just got a black image except for once where I got a bunch of mixed pixels, not sure why, couldn't reproduce. Attaching it here just in case it helps:

I'm trying to figure out cmake and the rest of the toolchain to see if I can help any. Please feel free to share any tips/guidelines on this, especially docs around mmal that you think could help me to come up to speed.
thanks,
Spike

Here is a tiny clip taking using the previous non-split version which seems to be working pretty well as long as I set buffer to 10000
https://dl.dropboxusercontent.com/u/18838174/Vectors2.avi
For changing / compiling directly on Raspberry it's very easy you just sudo apt-get install cmake ,then clone this userland, cd into it and run ./buildme. First build takes a while but incremental builds after that are much quicker. On a Pi2 an incremental takes about 15 seconds. The only files of interest are those beginning RaspiM. If you want to build on an external platform then there is some guidance on the raspberry userland github on doing that on linux.
userlandmb/host_applications/linux/apps/raspicam
after a build a new version ends up in build/bin

New progress: it is now possible to change between motion-vectors and live-preview on the webinterface without a buffer. Depending on the preview the setup is
camera -> splitter
|----> resizer -> jpg-encoder -> webpage
|----> h264-encoder -> motion detection
or
camera -> splitter
|----> null-sink
|----> h264-encoder -> motion detection -> jpg-encoder -> webpage
At the moment, capturing or buffering is not possible. If you press capture, the h264-encoder is disconnected from the splitter and connects directly with the video-output of the camera. This works and a file is generated, however MP4Box can't convert this file, but I can't find the bug right now.
Please test it a bit and say if it works for you. And if anybody can find the problem with the recording, I would be very glad.

Runs providing buffer set to 0. I think you say this is expected.
Starts up with
mmal: mmal_vc_port_info_set: failed to set port info (3:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0xdf40a0 (EINVAL)
Will do more testing tomorrow.

same here (same error, same thing with the buffer, but that was expected per previous comment).
As an aside, I was looking at the motion video and kept seeing blips of light on the background and then I thought about my settings lock issue ( #44 ): if I set exposure mode to off the motion video is way more accurate: no more flashy background and random pixels deriving from the camera flickering as it keeps re-adjusting itself. I think it becomes especially important for this motion part to figure out settings locking (I put more info about fps 0, shutter speed and gains in the other thread on dark preview).

fwiw, I've also been keeping an eye on cpu usage and performances and so far on the MotionDetection branch. I get consistent number with raspimjpeg running at about 8% cpu as it used to do in normal video mode and ramp up to 14% when vector preview is turned on. I'm unclear at this point what's taking the cpu since the vector computation is on the GPU, but I've yet to look at the implementation and understand the code.

I think the converting of the vector-data to a yuv420p-frame for the
jpg-converter is responsible because this runs on the cpu and is executed
for every single frame. But I think this isn't a real problem because
you're normally not interested in the vector preview itself.

What if buffering were always on in the background with either user setting or some minimal level to make things like vectors work? Then if user has buffer set to 0 then we ignore the contents of the capture buffer for the purpose of building the capture file. I.e. we just do the normal h264 streaming into the file.
I am still trying to understand the splitter piece, but I'm not sure I understand the connection back to the camera during capture. Haven't we got to keep the motion vectors going during capture as well so we can detect stops?

wfm. fwiw to me that's also worthwhile to resolve the discrepancy between preview and actual recording (re convo on brightness of preview and fps).

The problem with this approach is that it's not possible to detect motion
at night and then take a snapshot for example or something else. This is
possible with motion and I don't want to publish an "upgraded, better"
version of raspimjpeg which doesn't support things that were possible
before.
If this inconsistency between preview and motion is a problem for you, you
could still enable the buffer, this way the h264-encoder is connected to
the video-port again as it was usual until now.

provided that my case should not in any way dictate what is useful to the larger community - you'll never hear anything from me like that - I don't understand why it's not possible to detect motion at night if the buffer is on. If the issue is high fps that would be honored if recording, therefore forcing a darker preview that is currently split to feed motion, then I'd argue you have a setup problem. Especially since there's support for day/night periods in the scheduler, when night hits, fps setting should lowered anyway and you should have an IR light. The sensor is rated as 0.67 lux which is almost full moon (1 lux). In anything less than that my direct experience is that you can't basically make anything useful out from the image. In my current night period I switch to 10-15 frames, raise brightness to 70 and contrast to 60 and I can get a pretty decent preview that's consistent with the recorded video.
again, I totally understand your reasoning, just trying to share a different point of view, but maybe I'm simply not understanding you.

To be honest, I don't understand why you don't like the version where
motion detection is possible with and without buffering. This way you can
either use raspimjpeg as usual or use buffering if you don't like the
inconcistency between preview and capture.

@silvanmelchior, thank you for discussing this with me. It's not that I don't like to have the choice between motion with or without buffering, to me it comes down to safe defaults and protecting the user. Again I might be wrong, but I expect most people will be thrown off by the inconsistency between preview and videos in low light, with some maybe only finding out when it's too late. That being the case I'd rather have it on by default, explain the situation, and then let an aware user make the choice, while the unaware one gets no surprises out of the box.
Personally I'm perfectly fine the way it is now that I understand what's going on and I don't mean to insist on this, that's not the intention.
best,

I had some spare time this evening, so I continued my work and updated raspimjpeg. It should be possible now to see the motion-vectors or live-preview while streaming, buffering or capturing. My next step is to implement motion detection itself and then merge the branch back into master.
I would be glad if you could test the new version exactly and report every single bug you've found.

This version seems pretty stable. I got one error Error: Could not send buffer with motion data but that was during a change in video mode. Otherwise I can change settings around and set vector preview on and off without problems.
I do notice a difference in vector display with buffer on or 0. With 0 although I see the vectors of moving objects there is a lot of background 'noise' even with nothing changing. If I turn on buffering even a small value then this noise disappears and I see just motion and if there is no motion then the screen is effectively black.

on a B+ with everything up to date:

I did a git pull + install, refreshed the browser and it was locked up (I had uconfig enabling motion preview)
I rebooted/refreshed everything, and things looked ok for about 5 minutes switching between video and motion and back, etc (no recording, 0 buffer). then I started noticing motion video skipping, there was visible lag. A few seconds later it completely locked, would not respond to a kill, had to kill -9 just like @roberttidey reported.
regardin background noise that @roberttidey mentioned, I get a lot of it, and as I already commented on it previously, for me it goes away as soon as I set exposure mode to off. This goes back to locking settings, seems even more important for motion
while watching motion vectors, I tried to hit record and saw in console that recording happened and then boxing kicked off and that's it, nothing else happened and motion vectors froze. The boxing completed and I could watch the video from the preview (as an aside, I thought I was gonna record the motion vector video, but instead it was the normal one). Any further change in the web frontend came up in the logs, but raspimjpeg never recovered and the image remained black. Previoiusly when things locked up I saw plenty errors "could not get buffer data", but this time there was nothing.
with the buffer set to 1000 I got no crashes over 30mins or so of testing
I tried to lower the buffer to 100 and that was still ok
I tried to set it to 0 and as soon as I hit OK the whole thing crashed. Interestingly enough first I got a bunch of "Error: Error in iframe list" then " Change: video_buffer = 0" got logger and immediately after that "Error: Could not get buffer for motion data".  Note that this time it died nicely at I hit C-c.
I tried to restart it from console with the buffer set to 0 from the previous session and it immediately locked up spitting "Could not get buffer for motion data". This time a C-c would not do and I had to login from another terminal and kill -9

I think I'll leave it at that for now.
two asides:

I noticed the motion preview shrunk considerably, were you planning on adding options to choose size in the web panel at some point?
the preview even of normal video seems slightly lagged, but maybe I'm misremembering and that's always been the case, might setup another cam up later to compare with an older version of the code.

So it would be good to find an exact scenario where it freezes every time with allowed buffer settings >= 1000 or without buffering. If the buffer is set so 100 or other small values, the intraperiod is set very very low too, so this could be a problem. But I think a buffer smaller than 1 second doesn't make sense, so this shouldn't be a problem.
@spikedrba I shrinked the motion preview because the CPU + GPU were to slow to create an yuv420p-image out of the motion vectors and convert these to jpeg, especially together with the splitter and so on, this was the reasons for the buffer-errors. And the lagging could be a problem with the splitter, my idea is to redesign raspimjpeg again a bit, so the splitter is just used if it is really necessary (live-preview, no buffering and motion detection turned on)

makes sense. I will re-run a batch of tests before EOD with buffer 0 and buffer 2000. Should you then have a check on buffer length to make sure it's >=1000? that seems to make sense if we know it will cause problems and I completely agree that it doesn't make sense to anyway. Maybe the whole parameter could simply becomes expressed in seconds rather than ms.

with 0 buffer:

enabled vectors, watched it for a while while juggling, no problems
enabled motion detection recording, nothing happened (I think this is normal, still not triggering recording, right?)
disabled motion, hit recording, worked just fine
disabled vectors
started recording from normal picture and then enabled motion vectors view. it got super laggy for a few moments but then I could see motion for another few seconds and then it segfaulted (in the past it just sat there not responding, this time it literally segfaulted). Nothing in the logs. It also seemed to have left with an incosistent setting state, after restarting the process it showed a black square like it was doing motion vectors and showing the dropdown as off and no motion movement. Had to toggle that and then it went back to normal image.
I tride to repro and behavior was slightly different: while recording and switching vector on I got no crash, but I did get another segmentation fault when I turned it back to off and this time I also saw "{2015/04/28 10:52:22} Error: Could not send buffer with motion data" in the log. Also after restarting raspimjpeg the web interface was responding and correctly showing motion vectors, BUT, the settings on the webinterface said off.
as an aside, one time out of 3 when I clicked recording while watching motions the thumbnail picked up was of the motion vectors even tho the recording was of the normal video
I tried to snap a picture and thumbnail was also of the motion vector
similar but different, I also got a segfault while trying to stop recording while watching motion vectors. In the log it never shows anything about capture being stopped.

with buffer set to 2K:

lag went up in normal video
after taking two snapshots motion vecors image was not responding, but no crashes, had to toggle it and then it came back
got one error "{2015/04/28 11:01:19} Error: Could not send buffer with motion data"
I started capturing from normal video, switched to vectors, no crashes however tons of "{2015/04/28 11:04:43} Error: Error in iframe list" errors. When I tried to switch vectors off it went back to normal image, but capturing also stopped tho there's no trace of it in the logs. There is h264 video created and omxplayer will play it. No sign of mp4boxing even tho the web interface says busy,

that's as much as I can test right now, please let me know if there's any more specific test cases you'd like me to run through.
All of this is on a B+ with 128M dedicated to GPU.
Spike

Great, thanks a lot! This will help me to develop a new version. I let you
guys know as soon as I'm ready. And of course, feel free to search the bugs
on your own if you have some time left ;).

New update, the splitter is now just added if it is really used (motion detection active and live-preview without buffering)
About the changing of the preview while recording: The problem is that it needs a lot of reconnection in raspimjpeg to switch between the previews and this is done by recreating the GPU-structure from scratch, so it is not possible to change between the previews while recording. I also think this isn't really necessary, so I suppose I'm going to disable the html-selection input while recording.
I couldn't fix the other problems yet, so some tests may result in the same behavior.

Built that one and working OK so far.
I don't seem to get the rippling effect on normal 0 buffer image and motion vector is clean as well with 0 buffer.

I did a small tweak to the motion vector scaling. I changed the index.php to scale image display to preview width. This doesn't affect normal view but upscales the motion vectors to same size. Then I removed the 2x2 scaling in raspimjpeg which lowers the load there and also makes the buffer suitable to pass to motion detect.
The live motion correlates with the image and looks suitably blocky as one would expect.

I didn't built anything, should have I? I'm just testing with whatever was in the branch. @roberttidey your upscaling changes look great, thank you!
@silvanmelchior totally makes sense not to be able to change preview while recording and certainly not an issue, thanks.
Here's what I'm seeing.
With buffer 1K:

The camera started with a 2000 buffer, as I changed it I got about 20 iframe errors right after the change in the same second (runnig at 25fps):

{2015/05/03 10:10:11} Change: video_buffer = 1000
{2015/05/03 10:10:12} Error: Error in iframe list
...
{2015/05/03 10:10:12} Error: Error in iframe list

I'm seeing a lot of noise, lots of pixels all over the place constantly flickering, but then the image itself is also noisier. I will double check the hw again, but it's the same setup/noir camera sitting on my desk indoor with the same light conditions so a bit puzzled.
I actually went to platy with the light and changed FOV back and forth and now most of that noise is gone. quite puzzled by it.
video recording while watching motion vectors worked fine with no errors, altho snapshot is still a black square of the motion vectors
after recording I went to watch the video and going back from the preview page yield a stuck black image. refreshing didn't change anything, same frozen black pic. I had to set motion off and it went back to the right normal preview and then on again for it to work again. No errors in the logs

{2015/05/03 11:17:49} Add /var/www/media/vi_0012_20150503_111739.mp4 to Box Queue at pos 0
{2015/05/03 11:17:49} Start boxing /var/www/media/vi_0012_20150503_111739.mp4 from Box Queue at pos 0
{2015/05/03 11:17:51} Finished boxing /var/www/media/vi_0012_20150503_111739.mp4 from Box Queue at pos 0
{2015/05/03 11:20:38} Change: vector_preview = 0
{2015/05/03 11:20:51} Change: vector_preview = 1

With buffer 0:

changed buffer to 0 while watching motion vectors, no problems
same issue about things being stuck when going back to motion vectors stream after watching a recorded video. Setting motion off and on solved again the issue. Do not, setting it to off by itself didn't do it, I had to refresh the page to see the normal image and then set it to on again

I know you said you didn't change anything else but modifying the behavior of the splitter, but I could not get any of the segfaults I used to. I did not try to change mode while recording since we said that will be disabled, but starting/stopping recording while watching motion vectors never segfaulted with or without buffer.
As it stands I'm only seeing minor issues with thumbnails and comign back from preview, but otherwise everything look good to me!
looking forward to motion detection working! very excited!
thanks a ton,
Spike

I've had my first internally triggered motion capture. It is using a fairly crude change count algorithm at the moment just to get the overall process checked out.
As the counts are on the full 25 fps frame rate one wants relatively high changed frame counts to start and even higher ones to stop. I started with noise 20, threshold 200, start frame 5, stop frame 50. I suspect we may want to use time based at least for the stop (like motion gap).
The problem I have is that a motion stop is not clean and throws an error Could not enable connection splitter -> video converter and I sometimes get  Error in header bytes so I haven't pushed the binary at the moment.

\o/
as an aside, could you explain me a little more how noise and thresholds are going to be used? Time based stop condition like gap makes sense to me, also the more we're consistent with motion settings the easier it's going to be for people to transition.
Also as I was trying to improve my night captures and looked more closely to motion I learned more about despeckle and found it quite powerful and useful (as opposed to an image mask). Here's the best explanation I found along with a C implementation that's currently used in Motion:
http://emit.demon.co.uk/motion/
http://emit.demon.co.uk/motion/despeckle.c
Not sure if that's going to be too heavy to be plugged into raspimjpeg, but from the standpoint of a security camera I think it's a really important feature, especially for outdoor installations.

Currently I was using those in a similar way to 'motion' but working on vector data rather than pixel data so should be better. As I said this is crude but is just to get things going.
So for the vector data it goes through counting the vectors that have a value above the 'noise' level. I.e small vector changes have no effect. Then if the counts in any one one frame is above the threshold level (movement) then frame count is increased. If it is below the threshold frame count gets reset to 0. This means that it needs n frames of 'movement' in a row to trigger a motion start.

Although the base code for a mask is in there I haven't added that in yet. Even if we make improvements in the base algorithm, a mask is really useful to cut out regions you really don't want to get triggers from. For example, in my case, I have one camera which sees a bit of a road in a corner and I do not want it triggering on passing cars.
I think it would be good to get a version that is at least as good as 'motion' going and then there are lots of room for improvements. For example rather than using a noise and count it might be better just to get an overall frame vector metric. SImilarly, one could be smarter with interframe filters rather than the count and reset method i put in for starters.
I have pushed this binary as I don't think it is any worse in no detect mode and it does seem to work after a fashion.

Any ideas on how to prevent the image/video preview to be the motion vectors? A resizing of the large image/videoframe on the gpu would be complex, on the cpu it would be slow.

I guess you mean the thumbnails in the download screen which are 'vectors' whereas the images / videos associated with them are 'normal'.
The way those are captured at the moment is by grabbing the cam.jpg at the start which is a very cost effective way of doing it and has other advantages like showing data which caused a motion trigger whereas converting a frame from video would show nothing if buffering is in use.
I did originally try converting when I first started to put thumbnails on the download screen but it can be painfully slow on the CPU.
The reason why we get the vector thumbnail is because we are sharing the cam.jpg between this thumbnail capture and the live preview. So in vector preview mode it is the vector -> jpeg process which is producing the cam.jpg instead of the resized jpg stream from MMAL.
So this does lead to a possible solution where in motion_preview mode the cam.jpg is produced like it is today from the vector data as that keeps the web interfacing side simple, but we direct the normal jpeg to a separate file name and that is the one grabbed for the thumbnail.
But my other question would be is this a big deal as I suspect the motion view is really more for set up, or do you see it as useful in its own right?

I'd +1 to split the preview/thumbnail thing to its own issue and make it low priority as I agree with @roberttidey that it's not a big deal - at least personally I don't anticipate having vectors on and recording other than in a setup phase - it just came up on testing as I was trying to be thorough with possible combinations people might ran into.

New update: I changed the iframe-handling a bit, I suppose the problem was that the intraperiod-change was not fast enough and so there were iframe-errors until it really changed.
Additionally I made some changes to the webinterface to prevent the change of the preview while recording and I adjusted the motion settings a bit.
I think we are moving to a version where it is possible to merge it back into the master and update the community about the change. So it is important now to test everything again and find possible bugs.

on it! and can't wait for it, I've been literally wasting the morning trying to figure out issues with motion, for whatever reason it's picking up things too late or completely missing the end of the action even tho settings look alright. Sounds like something I'm doing wrong but can't see how. Also since @roberttidey pushed out changes to start processes as www-data I seem to be having some serious problems, but holding off to investigate/report if we can complete this move!

had motion turned on before the update. went ahead pulled/installed, rebooted, went to the browser, loaded the interface and had a black image, no motion. I had to turn motion off and then back on for it to work. everything was fine after that.
with motion detection on, I was still able to turn motion preview on, which incidentally triggered a recording. Should this be allowed? It seems to be equivalent to having a record process going almost, which is indeed now not allowing to change mode.
I'm unclear about usage of the cirular buffer along with motion. I thought it was happended, but that doesn't seem to be the case. I now have 10000ms/10s buffer and recorded myself with a stopwatch counting to 6 secs. the video is 13s long. shouldn't it be ~16?
tested changing FOV with vectors on and had no issues

fwiw stats wise I saw an avg of 10% cpu utilization and 0.6% mem on a B+, overall the PI is super unloaded (0.43 load 5, most likely from mboxing, 0.14 on load 1). Pretty sure this has a chance to work on an A+ too \o/.
I've also tried to play with noise and threshold and this is where I think we need to improve, but nothing to do with breaking away from motion. Basically it's just too hard simply by watching the video to understand what good values for those should be. Somehow it would be great to have some live preview of the counters so I could wave my hand and say "ok that changes this many vectors by this much so if I want to avoid a moth, but pick up a person A and B are two good settings". Does that make sense?
I'm gonna upgrade the camera outdoor I was going to have problems with and let it run for the rest of the day and see what happens, but manual fiddling is all coming back super positive.
fantastic work @silvanmelchior and @roberttidey , I'm incredibly thankful!
Spike

this stuff showed up on the outside camera with 1s buffer and very low noise/threshold (it's a wide angle and it wouldn't see my motion if I showed up at the edges. With thsoe low settings it picked up a lot of tall grass being blown by the wind, so I turned it down, but in the meantime I noticed this in the log:
{2015/05/04 11:56:08} Finished boxing /var/www/media/vi_0317_20150504_115538.mp4 from Box Queue at pos 2
{2015/05/04 11:56:47} send smd 0
[2015/05/04 11:56:47] Stop capture requested
[2015/05/04 11:56:47] Send ca 0
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Error: Error in iframe list
{2015/05/04 11:56:47} Capturing stopped
{2015/05/04 11:56:47} Add /var/www/media/vi_0318_20150504_115604.mp4 to Box Queue at pos 3
{2015/05/04 11:56:47} Start boxing /var/www/media/vi_0318_20150504_115604.mp4 from Box Queue at pos 3
{2015/05/04 11:56:48} send smd 1
[2015/05/04 11:56:49] Start capture requested
[2015/05/04 11:56:49] Send ca 1
{2015/05/04 11:56:49} Capturing started
{2015/05/04 11:56:52} send smd 0
[2015/05/04 11:56:58] Stop capture requested
[2015/05/04 11:56:58] Send ca 0
{2015/05/04 11:56:58} Capturing stopped
{2015/05/04 11:56:59} Error: Error in header bytes

never seen that last error in header bytes before.

after quite a few hours the outdoor camera has been perfectly stable, however I don't think motion detection at this point is sufficiently useful to be deployed at least in my use case. I'm unfortunately still not able to look at the code and understand it, but per previous conversations, if we're looking about total amount of vectors changed in the image above a certain noise level, that's going to create far too many false positives in low settings or require thresholds to be so high that any wide angle lense will be basically useless. I'm just now really seeing how important the despeckle stuff is. At the very least it seems that an improvement, even if still coarser, would look for clusters of vectors so instead of total number of vectors per image do a total number of vectors above noise per macroblock or group of macroblocks.
It's understandable if you don't feel this is needed for you or don't consider this important enough to block merging in master, but wanted to surface it as I know several people doing outdoor stuff with noisy backgrounds for which this just won't work as far as I understand it.

Yes. I too am encouraged by low CPU usage which gives more scope for sophistication on motion algorithms. I think two areas are despeckle, although one has to be careful here as the vector resolution is already low. The other area is the frame counting. Currently 1 frame of stillness resets to 0 which with the full frame fps is I think too aggressive and means one has to lower noise and threshold to get detection. A bit of filtering or even something crude like stillness cuts frame count by 2 may be helpful.
There was some comment in the forum about continuing to use external motion when this is first rolled out until proven. I think this could be handled just by freezing that version as a possible clone branch during install.

On live preview of counters, what about if we added some annotate variables for those? These would obviously only show up on normal view not vector but might be quite a nice way to track what is going on.

My idea was to choose the preview-point in the pipeline, eg:

raw
after threshhold
after counter
after noise

To the problem with dark environment: what if every pixel has a separate
counter, this was it would be pretty easy to ignore random distortions.

re motion data: I like the counters annotations on live preview, but I'd have them on both, live preview and vectors. This combined with different preview points as @silvanmelchior was mentioning would definitely make for a robust tool to figure out a sweet spot for settings.
re motion detection: the frame of stillness stuff is certainly valuable, but it doesn't help with the case where a bunch of grass blown by the wind is picked up and a person isn't (the case I was testing). Without going down the road of despeckle the simplest thing I can think of is clustering/distribution of vectors above threshold across all macroblocks. This would be more likely to identify if a body entered the picture (a human, a car, they will all be continuos blobls of pixels) Vs a bunch of birds flying or tips of leaves shaking.
@silvanmelchior I'm not sure I get the comment about dark environments, sorry.
also as an aside looking at the logs for last night, I got a huge flood of "Error in iframe list" at 8:41pm, like 100/s for less than a minute right after a capture was initiated.

at a closer inspection, situation was worse, I didn't realize everything was completely frozen.
pi@raspberrypi ~ $ stat /dev/shm/mjpeg/cam.jpg
  File: `/dev/shm/mjpeg/cam.jpg'
  Size: 54302     	Blocks: 112        IO Block: 4096   regular file
Device: 10h/16d	Inode: 486968      Links: 1
Access: (0644/-rw-r--r--)  Uid: (   33/www-data)   Gid: (   33/www-data)
Access: 2015-05-04 20:43:11.153203152 -0700
Modify: 2015-05-04 20:43:11.083204213 -0700
Change: 2015-05-04 20:43:11.083204213 -0700

so you can see the file was last touched right after those iframe errors I mentioned in the previous comment. Which also mean the watchdog didn't work, case in point the raspimjpeg process has been up for 30+hrs since yesterday.  Things were still not good tho, controls on the web interface came back, but image was still stuck and nothing worked. Killing both also did not work, everything was still stuck and I was forced to reboot. Stuff worked after that. It looks to me like connection to the camera was completely lost or something that even restarting the process did't help, but I wonder why at that point raspimjpeg didn't complain.

The reason I said about annotations on live rather than vectors was purely pragmatic. Currently annotation is added by the MMAL routines and these don't apply to the vector data. Also the vector resolution is only 121 x 68 for an HD stream so one can't embed annotation in that feed. The only way this could work is to write the metrics to a separate file that would then have to be picked up and overlayed in the html domain. That seriously complicates the current jpeg/mjpeg streaming.
Maybe we go for different types of vector preview and annotation on live preview. For the former we probably want to more tightly integrate the vector analysis with vector preview generation. E.g. analyse_vectors is called first and returns an intermediate buffer depending on the type of vector view needed.

@roberttidey , thanks for elaborating on vector annotations, make sense and for the sake of simplicity and use cases let's just stick with that at least to begin with and don't worry about the rest.
As the crash, yeah, it was nasty. Is there anything I can try to do to debug it further was it to happen again? in the past I played a bit with gdb, wondering if something along those lines would be useful here at all.
thanks,
Spike

I've added %f (frame count, and %c (changes) as annotation variables. We can change their meaning or add more as algorithm changes.
Quite useful I think. The noise threshold / changes within a single frame looks too crude and I suspect we have to integrate over frames.
I did also change the frame counter to knock back 2 rather than reset to 0 to see the effect.

More unrecoverable crashes last night:
{2015/05/05 20:54:41} Capturing stopped
{2015/05/05 20:54:41} Error: Error in header bytes

{2015/05/05 20:54:41} Error: Error in header bytes

{2015/05/05 20:54:41} Add /var/www/media/vi_0810_20150505_205439.mp4 to Box Queue at pos 8
{2015/05/05 20:54:41} Start boxing /var/www/media/vi_0810_20150505_205439.mp4 from Box Queue at pos 8

that's the closest error I could find, however it went on to record 3 more videos of moving leaves and moths for another 2 minutes, then at 20:56 image got stuck and it's been "recording..." since then.

more fo the same tonight:
{2015/05/06 21:00:59} Capturing stopped
{2015/05/06 21:00:59} Error: Error in header bytes

{2015/05/06 21:00:59} Add /var/www/media/vi_1152_20150506_210054.mp4 to Box Queue at pos 30
...
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Capturing stopped
{2015/05/06 21:01:33} Error: Error in iframe list

there's several instances of those errors and they seem to be prevalent at night, but so far the pattern isn't super obvious, but definitely frequency higher after 8pm (by now it entered night mode if that makes a difference).
Spike

I'm running a test too and I also got a bunch of iframe-errors at night,
but I never got a crash as you described, so it's not really possible to
debug right now, I hope longer tests give me better results.
On 7 May 2015 06:10, "spikedrba"<EMAIL_ADDRESS>wrote:

more fo the same tonight:
{2015/05/06 21:00:59} Capturing stopped
{2015/05/06 21:00:59} Error: Error in header bytes
{2015/05/06 21:00:59} Add /var/www/media/vi_1152_20150506_210054.mp4 to Box Queue at pos 30
...
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Error: Error in iframe list
{2015/05/06 21:01:33} Capturing stopped
{2015/05/06 21:01:33} Error: Error in iframe list
there's several instances of those errors and they seem to be prevalent at
night, but so far the pattern isn't super obvious, but definitely frequency
higher after 8pm (by now it entered night mode if that makes a difference).
Spike
—
Reply to this email directly or view it on GitHub
https://github.com/silvanmelchior/RPi_Cam_Web_Interface/issues/56#issuecomment-99705918
.

@silvanmelchior well how do you debug? do you collect coredumps or something? can I do anything else to help?

are you guys testing with IR lenses/low light at all? I'm not quite sure how those vectors are calculated, but last night I tested with my night setup and results were real bad: unless I was within a few feet from the camera I couldn't get almost anything to show up on the preview. On the normal image however I could see just fine quite far (30 feet) away (IR light was on + brightness/contrast settings). Does it rely on color to recognize a macroblock has moved? I will test more tonight, maybe I did something wrong, just thought I'd check in given the lead time.

I runned a test until now to analyze the errors and their frequency. I got
many many iframe errors every night, interesting. I added some debug-code
now and started a new test, I hope the output helps to find the source of
the problem.
However, even after 72h it didn't crash, so I can't emulate the one you
described, maybe if you find a scenario which leads to a crash I can try to
debug it.
I don't have an IR-lense. However, during the night my camera often
detected motion because of the noise in the vectors, so we need to improve
the detection algorythm.
On Fri, May 8, 2015 at 2:18 PM, spikedrba<EMAIL_ADDRESS>wrote:

are you guys testing with IR lenses/low light at all? I'm not quite sure
how those vectors are calculated, but last night I tested with my night
setup and results were real bad: unless I was within a few feet from the
camera I couldn't get almost anything to show up on the preview. On the
normal image however I could see just fine quite far (30 feet) away (IR
light was on + brightness/contrast settings). Does it rely on color to
recognize a macroblock has moved? I will test more tonight, maybe I did
something wrong, just thought I'd check in given the lead time.
—
Reply to this email directly or view it on GitHub
https://github.com/silvanmelchior/RPi_Cam_Web_Interface/issues/56#issuecomment-100214693
.

@silvanmelchior do you have that debug code in a branch? maybe I could run it as well. I have not seen any crashes btw on the new camera I installed so maybe something was amiss with the previous one (it's on my desk for a lens change). I will put it back and test with it again.

It's nothing great, I just extended the log to print the iframe read- and
write-pointer if there is an iframe-error because I'm wondering why there
are so many iframe errors even though I increment the read pointer after
each error and it might be that the read-pointer somehow overtakes the
write-pointer.
On 8 May 2015 18:34, "spikedrba"<EMAIL_ADDRESS>wrote:

@silvanmelchior https://github.com/silvanmelchior do you have that
debug code in a branch? maybe I could run it as well. I have not seen any
crashes btw on the new camera I installed so maybe something was amiss with
the previous one (it's on my desk for a lens change). I will put it back
and test with it again.
—
Reply to this email directly or view it on GitHub
https://github.com/silvanmelchior/RPi_Cam_Web_Interface/issues/56#issuecomment-100288321
.

hey guys, I'm trying to learn how the code works so that I can hopefully start making some contributions, but having troubles following through parts of it. First question, how do you know what's in buffer->data that is passed to analyse_vectors? looking at the docs it says "Pointer to the start of the payload buffer", so I'd thought that was the actual h264 encoded YUV of a single frame? How do you know where/how to get the vectors? from the code and assuming it's a 4-bytes long, consisting of a signed 1-byte x vector, a signed 1-byte y vector, and an unsigned 2-byte SAD, then that's just vectors and the current code is never looking at the SAD but just the x and y. Is that correct?
Also in a previous comment @roberttidey mentioned "Also the vector resolution is only 121 x 68 for an HD stream" and I'm not sure I understand that. How big is this buffer? when the h264encoder component is created I don't see any format being set to c_video_* attributes, how come? does it simply get the ones  from the camera component to which it's connected to through the splitter component?
thanks and sorry for all the questions, hopefully it means I can be of more help in the future.

also a couple observations as I was experimenting and looking at the counter @roberttidey introduced related to my previous question about the size of the vectors: on a 4:3 full FOV video with a modified lense at about 10f I occupy ~2/7 the height of the image. I've taken a pic and put a grid on it of 16x16px, the size of a macroblock (MB) and roughly I take 50x30 MBs. That's substantial enough that I would expect motion to be detected if I moved around at that distance. Looking at the counters however when I move like crazy waving my arms (another good 10x30 MBs maybe) I barely even see a 50 changes counter showing up , meaning there's less than 50MBs with a vector movement > noise. With two arms waving like crazy and noise set to 20 I get no video recorded. If I reduce noise down to 2 I get up to about ~250.
If I can get some help figuring out what those vectors and buffers look like I'll try to take a stab at making some changes over the next few days, but as it is I don't think it's good enough to replace the external motion, which definitely flags me (I tested the diff with that from the master branch). I think it's working in the RBG space and colour so even me walking slowly into the picture that's mostly white with a black hoody gets picked up, which is good.

The video connection to camera is set to whatever is configured for video_width, video_height e.g 1920 x 1080. This in start_all routine around line 850. The vector buffer then is derived from that consisting of 1 byte vector for each 16x16 macroblock. So for this resolution the buffer is 121 x 68. This is the motion_width , motion_height set a bit lower down in same routine.
Each vector does consist of x,y and SAD and as you say we currently do nothing with SAD.
The analyse_vectors routine get a new vector buffer at full video fps to process. I think that is part of the problem with the current over-simplified counting technique. Because the frame rate is high then  there may be quite a few frames with relatively low vector counts and these still frames will lower sensitivity.
A better approach may be to just add all the vector data into a proper rolling frame average and then use that as the trigger. By changing the averaging factor one could reduce the impact of the vector data in any one frame. For example taking 0.9 of current count and adding 0.1 of current frame frame count. One could then threshold that and use hysteresis to give the difference between start and stop.
Taking SAD data into account also sounds interesting but I haven't looked at that all at the moment.
I used the motion_file config which stores vector frame to look at. It then holds 1 vector buffer in raw binary and just continuously overwrites. It might be worth changing this somehow to capture more of a sequence, say 50 frames for off-line analysis.

@roberttidey thanks for your reply, couple things still not clear to me, sorry. You said:

This is the motion_width , motion_height set a bit lower down in same routine.
so this part I couldn't figure out. what's the VCOS_ALIGN_UP supposed to do? and why is the width set to 32 and the height to 16?

also from the code it seems that we're getting the motion image from 0,0 to motion_**, so I'd expect any movement on the bottom right region of the image not to be captured, but that doesn't seem to be the case when I look at the preview. Looking at the docs it seems to suggest it's supposed to crop that region only, but that doesn't seem what happens in reality

Each vector does consist of x,y and SAD and as you say we currently do nothing with SAD.

And maybe that's fine actually. I found this article that explains why looking at SAD might be a bad idea to identify motion:
https://e2e.ti.com/support/embedded/multimedia_software_codecs/f/356/t/52390
The suggestion is to compute a SAM (Sum of Absolulte Motion [vectors]), but I'm equally skeptical that that would result in something sufficiently accurate as it suffer from the same noise/grass blown by the wind problems.

A better approach may be to just add all the vector data into a proper rolling frame average and then use that as the trigger. By changing the averaging factor one could reduce the impact of the vector data in any one frame. For example taking 0.9 of current count and adding 0.1 of current frame frame count. One could then threshold that and use hysteresis to give the difference between start and stop.

like I mentioned above, I'm honestly quite skeptical of anything that relies on sums across the frame. Outdoors IR light attracts a lot of bugs so it's not unusual for me to get a bunch of little dots on camera. I think the key is contigouos blocks. I've looked at a bunch of papers and it looks like a triangle search pattern would work well for this. I'm not quite sure how to implement that, but the idea I'm toying with is the following: scan the first row and when you find a MB that has movement > noise kick off a recursive function that looks at row+1 and MB-+1 and stops when that search can't find anything > noise in any of those contiguos blocks. It returns how many rows it found, which would be matched to our threshold (so for example if I only care about large objects that edge needs to be substantial). That's basically a very simple edge detection that should compute fast enough, but I haven't tried and there may very well be more optimized ways to do the same thing.
It would then store that so we keep a history for each frame of how many edges we detected above threshold, and average over 1/3 of fps (just making that up). If that's greater than another threshold we have motion detected.
That could be further improved assuming that an object actually moving will have its edges moving in one of 8 directions (up/down left/right upleft/upright downleft/downright), so we could in theory store the co-ords of all the MBs we identified at the previous frame along with the x/y vectors and based on those guess the direction and go look if there's any similar set of MBs with vectors > noise at the previous frame's MBs co-ord + Z.
I will see if I can toy with this in picamera a bit to test an algo, it's very easy to access the vector info with that, still struggling with C.
what do you think?

I'm not absolutely sure about the VCOS_ALIGN_UP but I think it is something like ... This is a macro that just takes the first value and rounds it up to fit on a boundary dictated by the second value. So motion_width 121 becomes 128. This I think is just setting internal buffers inside the MMAL and the 32 and 16 are just fitting in with the way it works internally. The mapping of real macro blocks is then being controlled by the actual motion_width and height as the crop parameters fed into MMAL. If this is correct then the 121x68 buffer we get back is a full non-cropped representation of the full screen macro-blocks. It is the same buffer used for the preview so if you should get a 1 to 1 movement activity there.
I agree that pure summing over the frame may be too crude but we could an approach similar to despeckle to weight clusters of vector data during the course of the sum. This would mean that an isolated bit of vector would be contributing less than larger objects groups. This type of algorithm is pretty simple. More sophisticated would be to keep 1 buffer and do weighting across frames as well without necessarily getting into full object detection.

I made a few more changes that should fix some problems and added some code to write debugging-information to the log-file, so if you do tests with the new version and get errors, it would be great if you post the logfile, I hope I find the iframe- and other problems this way.

great, thanks. I've deployed it to one of my outdoor cameras and will report back tomorrow morning.

Been running for about 11 hours in normal view, motion detect on with sensitive settings so it triggers quite a lot.
So far all I see in logs is Debug 3 at end of each capture which looks normal, I think. No other errors so far.

same here, so far so good, but I had the settings somewhat high so didn't get the usual high grass blown by the wind videos. tuned that down and will keep it running for the day.

ok, got a bunch. Here's the relevant log bit:
{2015/05/16 05:45:14} Motion detection started
{2015/05/16 05:45:14} DEBUG 3
{2015/05/16 06:38:54} Change: motion_startframes = 3
{2015/05/16 06:39:08} Change: motion_noise = 15
{2015/05/16 06:39:09} Change: motion_threshold = 70
{2015/05/16 06:40:56} Change: annotation = RPi Cam %Y.%M.%D_%h:%m:%s (%c %f)
[2015/05/16 06:41:04] Scheduled management tasks. Next at<PHONE_NUMBER>
{2015/05/16 07:36:42} DEBUG 2: 55868, 55897
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55869, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55870, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55871, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55872, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55873, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55874, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55875, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55876, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55877, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55878, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55879, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55880, 55898
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55881, 55899
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55882, 55899
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55883, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55884, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55885, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55886, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55887, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55888, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55889, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55890, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55891, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55892, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:42} DEBUG 2: 55893, 55900
{2015/05/16 07:36:42} Error: Error in iframe list
{2015/05/16 07:36:43} DEBUG 2: 55894, 55900
{2015/05/16 07:36:43} Error: Error in iframe list
{2015/05/16 07:36:43} DEBUG 2: 55895, 55901
{2015/05/16 07:36:43} Error: Error in iframe list
{2015/05/16 07:36:43} DEBUG 2: 55896, 55901
{2015/05/16 07:36:43} Error: Error in iframe list
{2015/05/16 07:36:43} DEBUG 2: 55897, 55902
{2015/05/16 07:36:43} Error: Error in iframe list
[2015/05/16 07:41:05] Scheduled management tasks. Next at<PHONE_NUMBER>

hope that helps,

@roberttidey sorry for going back to this again, but I'm not clear: are the motion vectors in the same buffer as the rest of the image data? I'm looking at the code again and all I can see is the h264encoder_buffer_callback(...) function, which both calls analyse_vectors and the fwrites to the video recording file (h264output_file), passing in buffer->data which should be a pointer to the actual payload, ie the video frame or whatever the encoder has returned.
I don't see anything that says "this is a vector paylod, give it to the analyse", while "this is a video frame, save it to disk if we're recording". Mind clueing me in? thanks!

in the callback right at the top there is a test
if((buffer->flags & MMAL_BUFFER_HEADER_FLAG_CODECSIDEINFO)) {
So if that is true then we are dealing with callbacks with motion vector data, otherwise we are dealing with normal video data.

As there are a few people suffering from iframe errors on the forums and the motion branch seems quite a bit more stable, I wonder whether it is worth merging the Cam module from the motion branch to update the master whilst further work is done on motion branch. It would also mean that we would get any debug if people have problems.
If so then I can have a go along the lines of incorporating most of this branch but using the external motion handling. The internal motion stuff would be there but not used or accessible via web interface.

Great idea, although there are still situations where a lot of iframe
errors happen with the motiondetection version.
On 18 May 2015 22:13, "roberttidey"<EMAIL_ADDRESS>wrote:

As there are a few people suffering from iframe errors on the forums and
the motion branch seems quite a bit more stable, I wonder whether it is
worth merging the Cam module from the motion branch to update the master
whilst further work is done on motion branch. It would also mean that we
would get any debug if people have problems.
If so then I can have a go along the lines of incorporating most of this
branch but using the external motion handling. The internal motion stuff
would be there but not used or accessible via web interface.
—
Reply to this email directly or view it on GitHub
https://github.com/silvanmelchior/RPi_Cam_Web_Interface/issues/56#issuecomment-103195862
.

speaking of which I got a whole new slew of errors this morning:
[2015/05/18 06:41:52] Scheduled management tasks. Next at<PHONE_NUMBER>
{2015/05/18 07:26:12} DEBUG 2: 30839, 30866
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30840, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30841, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30842, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30843, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30844, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30845, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30846, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30847, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30848, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30849, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30850, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:12} DEBUG 2: 30851, 30867
{2015/05/18 07:26:12} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30852, 30868
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30853, 30868
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30854, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30855, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30856, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30857, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30858, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30859, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30860, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30861, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30862, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30863, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30864, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30865, 30869
{2015/05/18 07:26:13} Error: Error in iframe list
{2015/05/18 07:26:13} DEBUG 2: 30866, 30870
{2015/05/18 07:26:13} Error: Error in iframe list
[2015/05/18 07:41:53] Scheduled management tasks. Next at<PHONE_NUMBER>

@silvanmelchior is it helpful that I keep posting this? I don't see any difference with the previous paste and incidentally they both happened at a very similar time (7:30am).
Do you have any theory what's causing this?
Spike

To be honest, I have no idea at the moment why there are still so many
iframe-errors, so any help here would be appreciated a lot.

I have made a dynamically switchable version between internal and external motion detection to see if that helps any users with iframes and give debug when this happens.
It can be switched between the two with a control under camera settings which then exposes either old or new motion settings. On the forum I said to keep using external for the moment.
As this is in the main branch for the full install I have posted these raspimjpeg changes to the master branch in userland. As the code is the same except for the detection mode switching I think that can be used for developing the internal motion detection further.
I'll see if I can look at iframe handling to get any ideas. As this came originally from a raspivid mod I wonder whether the author of that may be available for comment on whether he had seen this behaviour

I have a new idea: It might be that the function "h264encoder_buffer_callback" is called before an old call of this function has ended yet. If so, this interrupt could happen right before "mmal_buffer_header_mem_lock(buffer);", data would be written into the buffer from the new call, cb_wptr would be increased and the the old call of the function writes at this position and overwrites iframes without adusting iframe_buff_rpos again. I added some debug-code to see if this happens and let you know the results.

That sounds like a good theory. Somebody on the forums suffering from this said their Pi2 was more stable which would make sense if this was the case.

Silvanmelchior's idea about buffer overwrite seems plausible. In addition to the iframe error, I sometimes see garbled pieces of recorded video, as if some segments were missing during record.  I posted an iframe error log when running with code updated yesterday, at https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=63276&start=2200#p762735

The motion_file config previously allowed for capturing the most recent vector buffer to this file.
To make it more useful for analysis I have changed this so it appends the data, so one can see successive buffers. To stop the size getting out of hand it appends up to 400 frames and then stops. The file and count are cleared by starting motion_detection, so basically the file gets the 400 frames after motion_detection is initiated.

in the future, could we make this user configurable? I would very much like to save and preserve motion data at least in some circumstances, for example to create a database to train a machine learning algo to recognize things like rainy days or cloudy days.
thanks,
Spike

Several thoughts here. We could make the length configurable from web side. We could treat the filename like the others so they could be timestamp labelled. But how would we want to stop and start. We can't really leave it going all the time. It's about 0.5MB/s. Currently I made that just started when motion detect was turned on. We could give it a separate on/off control.

I think just an additional toggle would be enough (maybe along with a warning on space). So you simple have a yes/no to the question "do you want to save motion data along with video recording"? and if the user says yes whenever recording goes on (either through motion or manually) a file with motion vectors is also saved (and timestamped etc just like the video files - basically it's the same function that with an additional if and file write).

I've reworked the vector save now. I moved all the main code for this into the MMotion source file as that seems to make most sense.
The vector data is saved to the video capture file with an extra .dat on the end. The motion_file config which previously held the name now becomes an on/off for saving vector data this way and I added a new command and control on the index page for this.
The vectors saved to the file consist of the 100 preceding frames and then all the corresponding frames of the video for the duration until it is stopped. It won't line up perfectly with the video because of the video buffering if that is turned on.
The web pages know to delete the .dat file if present and they will also be included in any zip download.

@roberttidey fantastic, thanks so much for this!

just to be sure,what should one be testing at this point? I thought all this motion stuff was still in the MotionDetection branch, but I just tried to give this a go and actually found it in the master branch.

@roberttidey I compiled master to give the motion vector data a spin, but found that the process crashes immediately. The camera LED is basically flashing and in the logs I see this:
pi@raspberrypi ~ $ ./userland/build/bin/raspimjpeg
start monitoring for pid: 4036
{2015/05/24 17:12:54} start monitoring for pid: 4036
RaspiMJPEG Version 5.2.1
Set up internal detect width=82 height=61
start monitoring for pid: 4048
{2015/05/24 17:12:56} start monitoring for pid: 4048
RaspiMJPEG Version 5.2.1
Set up internal detect width=82 height=61
start monitoring for pid: 4060
{2015/05/24 17:12:57} start monitoring for pid: 4060
RaspiMJPEG Version 5.2.1
Set up internal detect width=82 height=61
start monitoring for pid: 4072
{2015/05/24 17:12:58} start monitoring for pid: 4072
RaspiMJPEG Version 5.2.1
Set up internal detect width=82 height=61

unfortunately I don't get anything else and I wasn't sure if there was any way of bypassing the daemon watchdog and do some more debugging. This is on a pi2, incremental build after having pull'ed from the master branch.

and just in case, I cloned a fresh repo and recompiled, wasn't sure if something from the previous MotionDetection build might be interfering, but I got the same exact results.
The one thing I'll add, if I put my hand in front of it after a while I'll see it showing up, so wherever it's crashing it must be after it sets up the camera and get most things going, becuase it's clearly suceeding at writing a frame to disk.

I am now working on the master branch since I made the change to allow both external and internal motion detect in the build. This applies to both userland and the main install repository. As this is being published on forums where most would fetch from the master branch of RPi_Cam then that made most sense.
I think we could probably now abandon the motion detect branches in both repositories. We could to then set up 'dev' branches to do any experimental work that we don't want to affect master.
Does the binary build from the main install site work? Also try setting to external motion and motion_detection false to see which factor is causing it to crash.

ok, so, working off of master of both repos. If I just use the RPi_Cam everything is fine. If I build my own binary from userland things go south. And it's indeed the motion code somehow becuase if I have motion external things re fine. As soon as switch to internal and either set preview on or start motion detection stuff starts crashing as per log I pasted before. All settings are default so no .dat file or bizarre noise/threshold values.

After some testing I've come to the conclusion that there are no interleaving calls of the callback-function, not even if iframe errors occur. So my suspicion seems to be wrong. This is a bit frustrating, does anybody have an idea what could be wrong? According to the log, sometimes all iframes in the buffer are incorrect at once, sometimes just a few of them and sometimes, the read pointer overtakes the write pointer, pretty strange...

How does the callback work in general? Is it a regular stream of callbacks that happen or does it rely on the callback pushing a new buffer into the pool before the next one can happen? If it is the latter then the calls wouldn't overlap but we might be losing sequential buffers back in the MMAL layer still. That would probably have the same effect as interleaving.
If that was the case could one push several buffers and then protect the callback from any interleaving that could occur in that case?

I have the feeling iframe errors happen only when I have a non-zero value set for the pre-event buffer. Not sure if that is 100% true, or if it suggests any causal mechanism.  Sorry I know little of the MMAL layer.

I also notice that there are sometimes missing or garbled video frames even when the log does not report an iframe error. I guess some buffers are just going missing (ok we knew that...)  What happens when the CPU is too busy and doesn't respond to callback (n) before it is time for callback (n+1) ? Is there a priority structure that prevents that?

As far as I can see at the moment I don't think there is a specific priority on the callback but it looks like there can be a queue of buffers so that in principle even if one takes a bit longer it shouldn't matter so much. However, I logged the max variable in h264_enable_output which seems to be something to do with number of buffers being used. This had a value 1. I'm not quite sure where that is coming from.

OK. Just as an experiment I logged recommended and minimum buffer size and numbers returned from the h264 component.
Size recommended 65536, min 65536
Number recommended 1 min 1
I then hard wired the number to 4 instead of using the recommended. Video capture still seems to function so the rest of the code seems to be happy with this.
I do not know if this is going to help and I get very few iframe errors  myself.

I seem to get more iframe errors than you. One difference might be my systems also have a "sitecopy" program run every 10 minutes from cron that copies any new .jpg files found in /var/www/media to a remote FTP site, so there's more CPU and network activity as well as contention for access to the SD card.  ...hmm, maybe I should set sitecopy to run at minimum priority.

@jbeale1 when you say "more iframe errors than you" did you mean me? it's a flat convo so can't tell who you're referring to. Either way, what's your count? These days I seem to be getting bursts of about 100 errors a pop about twice or 3 times a day. The pi is doing nothing else, B+ model, no overlock.

@spikedrba I meant more errors than roberttidey, but yes that number sounds similar to what I have seen. I am running the software on four machines right now; A, A+ and 2x Model 2. I think I see more iframe errors with the pre-event buffer enabled (eg 3000 msec buffer setting, not default 0). I will check tonight to confirm.

I made a few more tests but without any results, it seems to be very tricky to find the problem. Unfortunately I can't continue my work for the time being because I need to finish other projects and I'm up to my ears with work, so I have to leave the project for now. Robert, feel free to add other contributors to this repo if you think, they can help and don't mess up to much ;). I'm excited what you guys will hack together in the next time and hope you'll find a way to make the intern motion detection as good as we hope it could be.
Thanks for all your work
Silvan


Refactor task input variable

The input and output variables are named the same. Even though it passes womtool validation and runs successfully on Cromwell, it fails on Terra. This PR renames the variable.

Thanks, @mwalker174! It was a tricky one since it worked as expected locally and on Cromwell but fail on Terra.


Dependency check v5 in jenkins

Hi
I have been using DC V4 for long time in the jenkins pipelines, using the method dependencyCheckAnalyzer. In this new version this method does not exist anymore
Is there any documentation on how to use this new version of dependency check in the pipelines?
Thanks

Use the pipeline sniplet generator included with Jenkins. It provides most options. See also https://github.com/jenkinsci/dependency-check-plugin/wiki/v5-Migration

Thanks. The arguments i need to pass are the same ones as if i was using the CLI, isnt it? Something like this:
-s mypath --project myname --disableCentral  --disableRetireJS -f JSON
Or does it need a new format?

Same as the CLI.


Add faraday-net_http_persistent gem as a dependency to fix a breaking change from faraday 2.0

Add faraday-net_http_persistent gem as a dependency to fix a breaking change from faraday 2.0
Resolves https://github.com/sirupsen/airrecord/issues/89

@sirupsen I tested this fix with these 2 sets of versions:

faraday 2.2.0 + faraday-net_http_persistent 2.0.1, which worked properly.
faraday 1.10.0 + faraday-net_http_persistent 1.2.0, which raised the following exception:

9: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/bundler/gems/airrecord-94b756a5e0a4/lib/airrecord/table.rb:166:in `save'
8: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/bundler/gems/airrecord-94b756a5e0a4/lib/airrecord/table.rb:153:in `create'
7: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/faraday-1.10.0/lib/faraday/connection.rb:281:in `post'
6: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/faraday-1.10.0/lib/faraday/connection.rb:516:in `run_request'
5: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:154:in `build_response'
4: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:167:in `app'
3: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:174:in `to_app'
2: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:57:in `build'
1: from /Users/xxxx/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/faraday-1.10.0/lib/faraday/dependency_loader.rb:18:in `new'
RuntimeError (missing dependency for Faraday::Adapter::NetHttpPersistent: cannot load such file -- net/http/persistent)

I don't exactly know why is this error appearing, since faraday-net_http_persistent is already a dependency of faraday 1.10.0, so adding it separately in the gemspec should be equivalent and shouldn't have any harmful effect.

I see the error appears because I removed the net-http-persistent dependency


Add gRPC service reflection support to arrow flight

Which issue does this PR close?
Closes https://github.com/apache/arrow-rs/issues/1706
Rationale for this change
We like to use the gRPC service reflection, via  grpcurl --plaintext localhost:8082 list, to list gRPC services in IOx
We have a flight based service that does not appear in the service list (see https://github.com/influxdata/influxdb_iox/issues/4543)
I think this is because the arrow-flight library doesn't create the appropriate descriptors, but I am not sure
What changes are included in this PR?
Try to generate the appropriate service descriptor
Are there any user-facing changes?
Can now use reflection (if using https://crates.io/crates/tonic-reflection)

I think we would likely need to make the generated descriptor set available as a constant within the produced binary, so that it can be passed to the reflection https://docs.rs/tonic-reflection/0.4.0/tonic_reflection/server/struct.Builder.html.
Effectively all gRPC reflection is, is a service that returns the compiled descriptor sets, which in turn are just a parsed version of the proto files, represented as protobuf - https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/descriptor.proto

Closing this PR as it isn't working -- have filed https://github.com/apache/arrow-rs/issues/1706 to track the real issue and maybe someone else will want to work on it


Debug Failure: no message - TS 2.7.1

Search Terms:
Debug failure, checkUnusedClassMembers
Code
It is tough to provide code as the entire repo is failing and we have no error

ts
Please clone this repo: https://github.com/BabylonJS/Babylon.js
Go to /tools/gulp
npm install
gulp
Expected behavior:
Compiling should work
Actual behavior:
We are receiving a "Debug failure" error with no more detail. Same code works with TS 2.6.2

Duplicate of https://github.com/Microsoft/TypeScript/issues/21491. Fix should be available in typescript@next today and in the upcoming<EMAIL_ADDRESS>release in the next 2 weeks.
For workarounds on 2.7.1, the crash is triggered by having ; after a method or accessor declaration in a class body, removing these should avoid the crash. See https://github.com/Microsoft/TypeScript/issues/21491#issuecomment-362390885 for more details.

Sorry for the duplicate, was not able to find it (Should improve my search skills)


As IDIM, I want customers to provide detailed info when requesting BCeID so that I can create the Service Agreement.

to be added once we have shared client journey experience


Twisted Backend

Thank you for your work on this async adapter. Really great.
I would like to implement a Twisted backend. This is prompted by issue 458 from the elasticsearch-py repository.
Should I submit a PR against this repository, or would a separate repository be better (i.e. elasticsearch-py-twisted)? If the former, then I would create two new directories elasticsearch-async/twisted and test_elasticsearch_async/twisted. (Some code duplication might crop up, but a subsequent PR could take care of refactoring.)

A PR to this repository would be great! We can then abstract away the common stuff needed for the two backends and have optional dependencies on asyncio/twisted.
Let me know if there is anything I can do to help!

For an existing twisted backend you might want to use
https://github.com/avihad/twistes

If this functionality still desirable you should open an issue on the elasticsearch repository.


fix: remove devnet ARM builds for now

Curious why? Causing flakes?

Curious why? Causing flakes?

@ludamad yeah, seems unable to pick an arm runner? Phil started a run yesterday which was trying to get a spot instance for 16h. Is there a fix I'm not aware of for this?

Arm build is taking 1hr+ to complete


Trouble installing this package

Hey there,
I am running MacOS, Go 1.7, running "go get github.com/lestrrat/go-libxml2" I get
'libxml/HTMLparser.h' file not found
include <libxml/HTMLparser.h>
I have done sudo brew install libxml2 and etc
I have made an alias to "libxml" inside my /usr/local/include/
I can't seem to figure out how to check where the default include path is when using go get, I can compile fine under GCC (not the Go library though)
I have worked with libxml in C before and it has worked, but I can't figure it out using Go; any help would be appreciated.
HOMEBREW_VERSION: 0.9.9
ORIGIN: https://github.com/Homebrew/brew
HEAD: 7c7e2d00af9d8a05f4736c368804713fba61a422
Last commit: 9 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 12f6a5e8a13f05bb045638deb0032882012af514
Core tap last commit: 10 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: octa-core 64-bit haswell
Homebrew Ruby: 2.0.0-p648
Clang: 7.3 build 703
Git: 2.6.4 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
Java: 1.8.0_45, 1.7.0_79
OS X: 10.11.5-x86_64
Xcode: 7.3
CLT: N/A
X11: 2.7.7 => /opt/X11
Edit:
It seems like Go expects my include path to be /usr/include/ - GCC was fine.
It wouldn't accept it as an alias so I had to copy the whole of /usr/local/include/ into /usr/include/
PS before copying my include folder into /usr/include - I tried go get -gccgoflags -I/usr/local/include,-I/usr/local/include/libxml2 ... and had the same issue.
Do you happen to know if I can reconfigure go to look in /usr/local/include instead ? Apart from that - library works great now 👯

Forgot to close, sorry.

Sorry for the late reply, but maybe you have a .pc (pkg-config file) somewhere in the discoverable path that points to /usr/include of /usr/local/include? Or, maybe you don't have a .pc file that points to /usr/local/include?
cgo isn't terrible smart about  finding paths and what not, and mostly relies on stuff like pkg-config to find those things

All good, I didn't have much luck with pkg-config it was pointing to some where else, how ever the current setup is working so I'll leave as is for now

Had this same problem on MacOS with libxml2 installed via brew.
PKG_CONFIG_PATH=/usr/local/opt/libxml2/lib/pkgconfig go get github.com/lestrrat-go/libxml2 worked for me.

How did you finally solve this problem

@xiaxichen read my edit it has how I solved it


Deccan Delight not working

hi,
Any problem with Deccan Delight, after the update. All files disappeared.

Is it really not working in Kodi? The distribution zip files and addons.xml is still here

Yes, it does not work in Kodi after update.
See this quick video link.
May help you where the problem is.
https://drive.google.com/file/d/1ktVKy1ppq3vt2FBJgfBMn-6D3FLVBlArjg/view?usp=sharing

Yes, two things

You need to install the latest repository file from zip
The repository zip is here https://github.com/Gujal00/GujalKodiWork/releases/download/1.6/repository.gujal-1.6.zip
Once you install the new repository DD should get updated to 1.4.0. You are currently having 1.3.8 which is two releases behind.

MHDTV has gone paid so wont work. Switch it off in the addon settings.

Yes Working. Thanks. I had to delete and reinstall to get the update for some reasons.
Thanks

Could you please tell me the latest steps to configure Deccan Delight in fire tv stick?

I cannot see latest serials on my PC. The Colors and Zee serials are atlas 4 days old. I can see on my cell phone running kodi

Why are you adding a comment to an issue closed 6 months ago? If you have a new issue, please raise a new one instead of adding comment to a closed issue as it will show up as a pending issue to be resolved.
Anyway two other people have reported the same issue and they are in open state. Will be resolved in the next release
Also what versions are running on your PC and on your cellphone? Just saying it is not working is not helpful

Could you plz post steps to configure Deccan delight in fire stick

@maddy0907 why would you comment on a issue closed 6 months ago? Use a new issue
Read this #45


cannot import circuitpython_typing.socket

One can't important circuitpython_typing.socket if the socket stub file is socket.pyi. If it's renamed to socket.py, it can be imported. I've asked about this on stackoverflow: https://stackoverflow.com/questions/71106529/cannot-import-pyi-file-in-stubs-package.

Fixed by 997ec78.


How to update index configuration

I defined an index with meilisearch do, indexed some documents, then realized I missed a filter attribute. I updated the meilisearch do block, and called Model.reindex!, but that does not seem to add the filter attribute. What do I need to do to update the index? Does this involve downtime for the index?
(Please document)

This was because

My defined attributes did not include the attribute I added to filterable_attributes.
Confusion on how to reindex (see #162)

Calling Model.reindex! after a change seems to work fine.


InvalidTemplateDeployment

I have tried to deploy 'Hyperledger Fabric on Azure Kubernetes Service' I'm getting the below error from Template:
{"code":"InvalidTemplateDeployment","details":[{"code":"AgentPoolK8sVersionNotSupported","message":"Provisioning of resource(s) for container service fbtest-HLF-AKS in resource group Fabric-dealer failed. Message: {\n "code": "AgentPoolK8sVersionNotSupported",\n "message": "Version 1.18.14 is not supported in this region. Please use [az aks get-versions] command to get the supported version list in this region. For more information, please check https://aka.ms/supported-version-list"\n }. Details: "}],"message":"The template deployment 'microsoft-azure-blockchain.azure-blockchain-hyper-20210607103318' is not valid according to the validation procedure. The tracking id is 'd71e1e5b-53d4-46b9-a70a-b77eb7aa69ef'. See inner errors for details."}

The supporting versions are not provided on the Azure GUI. Please refer to the attachment.

@alurujawahar is this issue related to AGIC in anyway ?