Origin as 'file://'. Support for multiple origins in config?

Hello!
Please, can anybody give an idea, what's wrong. I use Lumen on http://lumen.app URL on my local machine and get this error:
[2016-03-18 10:09:54] lumen.ERROR: exception 'InvalidArgumentException' with message 'url' in /home/vagrant/Code/lumen/vendor/neomerx/cors-psr7/src/Http/ParsedUrl.php:70
Stack trace:
#0 /home/vagrant/Code/lumen/vendor/neomerx/cors-psr7/src/Factory/Factory.php(51): Neomerx\Cors\Http\ParsedUrl->__construct('file://')
#1 /home/vagrant/Code/lumen/vendor/neomerx/cors-psr7/src/Analyzer.php(363): Neomerx\Cors\Factory\Factory->createParsedUrl('file://')
#2 /home/vagrant/Code/lumen/vendor/neomerx/cors-psr7/src/Analyzer.php(144): Neomerx\Cors\Analyzer->getOrigin(Object(Neomerx\CorsIlluminate\Adapters\IlluminateRequestToPsr7))
#3 /home/vagrant/Code/lumen/vendor/neomerx/cors-psr7/src/Analyzer.php(114): Neomerx\Cors\Analyzer->analyzeImplementation(Object(Neomerx\CorsIlluminate\Adapters\IlluminateRequestToPsr7))
#4 /home/vagrant/Code/lumen/vendor/neomerx/cors-illuminate/src/CorsMiddleware.php(116): Neomerx\Cors\Analyzer->analyze(Object(Neomerx\CorsIlluminate\Adapters\IlluminateRequestToPsr7))
#5 /home/vagrant/Code/lumen/vendor/neomerx/cors-illuminate/src/CorsMiddleware.php(63): Neomerx\CorsIlluminate\CorsMiddleware->getCorsAnalysis(Object(Illuminate\Http\Request))
#6 [internal function]: Neomerx\CorsIlluminate\CorsMiddleware->handle(Object(Illuminate\Http\Request), Object(Closure))
#7 /home/vagrant/Code/lumen/vendor/illuminate/pipeline/Pipeline.php(124): call_user_func_array(Array, Array)
#8 [internal function]: Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#9 /home/vagrant/Code/lumen/vendor/illuminate/pipeline/Pipeline.php(103): call_user_func(Object(Closure), Object(Illuminate\Http\Request))
#10 /home/vagrant/Code/lumen/vendor/laravel/lumen-framework/src/Application.php(1439): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#11 /home/vagrant/Code/lumen/vendor/laravel/lumen-framework/src/Application.php(1213): Laravel\Lumen\Application->sendThroughPipeline(Array, Object(Closure))
#12 /home/vagrant/Code/lumen/vendor/laravel/lumen-framework/src/Application.php(1153): Laravel\Lumen\Application->dispatch(NULL)
#13 /home/vagrant/Code/lumen/public/index.php(28): Laravel\Lumen\Application->run()
#14 {main}

It appears only on POST requests (in Postman).
My config file has:
S::KEY_SERVER_ORIGIN => [
        'scheme' => 'http',
        'host'   => 'lumen.app',
        'port'   => 8080
    ],

I'm not sure it's correct, but I can't find more documentation for this package. So I don't fully understand why should I specify this parameter.
By the way, how can I specify several hosts in the config file? For production server, for example? Can it automatically detect current origin?

Header Origin of your request has value file://
It seems the request is not send from page served by server but just an ordinary file. A correct value must be OK for parse_url.
You need to send requests from files served by web server.

Though error handling could be improved for this scenario

I've released a new version which writea to logs that such URLs cannot be parsed. You should
$ composer update

I'm closing the issue. If you feel it still exists please let me know.

Just ran into this and voila, here's the fix.
Thanks and keep up the good work! :)

@pavoltanuska thanks for your support :star:

Thank you the fix! I will try to it!
And what about several servers in S::KEY_SERVER_ORIGIN?

@Chimit If more people do need multiple origins for purpose I will look closer it it's OK with spec and common practices.

Sorry, may be I'm wrong, but is it not a common practice to use dev and production environments with different URLs? How can I solve this problem?

you can use .env file with env() function

Still have a strange problem on POST requests from Postman. I get 400 Bad request. Lumen logs are empty. If I disable this package, it starts working again. I don't have any idea what's the reason.

Hm... the problem is in Chrome version of Postman. Standalone Postman works perfect with this package.

@Chimit Can you turn logging on for the package and post logs here?

[2016-03-22 09:37:53] lumen.DEBUG: CORS analysis for request started.
[2016-03-22 09:37:53] lumen.INFO: Request origin is not allowed. Check config settings for Allowed Origins. {"origin":"[object] (Neomerx\\Cors\\Http\\ParsedUrl: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop)"}
[2016-03-22 09:37:53] lumen.DEBUG: CORS analysis for request completed.

@Chimit you can allow all origins by setting this to true


go1.7 gc-related runtime panic, "workbuf is empty"

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go1.7 (judging from the panic's stackstrace)
Does this issue reproduce with the latest release?
unknown
What operating system and processor architecture are you using (go env)?
$ uname -a
Linux server26 4.11.9-coreos #1 SMP Thu Jul 6 00:54:42 UTC 2017 x86_64 Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz GenuineIntel GNU/Linux
(32 cores, 256GiB ram)

the server didn't have go installed
What did you do?
The issue has been observed on a running CoreOS 1409.6.0 instance, in containerd, version 0.2.4 commit: v0.2.5
$ sha256sum containerd
c3061b5dd3f320ff8e1a3b62217b5ad01ba2f9c8db8a28ff359c6dfbc9280863  containerd

What did you see instead?
sanitized logs of journalctl -xu containerd
containerd[4145]: fatal error: workbuf is empty
containerd[4145]: fatal error: workbuf is empty
containerd[4145]: fatal error: workbuf is empty
containerd[4145]: goroutine 130 [running]:
containerd[4145]: runtime.throw(0x8853e3, 0x10)
containerd[4145]:         /usr/lib/go1.7/src/runtime/panic.go:566 +0x95 fp=0xc4202716a0 sp=0xc420271680
containerd[4145]: runtime.(*workbuf).checknonempty(0x7fa713f99240)
containerd[4145]:         /usr/lib/go1.7/src/runtime/mgcwork.go:295 +0x4e fp=0xc4202716c0 sp=0xc4202716a0
containerd[4145]: runtime.trygetfull(0xc420f00000)
containerd[4145]:         /usr/lib/go1.7/src/runtime/mgcwork.go:351 +0x45 fp=0xc4202716e8 sp=0xc4202716c0
containerd[4145]: runtime.(*gcWork).tryGet(0xc42001d228, 0xc42001d228)
containerd[4145]:         /usr/lib/go1.7/src/runtime/mgcwork.go:153 +0x8f fp=0xc420271710 sp=0xc4202716e8
containerd[4145]: runtime.gcDrain(0xc42001d228, 0x5)
containerd[4145]:         /usr/lib/go1.7/src/runtime/mgcmark.go:992 +0x2da fp=0xc420271748 sp=0xc420271710
containerd[4145]: runtime.gcBgMarkWorker(0xc42001c000)
containerd[4145]:         /usr/lib/go1.7/src/runtime/mgc.go:1460 +0x1d1 fp=0xc4202717b8 sp=0xc420271748
containerd[4145]: runtime.goexit()
containerd[4145]:         /usr/lib/go1.7/src/runtime/asm_amd64.s:2086 +0x1 fp=0xc4202717c0 sp=0xc4202717b8
containerd[4145]: created by runtime.gcBgMarkStartWorkers
containerd[4145]:         /usr/lib/go1.7/src/runtime/mgc.go:1349 +0x98

I cannot do further investigation on that instance, but I may do additional pointed out step when the issue re-occurs.

/cc @aclements
I believe that, with 1.9 already out, 1.7 doesn't even have security updates support, though. See https://github.com/golang/go/wiki/Go-Release-Cycle#release-maintenance

gHi @bohdantrotsenko. Thanks for the report, but Go 1.7 is no longer maintained. Are you able to reproduce this with a newer version of Go?

sadly, not yet. (namely, within a day)
although the significance of the issue, apparently, will change that.


docs: add databases section @ site home page.

Need to host the logos instead of using external URLs, other than that, pretty much done.
Logos order:

core dialects.
by revenue / funds raised - as a way to guess market share / marketing power.
by github stars.

Very cool. Some thoughts:

Would be useful if the the blocks were clickable (e.g. click on planetscale sends you to the planetscale docs, in another window, so we don't lose the session)
With regards to copy, what do you think of "official dialect" as opposed to "core dialect"?

@fhur
Good ideas! Also, we should probably optimize loading of these images, I lazily copied and used external urls for this PoC.
I think "official" might confuse people. e.g. thinking that we have an official Supabase dialect.
Sent you an invite.

Also, "official" sheds a negative light on "community".


Error: HTTP Status 0

I'm getting a csv file with:
Error: HTTP Status 0
And this in the logs:
TypeError: undefined is not an object (evaluating '$.browser.version.indexOf')
https://images-na.ssl-images-amazon.com/images/G/03/authportal/common/js/ap_global-1.1._CB309248194_.js:300
https://images-na.ssl-images-amazon.com/images/G/03/authportal/common/js/ap_global-1.1._CB309248194_.js:304
https://images-na.ssl-images-amazon.com/images/G/03/authportal/common/js/ap_global-1.1._CB309248194_.js:366 in loadApGlobalScript
https://images-na.ssl-images-amazon.com/images/G/03/authportal/common/js/ap_global-1.1._CB309248194_.js:7

your probably using version 2.1+ it works fine with version 1.9.8 can you do
phantom.js --version

I'm getting the same error, does someone know how to solve this?

Does changing your reporting interface to "Classic" fix this?

I don't know what you mean by "reporting interface". I'm running the Script straight from the command line.
Somehow it works perfectly now...
I think the problem was running the script multiple times per minute i.e. sending the request too often to amazon.
Thanks - great job.


[Bug]: Table column with image type shows 'Invalid image' if image url contains comma signs

Is there an existing issue for this?

[X] I have searched the existing issues

Description
In the table widget, a column of type image fails with the error 'Invalid Image' when trying to display the url 'https://res.cloudinary.com/beatly/w_100,h_100,c_fill/aqdmh8zr8dcfalwotogl.jpg'.
Looking at the debug console in chrome, it turns out appsmith split the url at the first comma sign and tries to download 'https://res.cloudinary.com/beatly/w_100' which results in the error.
Cloudinary is one of many tools that puts image manipulation commands inside the url, and such commands are often separated with commas. Such urls should be supported...
Steps To Reproduce

Create a table widget
Set the table data to:
[
{
"image": "https://res.cloudinary.com/beatly/w_100,h_100,c_fill/aqdmh8zr8dcfalwotogl.jpg"
}
]
Change the column type of the 'image' column to 'Image'
See 'Invalid Image' error

Public Sample App
No response
Environment
Production
Issue video log
No response
Version
Cloud

This is still an issue.
No amazon image thumbnails can be used due to the comma present. E.g.
https://images-eu.ssl-images-amazon.com/images/I/81ZV8BPI1aL._AC_UL300_SR300,200_.jpg

Thanks @tophat1986 for the workaround 👏


Fix bug 43

This PR ensures that the expected contents are written to spec/support/matchers/rambo_matchers.rb, correcting bug #43.

Coverage remained the same at 99.216% when pulling 427d1ce3d4ad0993e8e02136954acf4390477300 on 43-write-matcher-file into fd285e30474f65fc894d33221460bee2f63dd2d4 on master.


Error when runing main_v2

Hello,
I am trying to run you great work with the command:
python main_v2.py -c config/multimodal_context_v2.yml
but got an error:
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/torch/nn/functional.py", line 1237, in leaky_relu
result = torch._C.nn.leaky_relu(input, negative_slope)
RuntimeError: CUDA error: no kernel image is available for execution on the device
I googled it downgrade my pytorch version to 1.5.0 but the error still persists.
Do you know how to fix this error? Any help will be appreciate!!
My torch version:
pytorch                   1.5.0           py3.7_cuda10.2.89_cudnn7.6.5_0    pytorch
torchvision               0.6.0                py37_cu102    pytorch

Unfortunately, I cannot help much with versioning or cuda backend issues. If it helps, these are the configs I have:

python version: running python --version gives 3.7.13
torch version: running pip freeze  gives torch==1.12.0+cu116
torchvision version: torch version: running pip freeze  gives torchvision==0.13.0+cu116
cudnn version: running torch.backends.cudnn.version() gives 8302
cuda version: running torch.version.cuda gives 11.6

Thank you for your reply!
I upgrade all related version to be consistent with cuda 11.7 and the problem is gone.
But at the every last step, another error occurs when executing the following code:
ani.save(video_path, fps=15, dpi=80)
And here is the error message:
MovieWriter stderr:
[libopenh264 @ 0x55eec1dec300] Incorrect library version loaded
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Traceback (most recent call last):
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/animation.py", line 234, in saving
yield self
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/animation.py", line 1093, in save
writer.grab_frame(**savefig_kwargs)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/animation.py", line 352, in grab_frame
dpi=self.dpi, **savefig_kwargs)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/figure.py", line 3058, in savefig
self.canvas.print_figure(fname, **kwargs)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 2325, in print_figure
**kwargs)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 1648, in wrapper
return func(*args, **kwargs)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/_api/deprecation.py", line 415, in wrapper
return func(*inner_args, **inner_kwargs)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 486, in print_raw
fh.write(renderer.buffer_rgba())
BrokenPipeError: [Errno 32] Broken pipe
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main_v2.py", line 149, in
s2ag_epoch=290, samples=samples, make_video=True, save_pkl=True)
File "/home/zhewei.qiu/audio2body/3d/speech2affective_gestures/processor_v2.py", line 1564, in generate_gestures_by_dataset
make_video=make_video, save_pkl=save_pkl)
File "/home/zhewei.qiu/audio2body/3d/speech2affective_gestures/processor_v2.py", line 1408, in render_clip
delete_audio_file=False)
File "/home/zhewei.qiu/audio2body/3d/speech2affective_gestures/utils/gen_utils.py", line 128, in create_video_and_save
ani.save(video_path, fps=15, dpi=80)  # dpi 150 for a higher resolution
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/animation.py", line 1093, in save
writer.grab_frame(**savefig_kwargs)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/animation.py", line 236, in saving
self.finish()
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/animation.py", line 342, in finish
self._cleanup()  # Inline _cleanup() once cleanup() is removed.
File "/home/zhewei.qiu/anaconda3/envs/s2ag-env/lib/python3.7/site-packages/matplotlib/animation.py", line 374, in _cleanup
self._proc.returncode, self._proc.args, out, err)
subprocess.CalledProcessError: Command '['ffmpeg', '-f', 'rawvideo', '-vcodec', 'rawvideo', '-s', '960x320', '-pix_fmt', 'rgba', '-r', '15', '-loglevel', 'error', '-i', 'pipe:', '-vcodec', 'h264', '-pix_fmt', 'yuv420p', '-y', '/home/zhewei.qiu/audio2body/3d/speech2affective_gestures/outputs/genea_challenge_2020/videos_trimodal_style/temp_TestSeq001_s0_0.00_128.83_290_0.mp4']' returned non-zero exit status 1.
Is that because the version of matplotlib? And can you share which version of matplotlib you use?

Yes, seems like another version issue. I am using matplotlib==3.5.2.

There is something weird happening. I downgrade matplotlib from 3.5.3 to 3.5.2 and still get same error. But when I try using debug mode is vs code, the script seems to run successfully and I can get the .wav and .mp4 file.
Also, can you provide pure inference script, e.g., the input is audio, text, identity and the output is the predict 3d position of all joint?

It's non-trivial to prepare a "pure" inference script because of all the library, path, and data dependencies. But if you run generate_gestures_by_dataset (beginning line 1441) in processor_v2.py with the dataset set to 'ted_db', that's essentially the default inference script.

Got you. And is there a way to get the rotation information from the 3d joint position output by your model or do you plan to use rotation information as a supervisor since 3d joint position is not friendly in existing render engine such as UE and blender?

A good question! I would recommend checking out our blender files inside the blender folder, where we essentially use a form of IK to obtain the rotations and render the motions on 3D meshes. Our approach here is similar to that of https://github.com/ai4r/Gesture-Generation-from-Trimodal-Context, so you can check out their paper and blender project as well for additional context.

I tried the repo you mentioned. I got the same error when making the result video. Seems that your repo and his are highly related. I tried different version of matplotlib but the error is not gone.

Is it the same situation where things run in debug mode but not otherwise? Are you able to generate the pose outputs? I'm not sure what the exact issue could be (and I can't seem to replicate it), but maybe you can try other animation scripts with the generated poses outputs.


5/24 Publish

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

2017-03-09-profile/docs-ref-autogen/ext/eventgrid/eventgrid/event-subscription.yml
:white_check_mark:Succeeded
View (azure-cli-2017-03-09-profile)

docs-ref-conceptual/azure-cli-extensions-list.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

docs-ref-conceptual/azure-cli-extensions-overview.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

docs-ref-conceptual/install-azure-cli-apt.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

latest/docs-ref-autogen/batchai/cluster.yml
:white_check_mark:Succeeded
View (azure-cli-latest)

latest/docs-ref-autogen/ext/eventgrid/eventgrid/event-subscription.yml
:white_check_mark:Succeeded
View (azure-cli-latest)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Extended matlab parser with structures, cell matrices and character matrices

Since https://discuss.mathdotnet.com/c/numerics is giving me a timeout I would like to use this PR to ask the Mathnet maintainers if there is a chance for this parsing code to be included.
What can it do?

Parse cell matrices both at the file root and when nested
Parse structures both at the file root and when nested
Parse character matrices (Only UTF8 tested right now)
Parsing regular numeric matrices is unchanged, also when they are inside a cell or structure

What still needs to be done for PR readiness (as far as im aware)

More commenting
Tests for different char array encodings
Fixing some code duplication between cell matrix and structure parsing

This is all based on the V5 spec at https://www.mathworks.com/help/pdf_doc/matlab/matfile_format.pdf

Thank you very much!


Release v1.4.0

Codecov Report
Attention: Patch coverage is 0% with 55 lines in your changes are missing coverage. Please review.

:exclamation: No coverage uploaded for pull request base (master@45a252b). Click here to learn what that means.

:exclamation: Current head b157376 differs from pull request most recent head f3ad740. Consider uploading reports for the commit f3ad740 to get more accurate results

Files
Patch %
Lines

.../com/webank/wecross/stub/fabric2/FabricDriver.java
0.00%
40 Missing :warning:

...a/com/webank/wecross/stub/fabric2/FabricBlock.java
0.00%
8 Missing :warning:

...webank/wecross/stub/fabric2/FabricTransaction.java
0.00%
6 Missing :warning:

...om/webank/wecross/stub/fabric2/utils/TarUtils.java
0.00%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff            @@
##             master      #34   +/-   ##
=========================================
  Coverage          ?   13.64%
  Complexity        ?      113
=========================================
  Files             ?       49
  Lines             ?     4616
  Branches          ?      396
=========================================
  Hits              ?      630
  Misses            ?     3933
  Partials          ?       53

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Memory leak on .NET Core

I'm suspecting a memory leak when running Microsoft.Data.Sqlite 1.1.1 with .NET Core (tested on runtime 1.1.0 on Windows 10).
I've made a small program that runs INSERT, UPDATE and SELECT in a while(true) loop. 5 threads is running the loop simultaneously and it eats memory quite quickly - ~25MB -> 300MB memory in about 5 mins. Also tested briefly on net461 where it goes from 15MB -> 25MB in about 5mins (seems slower). I've got a feeling that it eats quicker when larger strings (700-800 bytes) are being inserted/updated/selected.
I have profiled it with dotMemory from JetBrains but it doesnt get a good overview for unmanaged memory.
Connectionstring that is being used is "Data Source=./data/storage.db"
The ADO.NET code have the following pattern:
private IDbConnection GetOpenConnection()
{
    var connection = new SqliteConnection(ConnectionString);
    connection.Open();
    return connection;
}

public IEnumerable<Foo> GetFoos()
{
    using (var connection = GetOpenConnection())
    using (var command = connection.CreateCommand())
    {
        command.CommandText = @"some sql with params @p1 and @p2";

        var parameter = command.CreateParameter();
        parameter.ParameterName = "@p1";
        parameter.Value = p1 ?? DBNull.Value;
        command.Parameters.Add(parameter);

        var parameter = command.CreateParameter();
        parameter.ParameterName = "@p2";
        parameter.Value = p2 ?? DBNull.Value;
        command.Parameters.Add(parameter);

        using (var reader = command.ExecuteReader())
        {
            while (reader.Read())
            {
                yield return new Foo
                {
                    Bar = reader.GetGuid(0),
                    Baz = reader.GetString(1),
                    FooBar = reader.GetDateTime(2),
                };
            }
        }
    }
}

@malmgren80 Could you try this with the 2.0 preview2 release? The native interop was changed between the two releases, so it might be that this is fixed with 2.0 already.

I tested this on the nightly builds, and the private memory never went over 6MB.

I've experienced recently same issue while running load test on WebAPI with SQLite at the back-end.
Everything was running in docker container microsoft/aspnetcore:2.0 (.net core runtime: 2.0.5).
In fact load test was hardly doing much of load - 200 requests in 5 sec (2 threads), every request was doing simple SELECT operation (returning 5 records).
After the test process consumed +200MB (I was monitoring it in real time on server).
When I increased test load to 10k requests it burned around 4GB of RAM.

@yevgen1scr Can you please create a new issue including a runnable project/solution that demonstrates the behavior you are seeing?


New command: Get the number of Microsoft Teams daily unique users by device type

graph report getTeamsDeviceUsageUserCounts --period D7
API Reference - https://docs.microsoft.com/en-us/graph/api/reportroot-getteamsdeviceusageusercounts?view=graph-rest-1.0&tabs=cs
This is a continuation of the report located at office365-cli/docs/manual/docs/cmd/graph/report.
Please assign to me. Thank you.

Let's use graph report teamsdeviceusageusercounts as the command name. Other than that, all yours! 👏

@waldekmastykarz
Sure, Thank you.

Thank you Joseph (@sprider )! It is part of the v2 beta now.


Update instructions for chart release and installation.

We have a released chart but no instructions on how to use it.   This needs to merge ASAP.
Supersedes #751
Fixes #701

I'm waiting on @sfxworks to sign the CLA for the email he authored the commit with.

Sorry for the delay! Added services@ to my github account.

LGTM

/lgtm
Thanks @sfxworks @marun !


Helpme

New Feature Submissions:
To run tests locally, use make if you're on Mac or Linux.

[ ] Does your submission pass all the tests?
[ ] Have you lint your code locally prior to submission?
[ ] Have you written proper tests to your added code?

New Dialog Submissions:
User messages have - -, bot messages have   -

[ ] Does all your user messages have lower letters?
[ ] Have you lint your dialogs locally prior to submission?
[ ] Have you written proper tests to your added dialogs on test_dialogs?
[ ] Does your dialog have an existing category? If not, did you create it?

Changes to Core Features:

[ ] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests or modified old ones for your core changes, as applicable?
[ ] Have you successfully ran tests with your changes locally?

@vinicius0197 como new feature!


Verify email does not send after deployment: how to set from field

I am trying to send emails from my deployed app (at digital ocean) via Mandrill and the willio:mandrill package. After deployment, I am able to send an email from my app using the Mandrill API
Mandrill.messages.send({ message: { "text": "Greetings from XBdash.com!", "from_email"<EMAIL_ADDRESS>"from_name": "Keith - XBdash.com", "subject": "App Started", "to": [ { email<EMAIL_ADDRESS>name: "Keith Guirao" } ] } }, function( error, response ) { if ( error ) { console.log( error ); } else { console.log( response ); } });
This email sends so I know I have my MAIL_URL set and am able to access my mandrill account from the application. However, when I try to get an account-tied email such as verify email or reset password, it does not send. I believe this is because I am unable/dont know how to set the "from" field of email like I am with the above code. However I imagine that it could also be because of something else so here is the code that alters what the verifyEmail email looks like.
Accounts.emailTemplates.verifyEmail.subject = function(user) {     return 'Activate your XBdash account'; } Accounts.emailTemplates.verifyEmail.html = function (user, url) {     var result;     try {         result = Mandrill.templates.render({             template_name: 'verify-email',             template_content: [                 {                     name: 'CONFIRMURL',                     content: url                 },                 {                     name: 'FNAME',                     content: user.username                 }             ],             merge_vars: [                 {                     name: 'CONFIRMURL',                     content: url                 },                 {                     name: 'FNAME',                     content: user.username                 }             ]         });     } catch (error) {     	console.error('Error while rendering Mandrill template', error);     }     return result.data.html; }
How would I be able to set the from field or otherwise fix the verify email through mandrill?
BTW, I was using this guide for help but still havent been able to fix the issue. https://themeteorchef.com/snippets/sending-email-with-mandrill/

I tto would really like to know how to get emails working with useraccounts boilerplate. Any help appreciated. Thanks.


Not accurate measurements

Thanks for your contributions,
I have integrated your code, into https://github.com/vchoutas/smplify-x
I fitted an image then I get a 3D SMPLX Model, then I have used your code to calculate the measurments
The pose is little bit correct, however as first insight, my height is not 162.31, it's around 176.

And the following has been integrated into :
`   measurer = MeasureBody(model_type)
measurer.from_verts(torch.tensor(vertices))
measurement_names = measurer.all_possible_measurements
measurer.measure(measurement_names)
measurer.label_measurements(STANDARD_LABELS)
    # Add measurements to the result dictionary
    results[min_idx]['result']['measurements'] = measurer.measurements
    results[min_idx]['result']['labeled_measurements'] = measurer.labeled_measuremen`

in the following line :
https://github.com/vchoutas/smplify-x/blob/master/smplifyx/fit_single_frame.py#L508

tried the following also
      measurer.measure(measurement_names)
        new_height = 175
        measurer.height_normalize_measurements(new_height)

        measurer.label_measurements(STANDARD_LABELS)

        # Add measurements to the result dictionary
        results[min_idx]['result']['measurements'] = measurer.measurements
        results[min_idx]['result']['labeled_measurements'] = measurer.labeled_measurements

All the measurements are extracted exactly as shown in the Figure you attached. You might say their definition is "fixed" w.r.t. SMPL(-X) meshes, yes. If you specify a different height, the mesh is scaled and so are the corresponding body measurements. Maybe I misunderstood your question...

@kristijanbartol I have tried to export the betas, but they are not reflecting the actual shape parameters. Is what I did wrong? I mean the exported body,
def export_smpl_data_for_unity(updated_length_definitions, body_model, body_pose, measurer, camera, result_fn='smpl_data_for_unity.json'):
    # Extract SMPL keypoints (joints) and vertices
    model_output = body_model(return_verts=True, body_pose=body_pose)  # Run the body model
    vertices = model_output.vertices.detach().cpu().numpy().squeeze()
    faces = body_model.faces.astype(int)
    betas = body_model.betas.detach().cpu().numpy().squeeze()

    print("Available landmarks:")
    for key, value in measurer.landmarks.items():
        print(f"{key}: {value}")

    # Extract landmarks and their corresponding vertex indices
    landmarks = {}
    landmark_to_vertex = {}  # To map landmarks to their corresponding vertices
    for landmark_name, landmark_index in measurer.landmarks.items():
        if isinstance(landmark_index, tuple):
            # If it's a tuple, take the average of the specified vertices
            landmark_position = np.mean(vertices[list(landmark_index)], axis=0)
            landmark_to_vertex[landmark_name] = list(landmark_index)
        else:
            landmark_position = vertices[landmark_index]
            landmark_to_vertex[landmark_name] = [landmark_index]

        # Convert landmark position to a dict with x, y, z keys
        landmarks[landmark_name] = {
            "x": float(landmark_position[0]),
            "y": float(landmark_position[1]),
            "z": float(landmark_position[2])
        }

    # Prepare measurement definitions
    length_definitions = {}
    for name, definition in updated_length_definitions.items():
        valid_landmarks = [lm for lm in definition if lm in landmarks]
        if len(valid_landmarks) == len(definition):
            # Convert the landmarks to dict format for Unity
            length_definitions[name] = [landmarks[lm] for lm in valid_landmarks]
        else:
            print(f"Warning: Some landmarks missing for length definition '{name}'")

    circumference_definitions = {}
    for name, definition in measurer.circumf_definitions.items():
        circumference_definitions[name] = {
            "LANDMARKS": definition["LANDMARKS"],
            "JOINTS": definition["JOINTS"]
        }

    # Convert vertices to dict format with x, y, z keys
    vertex_dicts = [{"x": float(v[0]), "y": float(v[1]), "z": float(v[2])} for v in vertices]

    # Prepare data for export
    unity_data = {
        "landmarks": landmarks,
        "shape_parameters": betas.tolist(),  # Add shape parameters (betas) to the output
        "vertices": vertex_dicts,
        "faces": faces.tolist()
    }

    # Export to JSON
    with open(result_fn, 'w') as f:
        json.dump(unity_data, f, indent=2)

    print(f"SMPL data exported to {result_fn}")

Hi @AndreAhmed

Thanks for your contributions,
I have integrated your code, into https://github.com/vchoutas/smplify-x
I fitted an image then I get a 3D SMPLX Model, then I have used your code to calculate the measurments
The pose is little bit correct, however as first insight, my height is not 162.31, it's around 176.

Problem 1: ambiguity problem of estimating anything from a single image you discussed with @kristijanbartol

Problem 2: you are estimating measurements from a posed subject (as you show in the image attached). The measurements in this repo are estimated with cutting planes or simple distances between landmarks and are therefore meant to be estimated from T-pose. If you pose a subject, the measurements change (ex. the height is estimated as the distance between the top of the head and the floor located in between the heels - if you crouch, this distance changes)

@kristijanbartol
Thanks for your quick input. But how about other measurements, like Waist, stomach,..etc? I assume they are fixed ?

Measurements are defined in measurement_definitions.py and adding new measurements is explained in the Readme
https://github.com/DavidBoja/SMPL-Anthropometry/blob/bde9bb99dcdb95d07f5c398c85e5639ac2b0e029/README.md?plain=1#L144-L162

@kristijanbartol I have tried to export the betas, but they are not reflecting the actual shape parameters. Is what I did wrong? I mean the exported body. So What I did is to export the json, and render the SMPLX in Unity3D, and attach the shape parameters, but the body is not exactly the same as the one you constructed for measurments, in terms of height, body size,..etc.

Are you using the same SMPL / SMPLX model in all cases?

@DavidBoja Yes I'm using the same SMPL/SMPLX

@DavidBoja  The triangulated mesh has exact same shape parameters in terms of belly stomach, and other features. but using just the betas alone, they don't define well the exact smplx mesh if they are used alone.

@DavidBoja The triangulated mesh has exact same shape parameters in terms of belly stomach, and other features. but using just the betas alone, they don't define well the exact smplx mesh if they are used alone.

So you have an SMPL with its shape parameters.
You load those shape parameters into Unity3D to create the SMPL.
And then the SMPL mesh from this repository and from from Unity3D do not correspond?

Yes they don't correspond at all, I asked Shapy guys, but they didn't answer. Also how do I export your measurment points done in python to Unity3D ?

Well, then I'd presume that you are using different models; they should be equal if you are using the same model.
There are several SMPL models: SMPL, SMPL+H, SMPLX, STAR,.. so check what are you using. This repo uses either SMPL or SMPLX. I'm not familiar with Unity so I really couldn't tell you what is going on under the hood.
Measurement points for lengths are basically the landmarks which are defined in landmark_definitons.py.
To get the circumference points, you can edit the code here:
https://github.com/DavidBoja/SMPL-Anthropometry/blob/22324d1f23dcbd7922781ca26f7ac333354caf45/measure.py#L158-L198
and take the slice_segments_hull, which are basically segment points on the mesh that create the circumference measurement.


Crawling Issue For Crawl4ai

I have been using crawl4ai for my applications for a while now and it has worked pretty well. But I noticed a really huge issue for the past two days. I've been trying to crawl web content but instead it only returns a single string of hexadecimal code. I have been using Crawl4AI 0.4.23 .
import asyncio
from crawl4ai import AsyncWebCrawler

async def main():
    # Create an instance of AsyncWebCrawler
    async with AsyncWebCrawler(verbose=True) as crawler:
        # Run the crawler on a URL
        result = await crawler.arun(url="https://www.nbcnews.com/business")

        # Print the extracted content
        print(result)

# Run the async main function
asyncio.run(main())

This is the template code from the official crawl4ai documentation and on running that i get:
[INIT].... → Crawl4AI 0.4.23
[FETCH]... ↓ https://www.nbcnews.com/business... | Status: True | Time: 0.04s
[SCRAPE].. ◆ Processed https://www.nbcnews.com/business... | Time: 4ms
[COMPLETE] ● https://www.nbcnews.com/business... | Status: True | Total: 0.04s
url='https://www.nbcnews.com/business' html='c3d991002922a68c' success=True cleaned_html='
c3d991002922a68c
' media={'images': [], 'videos': [], 'audios': []} links={'internal': [], 'external': []} downloaded_files=None screenshot=None pdf=None markdown='c3d991002922a68c\n' markdown_v2=MarkdownGenerationResult(raw_markdown='c3d991002922a68c\n', markdown_with_citations='c3d991002922a68c\n', references_markdown='\n\n## References\n\n', fit_markdown='c3d991002922a68c\n', fit_html='

c3d991002922a68c

') fit_markdown='' fit_html='' extracted_content='91ec982f92fcfe29' metadata={} error_message='' session_id=None response_headers={} status_code=200

Please resolve this or provide a solution to work around this. Im using macOS Sequoia 15.2 .

And if you use
print(result.markdown)
What will be the result?

on using
print(result.markdown)

this is the result:
[INIT].... → Crawl4AI 0.4.23
[FETCH]... ↓ https://www.nbcnews.com/business... | Status: True | Time: 0.04s
[SCRAPE].. ◆ Processed https://www.nbcnews.com/business... | Time: 2ms
[COMPLETE] ● https://www.nbcnews.com/business... | Status: True | Total: 0.04s
c3d991002922a68c

is it just on my system or is it in everyones? would really like an update on this.

@Aashmit You can try my code. Works fine.
from crawl4ai import BrowserConfig, CrawlerRunConfig, AsyncWebCrawler, CacheMode
import asyncio
from playwright.async_api import Page

async def main():
    browser_config = BrowserConfig(
        headless=False,
        viewport_width=1920,
        viewport_height=1080,
        text_only=False,
    )
    crawler_config = CrawlerRunConfig(
        cache_mode=CacheMode.DISABLED,
        page_timeout=60000,
        delay_before_return_html=10,
        prettiify=True,
        keep_data_attributes=False,
        exclude_external_links=False,
        exclude_social_media_links=False,
        screenshot=False,
    )

    url = "https://www.nbcnews.com/business"
    async with AsyncWebCrawler(config=browser_config) as crawler:
        result = await crawler.arun(
            url=url,
            config=crawler_config,
        )
        print(result.success, result.error_message)

        # write raw html to file
        with open("result_raw.html", "w", encoding="utf-8") as f:
            f.write(result.html)

        # write markdown to file
        with open("result.md", "w", encoding="utf-8") as f:
            f.write(result.markdown)

if __name__ == "__main__":
    asyncio.run(main())

hmm interesting, i think its because the websites might have updated to prevent scraping content.
even in your code, if we comment out the
config=crawler_config

line , the code will not work and only an hexadecimal will be stored in the md file.
So i think its safe to say for all crawlers, the CrawlerRunConfig is important now.
thank you so much for your help  @pleomax0730 .


Vector DataContainer has no geometry

b = vg.allocate('random')
print(b.geometry)

Fixed here

Make a small PR just with this please.
On Sat, 23 Nov 2019, 17:42 Vaggelis Papoutsellis<EMAIL_ADDRESS>wrote:

Fixed here
https://github.com/vais-ral/CCPi-Framework/blob/new_Functions_version/Wrappers/Python/ccpi/framework/framework.py#L1552
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/vais-ral/CCPi-Framework/issues/452?email_source=notifications&email_token=ACMDSCHH4V4XJBBZNMASG3TQVFTQRA5CNFSM4JQ2HOUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE72BXQ#issuecomment-557818078,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACMDSCFRRKMRPNFBZTYBUTTQVFTQRANCNFSM4JQ2HOUA
.

and will close here

I think this is fixed now


Examples for Operations and Operators

AC
All operators and operations in the reference guide contain examples

Duplicate of https://github.com/cdisc-org/cdisc-rules-engine/issues/425


panic in decodeSymbol2

When unpacking Cscan(.net_2x_3x).exe from https://github.com/k8gege/K8tools/blob/master/K8Cscan5.4_20191101.rar, rardecode panics in decodeSymbol2, line 805:
for m.charMask[states[i].sym] == m.escCount {
Apparently i equals len(states) (both 22), causing the issue.
If there is any further information that you require, please let me know.

If I use the experimental branch, It no longer panic's but it still has corrupt ppm data errors. Are the archive files not valid/corrupt?

Possibly - I haven't created them and don't know. They're encrypted, might that cause the errors?

Yes, that would happen if they are encrypted.
I copied the development version of ppm_model.go to main branch and tagged a new version. It seem to at least not panic for that file.


Exposing NoteId and NoteIdUtilities; Fixing NoteId.Equals bug

Changing the access modifiers of the NoteId and NoteIdUtilities classes from internal to public, to enable external consumers to access these classes from their applications.
This change also includes a small fix for the NoteId.Equals method, which could theoretically incorrectly throw a NullReferenceException is certain cases.
The change also adds comments to the now-exposed methods and properties to address Visual Studio warnings. null checks have also been added for the parameters of the now public methods.

Please add <exception> tags to triple slash comments after <returns>.
<exception cref="ArgumentNullException"><paramref name="note"/> is null.</exception>

and
<exception cref="ArgumentNullException"><paramref name="noteEvent"/> is null.</exception>

And I'm will be ready to merge the PR :)

Also ideally we need to create corresponding unit tests but I can add it later after PR merge.

I've made the change you requested, and I've also added unit tests for the two files now. Let me know if you want some changes to the unit tests, as I wasn't 100% certain about the desired format for these.

@muiriswoulfe I've added minor comments on files with unit tests. Please resolve them for this PR and for second one.
By the way, nice work! I really appreciate your contribution!

@melanchall I have no problem making the updates to the unit test files, but I don't see the comments there. Could you verify you added them? Thanks.

Now comments should appear :)

Thanks @melanchall for the feedback. I believe you'll need to merge the PR as I don't have write access. Feel free to merge whenever you're ready.


Retrieve Latest Offset for Partition

Hi there! I'm attempting to migrate some code from Poseidon, and one feature I can't figure out how to replicate is fetching the latest offset for a partition.
Normally, I'd fetch the :ealiest message in the partition and use the highwater_mark_offset from the response, but I can't figure out a way to do this easily. It seems like that highwater mark is only accessible in the FetchedBatch response, and I can't figure out a way to get to it.
Both Client#fetch_messages and Client#each_message map to only the messages inside the batch, not the batch itself.
Consumer#each_batch seems to give me those batches back, but it seems crazy to go through that just to do a simple single fetch.
If all my above assumptions are correct, would you be open to a PR that adds Client#fetch_batch, which basically does fetch_messages but skips the flat_map?

What's your reason for wanting to fetch the latest offset?

I have some monitoring/dashboarding type use-cases where I need to understand the state of consumers vs the latest offsets on each partition.

Are your monitors separate from the consumers themselves? It's not documented yet, but kafka/datadog is an example of reporting e.g. consumer lag to a dashboard system, in this case Datadog. Events are published to the Active Support Notifications bus, and can easily be subscribed to.
This is of course all based on the assumption that you need this information in the same process that your consumer is running in.

Hmm, no this doesn't help me unfortunately. Happy to contribute a PR for this if you'd be willing to point me in the direction you'd like to go.

So basically you'd like to periodically poll the highwater mark offset for each partition in a set of topics?

Yep, exactly.

As far as I can see in the protocol, there's no Kafka API to give you this directly. The closest thing that doesn't involve fetching any messages would be to do an Offset Request with the special offset -1, meaning "offset of next message to be appended", if I recall correctly. You'd need to first get the complete list of partitions for the topic, which should be easy to do, though.
FetchOperation has some code for resolving offsets that could be extracted into a separate operation class. Let me have a look.

Once #230 has been merged I should be able to add an API for this.

@dstrelau can you take a look at #232?

I'm away from a computer until next week, so I can't actually run that branch to test it out, but it looks like it should work. Thanks!


Reading EDF/BDF files with n_records == -1

I've noticed in our lab that the headers of our BDF files often list the number of data records in the recording as -1. Currently, this breaks the read_raw_edf function, as it calculates the number of samples in the recording to be -1 times the number of samples per record.
Although we haven't yet determined what is causing our headers to list the number of records as -1, it is actually possible to determine the number of records in the recording without relying on the header. You can do this by determining the number of bytes in the file that are not part of the header (i.e. the number of data bytes), dividing that number by 3 (as there are 3 bytes per sample), dividing again by the number of channels (to get the number of samples per channel), then dividing by the number of samples per record. This gives the number of records in the recording, as it would be listed in the header. Note that this procedure assumes all channels have the same sampling rate.
Here is some code I added into my own copy of MNE's EDF/BDF reader to get around the issue. The custom code comes right after line 381 in the current version of edf.py on Git.
    (...)
    fid.read(32 * nchan).decode()  # reserved
    assert fid.tell() == header_nbytes

    # BEGIN CUSTOM CODE
    if n_records == -1:
        fid.seek(0,2)
        num_bytes = fid.tell()
        num_data_bytes = num_bytes - header_nbytes
        total_samples = num_data_bytes / 3 if subtype == '24BIT' else num_data_bytes / 2
        samps_per_chan = total_samples / nchan
        edf_info['n_records'] = n_records = samps_per_chan / n_samps[0]
    # END CUSTOM CODE

physical_ranges = physical_max - physical_min
cals = digital_max - digital_min
(...)

If anyone else is having this trouble, it may be helpful to add some sort of handling like this into the EDF/BDF reader.

Can you record some short (e.g, 1 or 2 sec) file to reproduce the problem?
Are you running custom software, or is this straight from the standard acquisition system?
I'm fine with adding this sort of change if you're on a stock setup. It would be good to identify why (acquisition version number maybe?) and put that as a comment in the code.
Actually, perhaps we should always do this calculation, and in your case use it as the true value, and in other cases, check against it and warn if there is a mismatch.

I'll try to get a quick recording for you tomorrow.
We're using the standard acquisition system (BioSemi's ActiView software). It's possible this is being caused by some sort of user error when our researchers save our data files. Even so, it's likely we're not the only ones with messed up n_record values.
I do think it could be good to do the calculation, warn if there is a mismatch, and use it as the default if the header says -1.

I tried recording a few short files, but all of them ended up with correct headers. Yet virtually all of our actual recordings end up with -1 as the number of records. I'm working with some of our research staff to figure out what's going on at our end with regards to this matter. I imagine we're probably not the only lab experiencing this issue.

UPDATE: I just figured it out. Our researchers have been pausing the recording, then exiting the program. The correct method is to stop the recording, then exit the program. I suspect the header item for number of records is not written until the recording is fully stopped, so all our recordings are ending up with the number of records never being written.
Regardless, here's a short recording that has a num_records == -1 error for your testing purposes, now that I know how to recreate the problem:
numrecs-1.bdf.zip

Are you up for making a PR to fix it?
First step would be to a PR to the mne-tools/mne-testing-data repo to add the files (probably in the EDF directory). Then you can make a PR to mne-python to add the functionality, with a test (which will really just be doing a roundtrip read of the file).

Sure, I'll look into this next week.

I have exactly the same issue with some files here in our lab, thanks @jpazdera for pointing out when this happens! Let me know if I can help in this fix.


Problems downloading from BBC Iplayer.

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2<IP_ADDRESS>. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
UK
Description
I'm using a VPN to access the BBC Iplayer. Videos from the site will work in Firefox on every single UK location on said VPN.
However, when I go to use yt-dlp to download a video, all but one UK location won't work and returns either output 1 or 2.
The one UK location that does work will give outputs 3 or 4 sometimes before working. When it does finally work and I request a format to download, I sometimes get output 5. Frustratingly, when the download does start, it goes extremely slow, usually somewhere between 20 to 200kb a second and often just hangs doing nothing for several seconds or minutes before moving again or just stops randomly and I need to run the command again for the download to continue where it left off.
If anyone knows what the problem here is or can help, it would be greatly appreciated.
I hope I've formatted everything properly. The preview function only lets me preview the description field for some reason.
Verbose log
Output 1
yt-dlp.exe -F https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard -vU
[debug] Command-line config: ['-F', 'https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2<IP_ADDRESS> [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-94421-gb3b7523feb, ffprobe N-94421-gb3b7523feb
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2<IP_ADDRESS>, Current version: 2<IP_ADDRESS>
yt-dlp is up to date (2<IP_ADDRESS>)
[debug] [bbc.co.uk] Extracting URL: https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard
[bbc.co.uk] b070p5yw: Downloading video page
[bbc.co.uk] b070p5yw: Downloading playlist JSON
Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 735, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3591, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 502: Bad Gateway

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 617, in extract
  File "yt_dlp\extractor\bbc.py", line 589, in _real_extract
  File "yt_dlp\extractor\bbc.py", line 460, in _download_playlist
  File "yt_dlp\extractor\common.py", line 997, in _download_json
  File "yt_dlp\extractor\common.py", line 976, in _download_json_handle
  File "yt_dlp\extractor\common.py", line 768, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 753, in _request_webpage
yt_dlp.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 502: Bad Gateway (caused by <HTTPError 502: 'Bad Gateway'>); please report this issue on  https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using  yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1389, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1459, in __extract_info
  File "yt_dlp\extractor\common.py", line 643, in extract
yt_dlp.utils.ExtractorError: [bbc.co.uk] b070p5yw: Unable to download JSON metadata: HTTP Error 502: Bad Gateway (caused by <HTTPError 502: 'Bad Gateway'>); please report this issue on  https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using  yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\__main__.py", line 19, in <module>
  File "yt_dlp\__init__.py", line 864, in main
  File "yt_dlp\__init__.py", line 854, in _real_main
  File "yt_dlp\YoutubeDL.py", line 3254, in download
  File "yt_dlp\YoutubeDL.py", line 3227, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1380, in extract_info
  File "yt_dlp\YoutubeDL.py", line 1407, in wrapper
  File "yt_dlp\utils.py", line 1088, in format_traceback
TypeError: format_exception() missing 2 required positional arguments: 'value' and 'tb'
[2128] Failed to execute script '__main__' due to unhandled exception!

Output 2
yt-dlp.exe -F https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard -vU
[debug] Command-line config: ['-F', 'https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2<IP_ADDRESS> [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-94421-gb3b7523feb, ffprobe N-94421-gb3b7523feb
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2<IP_ADDRESS>, Current version: 2<IP_ADDRESS>
yt-dlp is up to date (2<IP_ADDRESS>)
[debug] [bbc.co.uk] Extracting URL: https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard
[bbc.co.uk] b070p5yw: Downloading video page
Traceback (most recent call last):
  File "urllib\request.py", line 1354, in do_open
  File "http\client.py", line 1252, in request
  File "http\client.py", line 1298, in _send_request
  File "http\client.py", line 1247, in endheaders
  File "http\client.py", line 1007, in _send_output
  File "http\client.py", line 947, in send
  File "http\client.py", line 1421, in connect
  File "ssl.py", line 500, in wrap_socket
  File "ssl.py", line 1040, in _create
  File "ssl.py", line 1309, in do_handshake
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1131)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 735, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3591, in urlopen
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 542, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "yt_dlp\utils.py", line 1524, in https_open
  File "urllib\request.py", line 1357, in do_open
urllib.error.URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:1131)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 617, in extract
  File "yt_dlp\extractor\bbc.py", line 553, in _real_extract
  File "yt_dlp\extractor\common.py", line 900, in _download_webpage
  File "yt_dlp\extractor\common.py", line 768, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 753, in _request_webpage
yt_dlp.utils.ExtractorError: Unable to download webpage: <urlopen error EOF occurred in violation of protocol (_ssl.c:1131)> (caused by URLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1389, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1459, in __extract_info
  File "yt_dlp\extractor\common.py", line 643, in extract
yt_dlp.utils.ExtractorError: [bbc.co.uk] b070p5yw: Unable to download webpage: <urlopen error EOF occurred in violation of protocol (_ssl.c:1131)> (caused by URLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\__main__.py", line 19, in <module>
  File "yt_dlp\__init__.py", line 864, in main
  File "yt_dlp\__init__.py", line 854, in _real_main
  File "yt_dlp\YoutubeDL.py", line 3254, in download
  File "yt_dlp\YoutubeDL.py", line 3227, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1380, in extract_info
  File "yt_dlp\YoutubeDL.py", line 1407, in wrapper
  File "yt_dlp\utils.py", line 1088, in format_traceback
TypeError: format_exception() missing 2 required positional arguments: 'value' and 'tb'
[11808] Failed to execute script '__main__' due to unhandled exception!

Output 3
yt-dlp.exe -F https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard -vU
[debug] Command-line config: ['-F', 'https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2<IP_ADDRESS> [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-94421-gb3b7523feb, ffprobe N-94421-gb3b7523feb
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2<IP_ADDRESS>, Current version: 2<IP_ADDRESS>
yt-dlp is up to date (2<IP_ADDRESS>)
[debug] [bbc.co.uk] Extracting URL: https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard
[bbc.co.uk] b070p5yw: Downloading video page
[bbc.co.uk] b070p5yw: Downloading playlist JSON
Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 735, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3591, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 563, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 755, in http_error_302
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 542, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "yt_dlp\utils.py", line 1524, in https_open
  File "urllib\request.py", line 1358, in do_open
  File "http\client.py", line 1344, in getresponse
  File "http\client.py", line 307, in begin
  File "http\client.py", line 268, in _read_status
  File "socket.py", line 669, in readinto
  File "ssl.py", line 1241, in recv_into
  File "ssl.py", line 1099, in read
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 617, in extract
  File "yt_dlp\extractor\bbc.py", line 589, in _real_extract
  File "yt_dlp\extractor\bbc.py", line 460, in _download_playlist
  File "yt_dlp\extractor\common.py", line 997, in _download_json
  File "yt_dlp\extractor\common.py", line 976, in _download_json_handle
  File "yt_dlp\extractor\common.py", line 768, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 753, in _request_webpage
yt_dlp.utils.ExtractorError: Unable to download JSON metadata: The read operation timed out (caused by timeout('The read operation timed out')); please report this issue on  https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using  yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1389, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1459, in __extract_info
  File "yt_dlp\extractor\common.py", line 643, in extract
yt_dlp.utils.ExtractorError: [bbc.co.uk] b070p5yw: Unable to download JSON metadata: The read operation timed out (caused by timeout('The read operation timed out')); please report this issue on  https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using  yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\__main__.py", line 19, in <module>
  File "yt_dlp\__init__.py", line 864, in main
  File "yt_dlp\__init__.py", line 854, in _real_main
  File "yt_dlp\YoutubeDL.py", line 3254, in download
  File "yt_dlp\YoutubeDL.py", line 3227, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1380, in extract_info
  File "yt_dlp\YoutubeDL.py", line 1407, in wrapper
  File "yt_dlp\utils.py", line 1088, in format_traceback
TypeError: format_exception() missing 2 required positional arguments: 'value' and 'tb'
[18216] Failed to execute script '__main__' due to unhandled exception!

Output 4
yt-dlp.exe -F https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard -vU
[debug] Command-line config: ['-F', 'https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2<IP_ADDRESS> [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-94421-gb3b7523feb, ffprobe N-94421-gb3b7523feb
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2<IP_ADDRESS>, Current version: 2<IP_ADDRESS>
yt-dlp is up to date (2<IP_ADDRESS>)
[debug] [bbc.co.uk] Extracting URL: https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard
[bbc.co.uk] b070p5yw: Downloading video page
[bbc.co.uk] b070p5yw: Downloading playlist JSON
Traceback (most recent call last):
  File "urllib\request.py", line 1354, in do_open
  File "http\client.py", line 1252, in request
  File "http\client.py", line 1298, in _send_request
  File "http\client.py", line 1247, in endheaders
  File "http\client.py", line 1007, in _send_output
  File "http\client.py", line 947, in send
  File "http\client.py", line 1421, in connect
  File "ssl.py", line 500, in wrap_socket
  File "ssl.py", line 1040, in _create
  File "ssl.py", line 1309, in do_handshake
socket.timeout: _ssl.c:1114: The handshake operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 735, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3591, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 563, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 755, in http_error_302
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 542, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "yt_dlp\utils.py", line 1524, in https_open
  File "urllib\request.py", line 1357, in do_open
urllib.error.URLError: <urlopen error _ssl.c:1114: The handshake operation timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 617, in extract
  File "yt_dlp\extractor\bbc.py", line 589, in _real_extract
  File "yt_dlp\extractor\bbc.py", line 460, in _download_playlist
  File "yt_dlp\extractor\common.py", line 997, in _download_json
  File "yt_dlp\extractor\common.py", line 976, in _download_json_handle
  File "yt_dlp\extractor\common.py", line 768, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 753, in _request_webpage
yt_dlp.utils.ExtractorError: Unable to download JSON metadata: <urlopen error _ssl.c:1114: The handshake operation timed out> (caused by URLError(timeout('_ssl.c:1114: The handshake operation timed out')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1389, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1459, in __extract_info
  File "yt_dlp\extractor\common.py", line 643, in extract
yt_dlp.utils.ExtractorError: [bbc.co.uk] b070p5yw: Unable to download JSON metadata: <urlopen error _ssl.c:1114: The handshake operation timed out> (caused by URLError(timeout('_ssl.c:1114: The handshake operation timed out')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\__main__.py", line 19, in <module>
  File "yt_dlp\__init__.py", line 864, in main
  File "yt_dlp\__init__.py", line 854, in _real_main
  File "yt_dlp\YoutubeDL.py", line 3254, in download
  File "yt_dlp\YoutubeDL.py", line 3227, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1380, in extract_info
  File "yt_dlp\YoutubeDL.py", line 1407, in wrapper
  File "yt_dlp\utils.py", line 1088, in format_traceback
TypeError: format_exception() missing 2 required positional arguments: 'value' and 'tb'
[17840] Failed to execute script '__main__' due to unhandled exception!

Output 5
yt-dlp.exe -f mf_cloudfront-5510-1  https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard -vU
[debug] Command-line config: ['-f', 'mf_cloudfront-5510-1', 'https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2<IP_ADDRESS> [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-94421-gb3b7523feb, ffprobe N-94421-gb3b7523feb
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2<IP_ADDRESS>, Current version: 2<IP_ADDRESS>
yt-dlp is up to date (2<IP_ADDRESS>)
[debug] [bbc.co.uk] Extracting URL: https://www.bbc.co.uk/iplayer/episode/b070p5yw/earths-greatest-spectacles-2-svalbard
[bbc.co.uk] b070p5yw: Downloading video page
[bbc.co.uk] b070p5yw: Downloading playlist JSON
[bbc.co.uk] b070p5w2: Downloading media selection JSON
[bbc.co.uk] b070p5w2: Downloading MPD manifest
[bbc.co.uk] b070p5w2: Downloading MPD manifest
[bbc.co.uk] b070p5w2: Downloading MPD manifest
[bbc.co.uk] b070p5w2: Downloading MPD manifest
[bbc.co.uk] b070p5w2: Downloading MPD manifest
[bbc.co.uk] b070p5w2: Downloading MPD manifest
[bbc.co.uk] b070p5w2: Downloading m3u8 information
[bbc.co.uk] b070p5w2: Downloading m3u8 information
[bbc.co.uk] b070p5w2: Downloading m3u8 information
WARNING: [bbc.co.uk] Failed to download m3u8 information: timed out
[bbc.co.uk] b070p5w2: Downloading m3u8 information
[bbc.co.uk] b070p5w2: Downloading m3u8 information
WARNING: [bbc.co.uk] Failed to download m3u8 information: timed out
[bbc.co.uk] b070p5w2: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
ERROR: [bbc.co.uk] b070p5w2: Requested format is not available
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1389, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1473, in __extract_info
  File "yt_dlp\YoutubeDL.py", line 1529, in process_ie_result
  File "yt_dlp\YoutubeDL.py", line 2621, in process_video_result
yt_dlp.utils.ExtractorError: [bbc.co.uk] b070p5w2: Requested format is not available

Related: https://github.com/ytdl-org/youtube-dl/issues/30757


🎿 Skip vitis tests for ORT 1.16.1

Describe your changes
Skip vitis tests for ORT 1.16.1. Wait for VitisAI teams to fix this then to add the tests back.
Checklist before requesting a review

[ ] Add unit tests for this change.
[ ] Make sure all tests can pass.
[ ] Update documents if necessary.
[ ] Format your code by running pre-commit run --all-files
[ ] Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

do we need remove https://github.com/microsoft/Olive/blob/main/olive/passes/onnx/quant_pre_process.py

do we need remove https://github.com/microsoft/Olive/blob/main/olive/passes/onnx/quant_pre_process.py

Good catch. Removed!


Added 5 useful resources

Fixes #885 . Added 5 useful resources for web developers.

@A7abhilash resolve conflicts

@Ishaan28malik yeah sure... By mistak i created another pull request... Ill look into it!


⚠ Bofrost.fr has degraded performance

In f5bef30, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1140 ms

Resolved: Bofrost.fr performance has improved in e426184 after 6 minutes.


x509: certificate signed by unknown authority (cannot provide ca.crt)

Component
agent
Describe the bug
The bug
I want to build and push an OCI image (aka docker image) on my self-hosted registry. I'm running Woodpecker with Kubernetes backend.
I have this error : x509: certificate signed by unknown authority.
Explanation
The docker build is not aware of my ca.crt file.
How to fix the bug

Find a way to provide my ca.crt to the agent (easy to do, need a little hack of the helm chart)
Find a way for the agent to provide that ca.crt to plugins like plugin-docker-buildx (like WOODPECKER_BACKEND_DOCKER_VOLUMES but that does not work with Kubernetes backend)

What I did, what I tried...
I have read a lot of documentation on how to build+push my image. I have also read some parts of source code of Woodpecker Kubernetes backend. I have hacked the helm chart in order to put my ca.crt in a ConfigMap and mount it (with success) in either the agent pod or the dind pod (I will probably do a PR for this in repo woodpecker-ci/helm).
I have tried with WOODPECKER_BACKEND_DOCKER_VOLUMES variable but this is restricted to Docker backend. I found nothing similar for Kubernetes.
At last, it does not work whatever I do.
My favorite try (with and without dind enabled) :
  publish-woodpecker-branch:
    image: woodpeckerci/plugin-docker-buildx
    settings:
      registry: myregistry:5001
      logins:
        - registry: https://myregistry:5001/v2
      repo: myregistry:5001/awesome/tool
      platforms: linux/arm64
      tags:
        - ${CI_COMMIT_TAG=build}-${CI_COMMIT_SHA:0:8}-arm64

This should work but I fail at mounting the ConfigMap in the plugin-docker-buildx pod.
Another try (too complex I guess):
  publish-woodpecker-branch:
    image: docker:20.10.12-dind
    secrets:
      - registry_ca_crt
    commands:
      - mkdir -p "/etc/docker/myregistry:5001"
      - echo "$${REGISTRY_CA_CRT}" > "/etc/docker/myregistry:5001/ca.crt"
      - docker build -t "myregistry:5001/awesome/tool:$${CI_COMMIT_TAG=build}-$${CI_COMMIT_SHA:0:8}-arm64" .
      - docker push "myregistry:5001/awesome/tool:$${CI_COMMIT_TAG=build}-$${CI_COMMIT_SHA:0:8}-arm64"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

I get this error : Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
I have also tried with Podman but I'm getting problems to run in privileged mode.
System Info
### Version

{"source":"https://github.com/woodpecker-ci/woodpecker","version":"next-3033abc3"}

Helm values
Notice the hack with extraVolumes and extraVolumeMounts. registry-ca-crt is a ConfigMap that contains a key named ca.crt and the value is the CA certificate.
---
server:
  image:
    tag: next-3033abc3b4
  env:
    WOODPECKER_ADMIN: "<redacted>"
    WOODPECKER_HOST: "http://woodpecker.<redacted>"
    WOODPECKER_GITEA_URL: "http://gitea.<redacted>"
    WOODPECKER_GITEA: "true"
    WOODPECKER_GITEA_SKIP_VERIFY: "true"
    WOODPECKER_OPEN: "true"
    WOODPECKER_ORGS: "<redacted>"

  extraSecretNamesForEnvFrom:
    - woodpecker-gitea-client
    - woodpecker-gitea-secret
    - woodpecker-secret

  fullnameOverride: "woodpecker"

  persistentVolume:
    enabled: true
    size: 10Gi
    storageClass: "<redacted>"

agent:
  image:
    tag: next-3033abc3b4
  env:
    WOODPECKER_SERVER: "woodpecker.woodpecker.svc.cluster.local:9000"
    WOODPECKER_BACKEND: "kubernetes"
    WOODPECKER_BACKEND_K8S_STORAGE_CLASS: "<redacted (same as above)>"
#    WOODPECKER_BACKEND_DOCKER_VOLUMES: "/etc/docker/certs.d:/etc/docker/certs.d:ro"

  extraSecretNamesForEnvFrom:
    - woodpecker-secret

  extraVolumes:
    - name: registry-ca-crt
      configMap:
        name: registry-ca-crt

#  dind:
#    enabled: true
#    extraVolumeMounts:
#      - name: registry-ca-crt
#        mountPath: "/etc/docker/certs.d/myregistry:5001"

  fullnameOverride: "woodpecker-agent"

### Additional context

_No response_

### Validations

- [X] Read the [Contributing Guidelines](https://github.com/woodpecker-ci/woodpecker/blob/master/CONTRIBUTING.md).
- [X] Read the [docs](https://woodpecker-ci.org/docs/intro).
- [X] Check that there isn't [already an issue](https://github.com/woodpecker-ci/woodpecker/issues) that reports the same bug to avoid creating a duplicate.
- [X] Checked that the bug isn't fixed in the `next` version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]
- [X] Check that this is a concrete bug. For Q&A join our [Discord Chat Server](https://discord.gg/fcMQqSMXJy) or the [Matrix room](https://matrix.to/#/#woodpecker:matrix.org).

The PR mentionned in the bug description (extraVolumes and extraVolumeMounts) is https://github.com/woodpecker-ci/helm/pull/47.

Sorry, but I don't see how it's related to Woodpecker/. Especially as a bug.

Find a way to provide my ca.crt to the agent

For why?

Find a way for the agent to provide that ca.crt to plugins like plugin-docker-buildx

It's responsibility of used tool/container/plugin to expose to you some configuration parameters.
You can at least

Just store certificate in a repository
Download from external storage
Store in WP secrets as PEM and in step put it in a file

For Kubernetes backend I suggest to use kaniko. And this may help in your case.

Hello,
I consider this is a bug because https://woodpecker-ci.org/plugins/Docker Buildx says Use custom registries and includes a note on how to provide the CA certificate on runner server host. This seems to be possible with docker backend but not with kubernetes backend. Missing feature or bug as a regression ?
As far as I understand, this is not a bug of woodpecker-ci but a bug in woodpecker-ci plugin "docker-buildx" ? Sadly I cannot report it on https://github.com/woodpecker-ci/plugin-docker-buildx (no issue tracking system).
I just found https://codeberg.org/woodpecker-plugins/docker-buildx/issues : should I report it there ?
Thanks for the hint with Kaniko (I did not know about it).  If it works for me with Kaniko, I may report it somewhere in https://github.com/woodpecker-ci/woodpecker so you can add it to the documentation ?

Hello,
OK, it works with Kaniko.
Here is my .woodpecker.yml (extracts) :
workspace:
    base: /src
    path: /repo

steps:
  build:
    ...

  publish:
    image: gcr.io/kaniko-project/executor:debug
    secrets:
      - registry_ca_crt
    commands:
      - echo "$${REGISTRY_CA_CRT}" > "/kaniko/ssl/certs/additional-ca-cert-bundle.crt"
      - /kaniko/executor --destination "myregistry:5001/awesome/tool:$${CI_COMMIT_TAG=build}-$${CI_COMMIT_SHA:0:8}-arm64" --context "/src/repo" --dockerfile "/src/repo/Dockerfile"

It seems that --context is mandatory. I have tested with --dockerfile. No idea if it works without it.
Thanks for the help.
I have a solution for my problem, so I will not try to make it work with plugin/buildx. I still consider there is a bug somewhere (as I explained in my previous post) but this is no more a problem for me. I guess we can close this issue ?
Last point : it would be a nice feature if the contents of my secret registry_ca_crt above could be defined in a Kubernetes secret. Then I could use it either with volume or as an environment variable defined from my secret. It would be a nice feature because I would not have to define it as a Woodpecker secret in all my repos. Should I open an issue (tagged "feature") for that ?

Last point : it would be a nice feature if the contents of my secret registry_ca_crt above could be defined in a Kubernetes secret. Then I could use it either with volume or as an environment variable defined from my secret. It would be a nice feature because I would not have to define it as a Woodpecker secret in all my repos.

I looked at closed issues/PRs yesterday... Seems you can mount volume :)
You may try Global variables also.

I guess we can close this issue ?

You decide :) If you want to improve the docs, you can rename issue and use it as a base for following PR.

should I report it there?

Seems that kaniko, buildx and plugin-docker-buildx  can use custom certs. So there is no bug or missing functionality. You can just open PR with improved docs.

the contents of my secret registry_ca_crt above could be defined in a Kubernetes secret. Then I could use it as an environment variable defined from my secret.

If you want exactly that behavior, then it's not implemented as I know. Then it will be feature request (this issue or new one), I think.

looks more like an issue for our bildx plugin ?
can we close this here?

Hello,
I'll try to finish this issue with as much as infos as I can.
@zc-devs

I looked at closed issues/PRs yesterday... Seems you can mount volume :)

True, but with Kubernetes backend, it does not work with configMaps or secrets (yet ?).
I have not checked for an issue "wanted feature" for that but maybe it should be created ?
Again : for me it now works with kaniko so, with Agile methods in mind, I will not create an issue for what I don't need :)

You can try Global variables also.

Yes! Thanks for that tip. This is what I use now. I like the "Fill it once, use it everywhere" idea and it works.
@6543

looks more like an issue for our bildx plugin ?

Maybe. So I wrote it : https://codeberg.org/woodpecker-plugins/docker-buildx/issues/69
I have no reason to keep that issue open. I close it. Feel free to re-open it if I was too fast at closing !
And thanks for your help and tips !


Git push hangs with multiple repos (after reboot, or due to adding SSL)

Hi There,
We currently can't push changes to our system.
This is the result of pushing a new app:
Compressing objects: 100% (1560/1560), done.
error: RPC failed; HTTP 503 curl 22 The requested URL returned error: 503 Service Unavailable
fatal: The remote end hung up unexpectedly
Writing objects: 100% (1816/1816), 7.36 MiB | 12.00 KiB/s, done.
Total 1816 (delta 805), reused 998 (delta 225)
fatal: The remote end hung up unexpectedly
Everything up-to-date

Pushing a running app, stalls on the last stage:
-----> Preparing app for Rails asset pipeline
       Running: rake assets:precompile
       [Simple Form] Simple Form is not configured in the application and will use the default values. Use `rails generate simple_form:install` to generate the Simple Form configuration.
       I, [2017-02-28T13:01:49.599226 #336]  INFO -- : Copying assets to /tmp/build/app/public/assets/tinymce
       Asset precompilation completed (53.12s)
       Cleaning assets
       Running: rake assets:clean
       [Simple Form] Simple Form is not configured in the application and will use the default values. Use `rails generate simple_form:install` to generate the Simple Form configuration.

-----> Discovering process types
       Procfile declares types -> web, worker
       Default process types for Ruby -> rake, console, web, worker

Already upgraded the server with more memory, this didn't solve it.

It's a one server install

Flynn running v20170217.0

The apps are running without a problem.

Reboot didn't solve it.

Gitreceive is running. (no errors in its log)

Problem started when I added ssl to router (or due to reboot, not sure).
Anybody have an idea how to tackle this? :)

Please run sudo flynn-host collect-debug-info and post the logs here.

Hmmm, not working.
Is 142MB of logs not a bit much?
root@flynn-test:~# sudo flynn-host collect-debug-info
INFO[02-28|14:56:35] uploading logs and debug information to a private, anonymous gist
INFO[02-28|14:56:35] this may take a while depending on the size of your logs
INFO[02-28|14:56:35] getting flynn-host logs
INFO[02-28|14:56:35] getting sirenia metadata
INFO[02-28|14:56:35] getting scheduler state
INFO[02-28|14:56:35] getting job logs
INFO[02-28|14:59:03] getting system information
0 B / 142.05 MB [----------------------------------------------------------------------------------------------------------------------------------] 0.00 % INFO[02-28|14:59:05] creating anonymous gist
142.05 MB / 142.05 MB [=============================================================================================================] 100.00 % 17.73 MB/s 0sEROR[02-28|14:59:23] unexpected HTTP status: 405
142.05 MB / 142.05 MB [=============================================================================================================] 100.00 % 7.85 MB/s 18s
14:59:23.642047 host.go:164: unexpected HTTP status: 405

You can collect into a tarball with the --tarball option.

Ah nice!
Thanxs. Making a gist doesn't work with such a big file.
flynn-host-debug.tar.gz

I'm also facing similar issues
POST git-receive-pack (600 bytes)
error: RPC failed; result=22, HTTP code = 500
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
Everything up-to-date
Completed with errors, see above

Running on:
$ flynn version
v20170216.0

Hi thomas,
We just created a new system everything is working fine now. With us I think it was the SSL I added (wrongly). Or we ran out of memory.


require within location

I need to specify group/valid_user permission within different locations. Regarding this older article, it was possible to do it with auth_ldap_require.
With the actual version of the module, all those ldap_auth_* attributes described in the article above do not work anymore.
2018/05/02 09:38:53 [emerg] 4451#4451: unknown directive "auth_ldap_url" in /etc/nginx/conf.d/default.conf:10
Is there no more possibility to define required groups/valid_user inside a location and a global LDAP configuration?

is this fixed now? I'm haveing the same issue?


DOC: Migrate and update docs for building ITK Python wheels

Migrate documentation for building ITK Python distributions and build archives with ITKPythonPackage.
Original source: https://github.com/InsightSoftwareConsortium/ITKPythonPackage/blob/master/docs/Build_ITK_Python_packages.rst
Includes updates for ITK v5.3.0 and removed obsolete sdist instructions.

PR Checklist

[x] Updated API documentation (or API not changed)

Refer to the ITK Software Guide for
further development details if necessary.

Received offline approval to merge from @thewtex 👍


social partial where work has no image is incorrect

line 9 on bampfa_social.html.erb should be
<% unless doc_presenter.field_value('blob_ss').blank? %>
instead of
<% unless doc_presenter.field_value('blob_ss').nil? %>

@jblowe lol I forgot I opened this a few months ago


Export a document to Latex including citations from Mendeley

How hard can it be?

I usually work with plain latex code in the google doc. Insertion of \cite{lastname2012} would be more than sufficient for me even without the link and the extra functionality. Then I copy paste the entire document to a different place to render the latex, and I can export my entire Mendeley as a bib file.


Default WSO2 IS 5.9.0 installation errors on Admin console

I have a default installation of WSO2IS 5.9.0 and I  have noticed GET errors in the pages from the administration console when looking at the user profile:

The URL contains an extra carbon. The behaviour is also on other pages.
Thanks

Already a similar issue has been reported with https://github.com/wso2/product-is/issues/5346


Add test cases

[x] typically use cases
[x] failover
[x] resharding
[ ] node down
[x] multiple replicas
[x] connection drivers
[x] SSL
[ ] benchmarks

https://ruby-doc.org/stdlib-3.0.1/libdoc/minitest/rdoc/Minitest/Benchmark.html
https://github.com/ruby/benchmark

something was wrong at node down test case


Fix failed systemtests testAddressStatus

Type of change

Bugfix

Description
Fix failed systemtests testAddressStatus

Checklist

[ ] Update/write design documentation in ./documentation/design
[ ] Write tests and make sure they pass
[ ] Update documentation
[ ] Check RBAC rights for Kubernetes / OpenShift roles
[ ] Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
[ ] Reference relevant issue(s) and close them after merging
[ ] Update CHANGELOG.md

@enmasse-ci run tests testcase=*ConsoleTest

:x: Test Summary :x:
TEST_PROFILE: pr
TEST_CASE: *ConsoleTest
TOTAL: 82
PASS: 75
FAIL: 1
SKIP: 6
BUILD_NUMBER: 617
:heavy_exclamation_mark: Test Failures :heavy_exclamation_mark:

testFilterLinksByName in io.enmasse.systemtest.shared.standard.web.FirefoxConsoleTest


Please add Node Tool support for using nodejs nightly builds

Environment

Server - Azure Pipelines
Agent - Hosted VS2017

Issue Description
Travis CI provides the ability to create builds using the nightly build from Nodejs.  In the screenshot below (from the ExpressJS repository), they are specifying multiple versions of NodeJS, and specifying that certain versions use the nightly build.
The Node Tool should provide this support as well.  It should be provided in such as way that, if I am using a matrix, then, for each node version in the matrix, I can specify whether or not that version should use the nightly build.

I'm looking into this, a couple of clarifying questions/thoughts though:

What does it actually mean to get nightly builds for node v9 or v10? From here, new nightly builds are only being released for v12 since its the only one being actively developed, so I'm not sure what that would even mean.
To get some clarification on (1), I tried to look at the travis builds of the express repo you're referencing. It doesn't actually look like they're using nightly builds, regardless of what is in their YAML. Look at the nvm install steps here and here - they're pulling from https://nodejs.org/dist, not https://nodejs.org/download/nightly. Maybe because the env variable they're using is deprecated?
Why is this actually a useful feature? I don't really see a use case where you need this (though that's not to say there isn't one, I'm just struggling to think of one).

Overall, I'd be open to adding this, but I think we need a compelling reason beyond just maintaining exact feature parity with Travis, especially if they're not actually doing anything with this.

I'm not sure I'll be much help answering these questions, unfortunately.  If, in examining the repos, it doesn't look like they are using the nightly builds, then it could be because they didn't update the build file to use the new variable.  But if no one is complaining, then it may be that it doesn't matter as much.  My biggest argument for this feature was for parity, and so I could create an Azure Pipeline build that matched what they have.  I can always make the argument that the currently nightly feature is no longer being used.  So I'm not sure I have the silver bullet argument that says this feature MUST be added. However, it would be something nice to have, and would give parity with Travis.

Ok, I don't think that it makes sense to add this right now, then. Since we don't have a use case I think I'd rather avoid having 1 more thing to maintain/document. With that being said, I'd definitely be open to adding it in the future given a use case.

Is their a specific customer that's driving this need?  If so then that's a good use case ...
@mickeygousset ?

Specific customer at this point, no.  The test will be when I finish creating the Azure DevOps pipeline for the ExpressJS repository and submit it to them.  If they reject it because that functionality isn't in the build, then it definitely becomes an issue.
What I will do is submit it without it, and see what happens. I'll also point out that their current build isn't really using it either.


Submission flow result screens routing (EXPOSUREAPP-6034)

Adds the necessary routing to display all the test results and test cards on the home screen.
Updating the specific test result screens with the right info depending on what type of test it is, will be done in a different PR.

Quick finding:
1- Back button in Qr-Code scan screen is not working
2- Text in Negative test card is malformed:

Quick finding:
1- Back button in Qr-Code scan screen is not working
2- Text in Negative test card is malformed:
3- Positive test the same:

Yes, I'll address them in another PR.

Submission flow:
https://user-images.githubusercontent.com/25054729/115434862-7833d400-a209-11eb-80b8-f50a97c7b4f4.mp4

Submission flow:
device-2021-04-20-184752.mp4

I think the cards are still not wired up correctly, but afaik it's not a problem with this PR.
CC: @AlexanderAlferov & @d4rken

The missing registration dates on the cards should be fixed after #2882.

Submission flow:
device-2021-04-20-184752.mp4

That is also fixed in #2882 AFAICT, the "SubmissionDone" state was not wired up correctly.


New FFMPEG Pipeline: CUVID doesn't support 10-bit H.264

As the title says, h264_cuvid doesn't support decoding 10-bit H.264 video. This results in a "This channel is experiencing technical difficulties" screen for the duration of the scheduled item. When enabling FFMPEG logging this is at the end of the log for the item, informing of the lack of support:
[h264_cuvid @ 0x3717c80] Bit depth 10 is not supported.
[vist#0:0/h264 @ 0x3457cc0] [dec:h264_cuvid @ 0x3717200] Error while opening decoder: Invalid argument
[fc#0 @ 0x322c340] Error binding an input stream to complex filtergraph input scale_cuda:default.
Error binding filtergraph inputs/outputs: Invalid argument

10-bit H.264 works fine under the existing software decode -> hardware encode pipeline, so the solution would be to have the new pipeline fall back to software decoding for unsupported inputs

Can you provide the full ffmpeg command for the failing stream? Curious because we already attempt to use software decoding for h264>=10 bit: https://github.com/chrisbenincasa/tunarr/blob/dev/server/src/ffmpeg/builder/capabilities/NvidiaHardwareCapabilities.ts#L43 and here https://github.com/chrisbenincasa/tunarr/blob/dev/server/src/ffmpeg/builder/capabilities/NvidiaHardwareCapabilities.ts#L43
So, I’m wondering whether our detection of the incoming pixel format is off. The full ffmpeg may help determine that. I should also add some logging for the beginning stream state before passing it off to the pipeline builder.

From the ffmpeg-report-test log:
/usr/bin/ffmpeg -threads 4 -nostdin -hide_banner -nostats -loglevel error -fflags +genpts+discardcorrupt+igndts -hwaccel cuda -ss 542.655s -c:v h264_cuvid -hwaccel_output_format cuda -readrate 1.0 -readrate_initial_burst 45 -i "/MediaLinks/TV Shows/Sailor Moon (1992) {imdb-tt0103369}/Season 01/Sailor Moon (1992) - S01E01 - 001 - The Crybaby Usagis Beautiful Transformation [DVD v2][10bit][x264][FLAC 2.0][JA+EN].mkv" -filter_complex "[0:0]scale_cuda=1920:1080:format=nv12:force_original_aspect_ratio=decrease,setsar=1,hwdownload,format=nv12,pad=1920:1080:-1:-1:color=black,hwupload_cuda[v];[0:1]aresample=async=1:first_pts=0,apad=whole_dur=908171ms[a];[v]scale_cuda=format=yuv420p[vpf]" -map "[vpf]" -map "[a]" -muxdelay 0 -muxpreload 0 -flags cgop -movflags +faststart -t 908171ms -video_track_timescale 90000 -b:v 10000k -maxrate:v 10000k -bufsize:v 1000k -c:v h264_nvenc -c:a aac -b:a 192k -maxrate:a 192k -bufsize:a 50k -ar 48k -g 96 -keyint_min 96 -force_key_frames "expr:gte(t,n_forced*4)" -f hls -hls_time 4 -hls_list_size 0 -segment_list_flags +live -hls_segment_type mpegts -hls_segment_filename "streams/stream_93d6d42d-0409-4c08-9ba9-0b37a4728af9/data%06d.ts" -hls_base_url /stream/channels/93d6d42d-0409-4c08-9ba9-0b37a4728af9/hls/ -hls_flags program_date_time+append_list+omit_endlist+independent_segments streams/stream_93d6d42d-0409-4c08-9ba9-0b37a4728af9/stream.m3u8
also the video stream that FFMPEG sees in the file:
Stream #0:0(jpn): Video: h264, yuv420p10le(progressive), 768x576, SAR 1:1 DAR 4:3, 24 fps, 24 tbr, 1k tbn (default)

Nice, thanks. Judging from the other bug you opened, I'm gonna assume your media source is Plex. If so, I think the issue here is that Plex does not give us the pixel format (or bit depth) of the video stream in its metadata, which is what Tunarr uses to derive info. I think the fix here is to actually extract that info (and possibly all of the needed info) using ffprobe instead.


Don't load fileset records from solr for collections.

Speeds collection manifests up considerably.

Coverage remained the same at 100.0% when pulling 9ac1d76ad053572141c2d022da30c965d51d714e on profile_iiif into 53fa3e92eda1f43883a5ac8b8ac673e4080cf055 on master.


Iterating over nonzero elements of a GBMatrix sparse matrix

Is there any convenient accessor through Julia to iterate over the nonzero elements of a GBMatrix, with the full triplet information (i,j,v) instead of v alone? Similar to the SparseArrayspackage, i.e.,
rows = rowvals(A)
vals = nonzeros(A)
m, n = size(A)
for j = 1:n
   for i in nzrange(A, j)
      row = rows[i]
      val = vals[i]
      # ...
   end
end

This shall be useful when the (i,j) information is needed with the mapor apply functionality.
Thank you for integrating the GraphBLAS into the Julia ecosystem!

There will be in the next release likely Monday next week. It won't be an exported part of the interface yet though since it may change between releases.
Next week you'll be able to iterate by nz values by row (if GBMatrix is by-row), and by column (if GBMatrix is by-col).
A fuller interface will likely come with the revival of ArrayIteration.jl in the next month or so.

For map the missing functionality is user defined SelectOps. That will also be in the next release, I just overlooked them.

@Wimmerer Thank you! Was the functionality you mentioned included in v0.7.2? I could not find it in the docs.

Sorry this is not ready yet, I'll try to get it out this week.


Can't debug flask application the second time without reloading window

Environment data

VS Code version: 1.38.1
Extension version (available under the Extensions sidebar): 2019.9.34911
OS and version: Windows 10
Python version (& distribution if applicable, e.g. Anaconda): 3.7.3
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv
Relevant/affected Python packages and their versions: ptvsd 0.2.0, flask 1.1.1

Actual behaviour
Debugging a Flask application works on the first time (no launch.json), but on the second it times out and it fails. Not sure if this issue is on the extension side or with ptvsd
Steps to reproduce:

Clone https://github.com/luabud/wave
Create a venv and install requirements
Open debug view and hit play
Select Flask and add hello_app.webapp.py as path to app
Watch the debugger do its wonderful work 😎
Stop the debugger
Hit play again, select Flask and add hello_app.webapp.py as path to app once more
Watch the 🔥 🔥 🔥

Logs
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
User belongs to experiment group 'AlwaysDisplayTestExplorer - control'
User belongs to experiment group 'ShowPlayIcon - control'
User belongs to experiment group 'AA_testing - control'
> conda --version
> pyenv root
> python3.7 -c "import sys;print(sys.executable)"
> python3.6 -c "import sys;print(sys.executable)"
> python3 -c "import sys;print(sys.executable)"
> python2 -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> py -3.7 -c "import sys;print(sys.executable)"
> py -3.6 -c "import sys;print(sys.executable)"
> py -3 -c "import sys;print(sys.executable)"
> py -2 -c "import sys;print(sys.executable)"
> c:\Users\luabud\Apps\hello_flask\env\Scripts\python.exe -c "import sys;print(sys.executable)"
> ~\AppData\Local\Microsoft\WindowsApps\python.exe c:\Users\luabud\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python.exe c:\Users\luabud\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python3.7.exe c:\Users\luabud\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python3.7.exe c:\Users\luabud\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python3.exe c:\Users\luabud\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\interpreterInfo.py
> ~\AppData\Local\Microsoft\WindowsApps\python3.exe c:\Users\luabud\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\interpreterInfo.py
Starting Microsoft Python language server.
> conda info --json
> conda env list
> conda info --json

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

Full output:
flask_debugbug_output.txt

cc @karthiknadig

This should be address by the new python debug adapter. It takes care of process lifetime.

Closing as I can't repro anymore. Thanks for the triage @kimadeline!