restudio does not compile in Delphi

restudio.dpr doesn't compile in Delphi (I tried XE8, but I think it won't compile in any other version either):

unit 'interfaces' not found
subdirectories persistence and StopWatch do not exist in restudio directory (they are in the main directory, so the included unit file names should be ..\StopWatch\StopWatch.pas, ..\persistence\ansoRTTIHook.pas, ..\persistence\ansoStings.pas and ..\persistence\tynlist.pas
regexpr is in ..\src\regexpr
what about regexpr_compilers.inc ?

If I fix this, I get the following error in tynlist, line 341:
Left side cannot be assigned to.
initialization
tynItemClasses := TObjectList.Create;
That means the option "Assignable typed constants" must be enabled.
Unit stopwatch also doesn't compile as it has Windows listed in the interface and the implementation uses clause.
Which Delphi Version should it compile with?

It was ok very log time ago
And at the moment I cannot install Delphi on my PC (this is blocked by my company rules) so I cannot help you :((
Sorry :(
If you find the solution please share it with others - send PR, please.

i got report about some my component bug on Delphi. and i cannot test it too.

I didn't understand. The "restudio.dpr" file is in this project? I've searched but I didn't find.

@andgineer Since REStudio is absent in this repo - please code the issue

We removed the code
https://github.com/andgineer/TRegExpr/issues/201
It was outdated and anyway there are a lot of way better regex debuggers


Warning: Ruh roh! Could not save gist file, configuration not saved.

Warning: Ruh roh! Could not save gist file, configuration not saved.

After clicking Compile and Download at https://getbootstrap.com/docs/3.3/customize/
Tested on Windows 10 (1803) in Firefox 60 and Chrome 66.

Regrettably v3 isn't supported much anymore. In all likelihood we'll take down the customizer in the future to negate these issues. Sorry for the hiccups! Your best bet is a third party customizer or your own build system.


Failed to start docker container "apollo_dev" based on image: registry.docker-cn.com/apolloauto/apollo:dev-x86_64-20180530_1312

How to solve"Failed to start docker container "apollo_dev" based on image: registry.docker-cn.com/apolloauto/apollo:dev-x86_64-20180530_1312"

Can you give more detail information?

Closing this issue as there has been no communication in a while. Feel free to reopen it if you have any additional questions.


Uppy dataset upload

Caveat with this PR is the project now depends entirely on S3. Optional branch feat/dev-local-s3 contains a demo of a local S3 server, but it needs some work on the configuration: https://github.com/metaspace2020/metaspace/compare/feat/uppy-dataset-upload...feat/dev-local-s3

Think I'm missing the config to test the full flow locally, I'd to test it on staging before merging.

Think I'm missing the config to test the full flow locally, I'd to test it on staging before merging.

@LachlanStuart I think the code has changed quite significantly, so might be best to review the update if you don't mind. It's also deployed on Staging 👍

@LachlanStuart I think the code has changed quite significantly, so might be best to review the update if you don't mind. It's also deployed on Staging 👍


Noticed security issue async

Saw the nested alert with the vulnerability to async. https://github.com/advisories/GHSA-fwr7-v2mv-hh25
Let me know if any one else is looking to potentially update before spelunking

Looks like the relevant issue may be with ejs and potentially unneeded dep https://github.com/mde/ejs/issues/659

@jondot let me know if this is a concern. thanks for the time!

Closed as will not fix


Custom loaders don't take effect on IPFSInteract.load()

In case a custom loader is passed to IpfsInteract.load() I would expect that it would be used instead of the default loader set on init of IpfsInteract. Currently, the custom loader is only used for load_single_object, but not for load.

The load_single_object is utilized inside the load and therefore, the custom loader is also used from the load method:
https://github.com/valory-xyz/open-autonomy/blob/8abe8fd38b753d5ac32a27a2e43da08e46db875e/packages/valory/skills/abstract_round_abci/io_/load.py#L61

At the time of the creation of this issue, it was not behaving as you are explaining it here. I recall that I had to override init() on the behaviour for it to take effect.

Can you recall an example, ideally one for which we can find a specific commit?
In any case, I can try to create a test for that.


Fix issue #661 - buggy sidebar in fluid container

Having an absolute positioned sidebar means content overflows into footers in some cases.
I've fixed this by having the sidebar float left and use a negative margin to put it into place. This also solves the issue of the sidebar affecting the height of content rows.

Took a slightly alternate approach, but using the same technique. Here's the CSS and HTML.
// Fluid layouts (left aligned, with sidebar, min- & max-width content)
.fluid-container {
  position: relative;
  min-width: 940px;
  padding-left: 20px;
  padding-right: 20px;
  .clearfix();
}

// Sidebars (left and right options)
.fluid-sidebar {
  width: 220px;
  margin: 0 20px 18px;
}
.sidebar-left   { padding-left: 260px; }
.sidebar-right  { padding-right: 260px; }

// Float the sidebars accordingly
.sidebar-left .fluid-sidebar   { float: left; margin-left: -240px; }
.sidebar-right .fluid-sidebar  { float: right; margin-right: -240px; }

// The main content area
.fluid-content {
  float: left;
  width: 100%;
}

    ...

    ...

Again, slightly different but with the same outcome. Haven't cross browser tested yet, but this is now in 2.0-wip.


schemaless external function in ekuiper

I am running ekuiper with version [lfedge/ekuiper:v1.14.0-alpha.2] as a container in edgex running in docker environment. Ubuntu 22.04.4 LTS. I want to test schemaless external function. I created json file which mentions the function name. I have written REST API function in go language golang file. When i call the http call like "http://localhost:59722/tsschemaless" it is not calling from golang file.
I tested this same golang file in ubuntu shell. When i call the http call like http://localhost:8080/ getting reply from the file.
tsschemaless.json
tsschemaless.txt

@nbenhur Could you describe what's you issue you meet

I am running ekuiper with version [lfedge/ekuiper:v1.14.0-alpha.2] as a container in edgex running in docker environment. Ubuntu 22.04.4 LTS. I want to test schemaless external function. I am following link https://wiki.lfedge.org/display/EKUIPER/Schemaless+HTTP+Design to create schemaless external function service, stream and rule. I successfully created the service using json file and i can view the service by giving the command "GET http://localhost:59720/services". I successfully created stream in edgex ui.
When i try to create rule using sql command
{
"id": "tsschemalessrule-1",
"sql": "SELECT tsschemaless("POST", "", *) from schemalessStream",
"actions": [{
"memory": {
"topic": "tsschemalessTopic"
}
}]
}
This is the error i am getting for the above rule
error : invalid character 'P' after object key:value pair..
I again edited the sql command as below
"sql": "SELECT tsschemaless(POST, "", *) from schemalessStream",
This is the error i am getting for the above rule
error : invalid character '"' after object key:value pair..
How to solve this sql error.
I am attaching the json file for schemaless service and screenshot
tsschemaless.json

{
"id": "tsschemalessrule-1",
"sql": "SELECT tsschemaless(post, "/tsschemaless", *) from schemalessStream",
"actions": [{
"memory": {
"topic": "tsschemalessTopic"
}
}]
}
error : invalid character '/' after object key:value pair..
I tried with the other way by following link "https://ekuiper.org/docs/en/latest/extension/external/external_func.html", and here also getting error from sql command.

I found the solution for the above question. By using single quote it worked as below.
{
"id": "tsschemalessrule-1",
"sql": "SELECT tsschemaless(POST,'',*) from schemalessStream",
"actions": [{
"memory": {
"topic": "tsschemalessTopic"
}
}]
}
{
"id": "tsschemalessrules-1",
"sql": "SELECT tsschemaless(POST,'/tsschemaless',*) from schemalessStream",
"actions": [{
"memory": {
"topic": "tsschemalessTopic"
}
}]
}


Implement Clone for possible matchers

To simply reuse the same long matcher expression across multiple checks.

I run into temporary lifetime issue by taking the matcher by reference. However, in #356, I just made all &impl Matcher implement Matcher which seems to fix both issues.

@bjacotg Implementing Clone might still be useful for other reasons, but matching on &Matcher works just as well for me!


Add ASIO/WASAPI drivers to Reduce Audio Latency

Opening an issue for discussion since this is something that should be included in the official distribution.
osu! uses default BASS procedures, which use DirectSound on Windows, causing large latencies due to high latency software mixing in Windows Vista and Windows 7 (as I've tested). I get 70ms latency on average using 240FPS on both my Windows 7 64-bit PC and Windows Vista 32-bit laptop. This may be less of a problem on Linux since BASS defaults to the ALSA driver, which is a low-level & low-latency driver, but I have not tested it fully.
The BASS API has extensions for ASIO and WASAPI support under the same licenses.
We can effectively (hopefully) reduce the latency to less than 4ms on ASIO, and a worse case on WASAPI (Exclusive).
BASS does not unify their main API with ASIO and WASAPI APIs for some reason, so some changes and additions to audio code will be needed. These APIs are very easy to work with, and pretty much the same as using their main API once you've got them initialised. We have to set up a BASS MIXER and load sounds and music as BASS STREAMS, instead of SAMPLES. The STREAMS are connected to the MIXER and the MIXER feeds audio data to the ASIO/WASAPI event procedures. BASS CHANNEL procedures can be used with streams just like samples.
We will also need to add options for sound drivers in the options menu. It will simply list available drivers depending on your platform. A simple drop-down box can be created after detecting available drivers. Options could be "Default", "ASIO", "WASAPI (Shared)", and "WASAPI (Exclusive)". A note should be added to WASAPI (Exclusive) and ASIO since they take exclusive access of an audio device, so only osu! can be playing sound from that device.
Linux does not use ASIO, so this will not affect Linux. WASAPI is Windows-exclusive, but does not achieve lower latencies compared to ASIO.
This will also add some dynamic-link dependencies and the use of add-ons from ManagedBass. BASSMIX, BASSASIO, and BASSWASAPI will be used.
I can work on the code for ASIO & WASAPI support, but some minor design changes may be required.

The latency you describe is not inherent to windows audio mixing. I've had many sound cards, and for most of them there was no perceptible delay, even through DirectSound and an active audio mixer. Therefore, the fault can't lie with windows' audio pipeline. Seems to me like the fault lies with the hardware / driver and not DirectSound.
Of course, if you have a sound card where ASIO / WASAPI actually makes a difference (on all the ones I had, if there was a delay, then it persisted with ASIO / WASAPI; believe me, I tried hard to get rid of it, as I also find it annoying), then feel free to implement it. Design changes can be discussed as they come up.

I felt my conclusion is fact because I've worked on and compared low latency code myself and there was significant improvement when using ASIO or WASAPI. I conducted my tests by recording with a microphone and a click input and measured the space between both sounds. Sure enough there was 2ms latency using ASIO and a large 70ms when using DirectSound. The delay is easily noticeable if you make taps after hearing the instrument and getting that feedback.

I'm not claiming your setup doesn't exhibit delay. As I said, feel free to implement ASIO / WASAPI. I'm just saying, that it's most likely a specific problem with your setup (I had similar issues with my old Xonar Essence STX and a bunch of onboard sound cards), and not that of the windows audio mixer. I've got multiple working setups that prove that the latency with DirectSound and the audio mixer turned on can be (and should be) close to none.

Hi, I'm a chronic sufferer from audio latency. While it was possible for me to fix my setup, it was EXTREMELY difficult to do so, having to try out many different drivers and rebooting many times, and sometimes the driver only works under specific circumstances (if I unplug my external monitor and use the laptop monitor, it suddenly stops working!) and sometimes the driver will decide to uninstall itself (I don't have automatic updates enabled, I don't know what's uninstalling it...)
If ASIO / WASAPI is going to remove all this pain, it doesn't actually matter if it's possible to fix my setup or not, because it would actually fix my setup, and save a decent number of players some grief.
@posesix It'd be great if you could submit a PR for this, else I don't think much can be done ^^;

+1 To this. I've been through 6 months of reinstalls, multiple sound cards, custom drivers, and more. My headphones are connected directly to the sound card so there's definitely no hardware problem.  Prior to this I had some drivers which I somehow broke the APO on which had no latency. Obviously I haven't been able to replicate this. My hardware supports ASIO. Please please please add support.

In many cases ASIO is not implemented perfectly and can introduce issues (I've experienced this personally in the past). It should not be considered a solves-all solution but instead a fallback for users with problems.
Also please note we do not need +1 comments. We have no issue with ASIO support being added (although do note it is an extra BASS product with a separate license requirement) so feel free to PR support for this.

Audio system is switched to bassmix now, can we raise the priority of this issue?

You are welcome to give implementing this a shot at any point, if that's what you're asking!


Update tapir-akka-http-server, ... to 0.17.9

Updates

com.softwaremill.sttp.tapir:tapir-akka-http-server
com.softwaremill.sttp.tapir:tapir-enumeratum
com.softwaremill.sttp.tapir:tapir-json-circe
com.softwaremill.sttp.tapir:tapir-openapi-circe-yaml
com.softwaremill.sttp.tapir:tapir-openapi-docs
com.softwaremill.sttp.tapir:tapir-swagger-ui-akka-http

from 0.17.0-M8 to 0.17.9.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.softwaremill.sttp.tapir" } ]

labels: library-update, semver-pre-release

Superseded by #80.


nixos/wasabibackend: init <IP_ADDRESS>

Motivation for this change
Adding the backend service for wasabiwallet, a coinjoin thingy.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

@GrahamcOfBorg test wasabiwallet

@GrahamcOfBorg test wasabibackend

@GrahamcOfBorg test wasabibackend

Was excited to see this & didn't notice that it's a draft at first. Left some suggestions anyway.

Yes, still some work to finish on this one before it is ready :)

@GrahamcOfBorg test wasabibackend

@GrahamcOfBorg build wasabibackend

@GrahamcOfBorg test wasabibackend

@GrahamcOfBorg test wasabibackend

@GrahamcOfBorg build wasabibackend

Okay, tested and it is working fine. It is ready for review now.

@GrahamcOfBorg test wasabibackend
@GrahamcOfBorg build wasabibackend


marginalia:0.2.0

I am submitting

[X] a new package
[ ] an update for a package

Description: This Typst package provides functions to add (optionally numbered) margin notes to your documents.
This package is based on the drafting package but specializes on numbered margin notes, as opposed to drafting margin notes. As such, some functions have been removed and the defaults updated. Adds features for numbering the notes and controlling the numbering style, as well as the note text style.

I have read and followed the submission guidelines and, in particular, I

[X] selected a name in conformance with the guidelines
[X] added a typst.toml file with all required keys
[X] added a README.md with documentation for my package
[X] have chosen a license and added a LICENSE file or linked one in my README.md
[X] ensured that my submission does not infringe upon the rights of a third party
[X] tested my package locally on my system and it worked
[X] named this PR as name:version of the submitted package
[X] agree that my package will not be removed without good reason

This would probably be better as a pull request to the original package surely, rather than copying over their code?

This would probably be better as a pull request to the original package surely, rather than copying over their code?

I initially did that, but soon realized that the intent of the original package and of my needs for this package were different. I had to change the default settings because my purpose was not to make draft notes but rather to make permanent margin notes with numbers. This package has some functionality added.
My thoughts were that they could coexist as two different packages.

In the end it's up to you, but did you discuss it with the author of the other package? Maybe they'd like your extension to the package.

I'm not sure, but I think what might be a bit confusing with the PR right now is that it says it's based on the margin-notes package while (as per the README.md), this seems to be a typo and it is based on the drafting package.
And this, then, makes the distinction (and reason for the distinction) much clearer (if the source package was scoped as margin-notes, I would agree with @JamesxX, but with the name drafting, I agree that the scope of this package feels different and I wouldn't search for this feature in a package called drafting).

I'm not sure, but I think what might be a bit confusing with the PR right now is that it says it's based on the margin-notes package while (as per the README.md), this seems to be a typo and it is based on the drafting package.
And this, then, makes the distinction (and reason for the distinction) much clearer (if the source package was scoped as margin-notes, I would agree with @JamesxX, but with the name drafting, I agree that the scope of this package feels different and I wouldn't search for this feature in a package called drafting).

Whoops! My mistake! Yes, it is based on the drafting package. The default style of that package is one for commenting drafts (hence the name of that package). It does not include numbered margin notes and has some other features that are not needed in a margin notes package.

In the end it's up to you, but did you discuss it with the author of the other package? Maybe they'd like your extension to the package.

I could discuss it with him, though I am not sure he would agree to changing the defaults for the margin-note function...

Thank you for your package submission. We have to ask you to change the name of this package since it is merely descriptive. Once the name is changed, the merging process can continue!

👋 Author of the typst-drafting package here, I wasn't tagged so I didn't see this until now. I think there are a few ways that, if you want your own package, you can use #import @preview/drafting:XXX and build on top of it rather than copy-pasting code -- if you're interested, I'm happy to work with you if changes are needed in typst-drafting.
My thought is, if bugs are fixed here or in my package, it will be much easier to reap the benefits if they rely on a common library. I'm even happy to split mine into two packages if the community prefers. It also came up in discord (https://discord.com/channels/1054443721975922748/1148077628960542771/1148188972489654292) but I initially declined since the code base is quite small so far.
Looking at the examples, I think the feature set is a great addition and I welcome a PR if you would consider adding it

I wouldn't search for this feature in a package called drafting

@pklaschka That's a fair point as well -- I added it as a keyword and part of the description so that if you use the text filter on https://typst.app/docs/packages, it does show up

@jmml97 based on @ntjess's comments, how should we proceed here?

@jmml97 based on @ntjess's comments, how should we proceed here?

Hello! Sorry, it's been a few busy weeks. I will look at how the features could be integrated into drafting. My main concerns about it are two:

That the margin-note command defaults include the line and box decoration, making it more suitable for the drafting focus of that package rather than a more general margin note purpose.
That because of that drafting focus these functions to provide margin notes are not easily found.

Should this be kept open or closed?

Should this be kept open or closed?

I've been very busy recently and have not been able to work on this. Maybe it is best to close it and then reopen it once I have decided how to do the changes. Thanks for your time.


Border prop does not work for BlockStack component from checkout-ui

Please list the package(s) involved in the issue, and include the version you are using
"@shopify/checkout-ui-extensions-react": "^0.18.1"
Describe the bug
The following code doesn't render a border around the BlockStack component:
<BlockStack spacing="loose" border="base" borderWidth="base">
  {children}
</BlockStack>

Steps to reproduce the behavior:
Render a BlockStack with border property.
Expected behavior
A border should be rendered around the BlockStack component.

Thanks for flagging the issue. We'll have a fix soon.


Fit to schematic size

Global size of area is not correctly updated when schematic is not defined in the middle.

Hey @SaitouToshihide, please allow some time. I will go through and update the code. Thank you for indicating problem.


Error message not displaying

Going through regression tests for 1.6 and noticed that when filtering down to values that do not match any data points, an error message does not display. I spoke with Jeremy about it, he said it shouldn't delay this release but it'll get fixed for a future release.

Confirmed that a "No values selected. Update the filters above or load a different data set." message appears when the data is filtered down to values that do not match any data points during regression testing.


FAQ check_in_luca: Edit first sentence of answer to question 3

This PR solves #2064
DE:
EN:

@Ein-Tim Sorry, I didn't see you comment. We will have a look at rephrasing the question.

@dsarkar
No problem & thank you!


Security Reorg + Auth Config + TLS version

Creates a new Reference / Security section and consolidates descripti…ve security docs; updates references to TLS version; Adds authentication configuration docs.
Completes:
DOCS-458, DOCS-422, DOCS-2559, DOCS-985

This change is


exclude_from_weight_decay added

This modified LARS code was tested on CIFAR-10 with ResNet-50 backbone. Linear Evaluation Test Accuracy is ~85% after 100 epochs.

This modified LARS code was tested on CIFAR-10 with ResNet-50 backbone. Linear Evaluation Test Accuracy is ~85% after 100 epochs.

Hey, I am not able to achieve the linear evaluation accuracy reported in paper with this repo code. After modifying the LARS code, you are getting around 85%, so may I please know what hyper-parameters (like batchsize, LR, temperature) did you use?
Thank you.

@Ektagavas  Hi, I used a batch size of 64, LR 0.2, and temperature 0.5. 100 epochs for contrastive learning. Also, I used a pre-trained encoder. The results may differ if you train an encoder from scratch.

@Ektagavas Hi, I used a batch size of 64, LR 0.2, and temperature 0.5. 100 epochs for contrastive learning. Also, I used a pre-trained encoder. The results may differ if you train an encoder from scratch.

@sadimanna Thanks for your quick response and for sharing the details! Can you pls also share from where you got your pre-trained encoder?
Thank you.

@Ektagavas I used torchvision.models.resnet50(pretrained = True)

@Ektagavas I used torchvision.models.resnet50(pretrained = True)

Oh I see, the PyTorch's pre-trained model. Cool, thanks! :)


"charset=utf8" string in the base64 encoded SVGResource not allowed

Hello,
I can see that when moving from pixi v4 to v5 I have sometimes a strange behaviour with the loading of texture, and some research led me to this line below, which doesn't take into account the potential "charset=utf8" string before the "base64", which is exactly the scenario I'm in, hence the strange behaviour I encountered sometimes.
https://github.com/pixijs/pixi.js/blob/86ec8422dfa859f3fb3aa2f6880ac53b1ab8ce93/packages/core/src/textures/resources/SVGResource.ts#L244
Is is something to be fixed by a PR, or is it better adding it to the documentation?
Cheers.

PR welcome! I imagine we can replace that check with a RegExp.

@guigzz Can you please provide an image with 'charset=utf8' before base64?


Per-forum opt out

Allow channels to be opted out of being indexed by having some special text in the channel description like <dforum:optout> or something.

should be noted that yes, people do use forums in place of what you'd expect channels for, not just help and support.
it's a nice to have anyways and it's free to implement so yeah sure


Clean up the way we display fields (get_field_order(), ModelViews)

It's hard to keep iterating unless we fix this now.  We understand the requirements better so it makes sense to fix.

That didn't happen. Open more specific bugs.


How to deal redirect_uris with extra params

Let's assume that there is a valid client whose redirect url is 'https://github.com/oauth/callback'. When it request authorizition, it add some extra param to the url like 'https://github.com/oauth/callback?custom_param=something'. In this case, it would be rejected by node-oauth2-server. Is there any solution to handle this situation?

Push


[Auth] Cannot relogin into existing account after signed up anonymously

After user signs up anonymously, he cannot login with his previous registered account.

Should we close it as we have #1470 ? @louischan-oursky @kiootic


HTTPRouteConfig file has a limitation for number of items in the Match object

Description:
When defining more than 8 paths for the match object in HTTPRouteConfig file, it returns the following error message.
The HTTPRoute "petstore" is invalid: spec.rules[0].matches: Too many: 11: must have at most 8 items
In line no 644 of gatewayapi-crds.yaml, maxItems attribute has the value 8 which is causing the issue. Is there a rationale behind that value 8? Or is it just an arbitrary number for the testing purposes which is completely fine with removing/changing?
[optional Relevant Links:]
https://github.com/envoyproxy/gateway/releases/download/v0.2.0-rc1/gatewayapi-crds.yaml

@BLasan this is a maximum set by the Gateway API spec and I'm unaware of why 8 was decided. If this restriction is an issue, can you provide additional context by creating a Gateway API issue and linking to this issue for tracking purposes?
xref: https://github.com/kubernetes-sigs/gateway-api/blob/main/apis/v1beta1/httproute_types.go#L170
cc: @youngnick

8 was a relatively arbitrary number chosen to maximise the chance that the whole record won't exceed the default etcd size limit. We felt that if you were matching more than 8 separate criteria, you could come and talk to us about why.
The Gateway API CRDs should not be changed though, they are imported from the upstream project, and changing them will make your installation incompatible with the upstream APIs.
I'd be interested to see why you need to go past 8 matches in one rule. Remember you can also have multiple rules.

Hi @youngnick , we have a sample petstore swagger file which consists of more than 8 paths which needs to be converted to a HTTPRoute config file. That httproute config should consists of all paths in the swagger definition irrespective of the gateway implementation. If this is not the best practice, could you please suggest the most effective way of doing this and exposing it as an API? Also, this doesn’t support the path parameters in a particular path. (eg: /pet/{petid} cannot be configured as it is since {petid} is not considered as a path parameter.). Following is the sample httproute config file created from swagger
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: demo-petstore
  labels:
    version: 1.0.1
spec:
  parentRefs:
  - name: eg
  hostnames:
  - www.example.com
  rules:
  - matches:
    - path:
        type: Exact
        value: /api/v3/pet/{petId}/uploadImage
      method: POST
    - path:
        type: Exact
        value: /api/v3/pet
      method: POST
    - path:
        type: Exact
        value: /api/v3/pet
      method: PUT
    - path:
        type: Exact
        value: /api/v3/pet/findByStatus
      method: GET
    - path:
        type: Exact
        value: /api/v3/pet/findByTags
      method: GET
    - path:
        type: Exact
        value: /api/v3/pet/{petId}
      method: GET
    - path:
        type: Exact
        value: /api/v3/pet/{petId}
      method: POST
    - path:
        type: Exact
        value: /api/v3/pet/{petId}
      method: DELETE
    - path:
        type: Exact
        value: /api/v3/store/order
      method: POST
    - path:
        type: Exact
        value: /api/v3/store/order/{orderId}
      method: GET
    - path:
        type: Exact
        value: /api/v3/store/order/{orderId}
      method: DELETE
    - path:
        type: Exact
        value: /api/v3/store/inventory
      method: GET
    - path:
        type: Exact
        value: /api/v3/user/createWithArray
      method: POST
    - path:
        type: Exact
        value: /api/v3/user/createWithList
      method: POST
    - path:
        type: Exact
        value: /api/v3/user/{username}
      method: PUT

This issue is created to track this issue in gateway api project - https://github.com/kubernetes-sigs/gateway-api/issues/1485

https://github.com/kubernetes-sigs/gateway-api/pull/3205 increased the limit to 64
marking this as competed


[Enhancement] Protect be from crash when failed to filter unused columns

Why I'm doing:
When failed to filter unused columns, be will crash.
What I'm doing:
Protect be from crash when failed to filter unused columns
Fixes #issue
What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

[ ] This is a backport pr

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.3
[x] 3.2
[x] 3.1
[ ] 3.0
[ ] 2.5

Please fix the root cause first.

the root cause will be fixed, and the protection is also needed.

Incorrect delete meta information will affect many optimizations and may cause crashes or unexpected behaviors in other places:
eg: meta scan auto rewrite
tablet limit prune.
scan column prune.
other physical info optimize...
.....
Avoiding crashes alone is not a wise choice.


Merkle tree/proof for data exchange

Hello
I've been chatting on the Ethereum Attestation Service's Telegram channel about my desire to design a JSON structure which I can give to users as the actual data behind their attestation (of the root hash, on chain).
This JSON would be useless to a non-technical user but I'd hope that as momentum builds around blockchain-aware apps, it would be used by other apps as proof of some action or fact.
Developers will have to build that functionality into all the other apps, and the wider developer community, who may be only dipping their toes into this rather arcane world and perhaps even resistant to this work, would need all the help they can get.
I'm interested in designing a Merkle tree structure in JSON which can be

Used as a format for data exchange; it is both the data and the proof and nothing else is needed.
Very easily understood so that the developer can look at it, read-up on how Merkle trees generally work, and write code in their own language to read the data, recompute the root hash and compare it against a signed (on-chain) copy, ideally without use of any specialised libraries except perhaps a hasher and with as few new esoteric concepts as possible (strange ABI-encoding vs. well-known web or JS standards). A bonus would be that an LLM could also understand the JSON and write this code.

The goal (1) for data exchange would require that byte arrays be paired with a content-type field. For example, if the bytes were a selfie photo, then the MIME type of that encoding would be needed for it to be useful. "This photo is a likeness of the person I video conferenced, and that data is hashed and signed along with their passport details." And if the bytes were plain text, then the encoding would be needed, e.g. UTF-8 in Base-64.
The goal (2) I think would entail a structure that trades compactness for screaming structure, "oh, it actually resembles a Merkle tree", where the leaf nodes and all intermediate nodes with hashes would be included, hierarchically. The hashing algorithm and salt bytes would also need to be included so that the JSON has everything needed to for the developer to "fall into the pit of success" when writing code to process it.
What are your thoughts?

Hello @lukepuplett
First of all, I don't think this is the right place for this discussion. This repo is about features request / bug tracking that are specific to this library. Your should probably discuss that on places like stackoverflow or the OpenZeppelin forum.
About your question:
Merkle tree are designed so that a sub portion of the data can be proven to be part of the tree easily. The idea is that by commiting a simple hash (reference) you can prove something is in the data, without showing the data. If you don't need that property, then merkle tree are probably not the right structure for you.
Think of how software packages (or .iso are distributed). They don't use Merkle tree. They use normal hashes. As a user, you retrieve all the data/package, you hash it, and you check that the checksum is correct.
The first question you should ask yoruself is "who needs to prove what? in what context? with what trust assumption?". Only once the issue you want to solve is clear can you hope to get the right technical solution for it.

Thanks for your prompt response. StackOverflow is the wrong place for any kind of discussion, but perhaps I'll move this to the forum.
The odd thing about posting this kind of discussion is that everyone wants to bat it away somewhere else. Where does anyone go to establish a new data standard?
Regarding the use case, suppose I have someone's passport and I've personally verified that some wallet owner is the holder of the passport and I want to attest to this fact. I have a set of data such as name, nationality, passport number and a JPEG of their passport photo page.
I produce a Merkle tree of this data and I attest the root hash on-chain using the passport holder's wallet, then my app/service attests to that attestation to say that this data has been checked. My app/service is itself attested to by a government agency. We now have a strong chain of attestation of some important privately-held data.
The passport holder will need to download a single file of the actual data and all the hashes which roll-up to the root hash which matches what's on-chain and is attested to by my KYC company.
They can now present this file which holds their digital passport to some other service who can use that data, decode and look at the photo, and also recompute the hashes and check it has been attested.
But I could also produce a redacted version (or the user could edit the file and delete the data field(s)) so only e.g. their date-of-birth is in the clear, and now they could use this evidence to get access to adult content without revealing who they are.
In this example, the JSON document is both the actual data and the proof of it having been checked. That's the goal, to create a file format for this purpose.
It could potentially be an important new data format.

Where does anyone go to establish a new data standard?
Not in the repo of a library that implement a specific datastructure.

Where does anyone go to establish a new data standard?

An ERC could be a good fit.

Thanks frangio, I appreciate any tips.
The problem is that this sits between the on-chain and off-chain world, which is why it's important that it doesn't adhere to a particular ecosystem and doesn't introduce too many new concepts at once.
It's not specific to Ethereum per se, e.g. Solana could come up with its own attestation service and this JSON would need to communicate how to locate the attestation on Solana's chain somehow.
I wrote it up, if you're curious. In fact, I call out the problem with leaking the Solidity type system and ABI encoding.
https://gist.github.com/lukepuplett/a0793be0b7765539173df8cabc6b2c06


add action title and subtitle

add action title and subtitle

Hello. We have not added these features as built-in actions, because we believe the already existent /minecraft:title command is better than anything we could offer.
If you do not know how to use the command, you can find the documentation here: https://www.digminecraft.com/game_commands/title_command.php
You can also find a nice user interface to help you generate title commands here: https://minecraft.tools/en/title.php
If you have good arguments for adding these as built-in actions, please reopen the issue or create another one and share those arguments. Thank You!


@conda-forge-admin, please add user conda-forge/jupyter-book

See https://github.com/conda-forge/jupyter-book-feedstock/pull/19#issuecomment-1456500310


about alignratio in the training dataset

What does the value of alignratio in the training dataset mean? The range of gt_image/alignratio is not at [0,1]. What does the ratio do ? How are the alignratio in the train dataset derived?

@sunyclj Refer to #10.


Delete old data automatically

It would be nice to be able to configure Exceptionless to automatically remove old data.
E.g., if it's been more than 2 weeks since the last exception occurrence, then remove the exception stack.

We have a retention job that runs that will automatically cleanup old data based on the plans retention period. It would take quite a bit of work to implement this tho and cause a lot of churn for little gain. Thoughts?

Do you by chance have a code sample of how you implemented it? Say, if we wanted to do something similar for self-hosted version.

Yes, the source is here: https://github.com/exceptionless/Exceptionless/blob/master/Source/Core/Jobs/RetentionLimitsJob.cs

So, if we wanted to make this job work on a self-hosted plan, we would need to add retention-days to the organization. Would it make sense to update API to allow updating retention_days for Unlimited Plan. Something like:
        protected override PermissionResult CanUpdate(Organization original, Delta<NewOrganization> changes) {
            var changed = changes.GetEntity();
            if (!IsOrganizationNameAvailableInternal(changed.Name))
                return PermissionResult.DenyWithMessage("A organization with this name already exists.");
			if(OrganizationRetentionDaysChanged(changed.RetentionDays) && !CanOrganizationRetentionDaysBeChanged(original.PlanId))
                return PermissionResult.DenyWithMessage("Your billing plan does not allow to change retention days.");
            return base.CanUpdate(original, changes);
        }
        private bool CanOrganizationRetentionDaysBeChanged(string billingPlanId)
        {
            return billingPlanId == BillingManager.UnlimitedPlan.Id;
        }
        private bool OrganizationRetentionDaysChanged(int retentionDays)
        {
            return !_repository.GetByIds(GetAssociatedOrganizationIds()).Documents.Any(o => o.RetentionDays == retentionDays);
        }

I wouldn't make this change. I think the proper way to fix this would be to add the functionality to post custom plans as global admin. And then you could select this plan with that retention period. But the thing now. I think our retention job never deletes stacks only old events, but empty stacks shouldn't show up in the ui. What are your thoughts?

Deleting just events should be fine - our main concern is maintaining ElasticSearch performance. I'll look at the code. Thank you!

No problem, You could always just update the organization in elastic with new retention limits. The jobs will run automatically and will manage this for you :).

I was thinking that too. Just need to figure out how to do it in ES.

I think we'll just do ES update. I used the query below in case someone else needs it. But it's not an issue anymore for us. Doesn't look like it's worth the effort.
PUT /organizations-v1/organization/youOrganizationId?pretty
{
"retention_days": "21",
"billing_change_date":"2015-09-30T15:11:16.5708671-04:00",
"modified_utc":"2015-09-30T15:11:16.5708671-04:00"
}
Thanks again for your help!

Awesome thanks! We really appreciate this. I was thinking more on your custom code above and I think it would work but it could be confusing because we'd be adding it to the update model as an allowed property and then rejecting it on specific cases. So for consistency I don't think it would work :(.

Is it fine if I close this issue?

Definitely. Thank you!

There is also a setting you can do to control the retention period for all projects (https://github.com/exceptionless/Exceptionless/blob/master/src/Exceptionless.Core/Settings.cs#L150) If you set a property called MaximumRetentionDays it will delete all data past this date.

I had to do this recently, and this was the simpler way to make the update in ES:
POST /organizations/organization/[guid]/_update?pretty
{
    "doc" : {
            "retention_days": 180
    }
}

@lordscarlet I had version 5.0 installed, and that code worked perfectly.
Now in version 6.0 it returns an error.
Can you tell me if anything has changed?
{
  "error": {
    "root_cause": [
      {
        "type": "document_missing_exception",
        "reason": "[organization][guid]: document missing",
        "index_uuid": "QLH3U8WXR3-CfCcWa4OcZg",
        "shard": "0",
        "index": "organizations"
      }
    ],
    "type": "document_missing_exception",
    "reason": "[organization][guid]: document missing",
    "index_uuid": "QLH3U8WXR3-CfCcWa4OcZg",
    "shard": "0",
    "index": "organizations"
  },
  "status": 404
}

Elasticsearch 7 removed index types. Use the following command.
POST /organizations/_doc/[guid]/_update?pretty
{
    "doc" : {
            "retention_days": 180
    }
}


Move to Dynamoid for data persistence

This disables active record and adds dynamoid to persist the FromResponse in DynamoDB.  We are switching from Postgres to using DynamoDB (a AWS managed NoSQL DB) to store the data. Doing this because we can set IAM permissions so the app is allowed to store data, but cannot read it (for security).

Can remove the pg gem and the service in .travis.yml
Will probably need to add dynamodb localstack in travis for tests, or use a stub

Most recent changes look 🥇

@huwd I'm leaving in those commented lines just so people understand that ActiveRecord isn't being used.

If you update the readme, i'll pull this branch and give it a test locally?


BUG: On $reset if load the same values the Local Storage is clear.

Bug: Browser storage mantein clear when $localstorage.$reset and try to load the same data.
$localstorage.$reset($localstorage);  // Brower Local Storage are empty
The cause: On $reset the _last$storage variable is not clear and this make that on $apply the conditon always is true:
if (!angular.equals($storage, _last$storage))
Fix (suggested):
                  $reset: function(items) {
                            for (var k in $storage) {
                                '$' === k[0] || (delete $storage[k] && webStorage.removeItem(storageKeyPrefix + k));
                            }
                            _last$storage={};
                            return $storage.$default(items);
                        },

A hack that would work without changing the code would be:
const items = $localstorage;
$localstorage.$reset();
$localstorage.$apply(); // force storing the data to make sure _last$storage is overwritten.
$localstorage.$reset(items);


fix some titles

so im like 20% this will work and not be overwritten
but lets try it and then try something else if it doesn't work

gonna try this first instead
https://github.com/pulumi/pulumi/pull/11506


DWARF symbol generation failure at -00

Currently, for every kernel Numba-ddpy generates two functions: a wrapper spir_kernel and a spir_func. All the kernel code is inside the spir_kernel.
Enabling debug metadata only adds DWARF symbols to the spir_kernel and not the outer spir_function. At O1 compilation levels the spir_func gents inlined into the spir_kernel and GDB is able to set break points. But, at -O0 inlining is turned off and GDB fails to register any breakpoint as the wrapper spir_function has no debug metadata.
The wrapper function is generated at target.py:generate_kernel_wrapper

@akharche I am thinking of a design where we do not need a wrapper function and generate a single spir_kernel. Till that happens, how we can add debug metadata to the wrapper function that we generate inside target.py:generate_kernel_wrapper.


FLAG Values?

I'm working on the JS implementation of this protocol. In the docs, I do not see explicit values for Flag. Does this information exist somewhere?

Flags not specified should be assumed to be set to 0 when sent and not checked when received. I'll add clarity in the doc under the frame header.

That's what I assumed. However, what about the values for the other flags such as ignore or metadata?

It would be nice to have a table for all of the possible flag values similar to errors and types.

Ignore and Metadata depend on the frame type and should already be specified for each frame type. Additionally flags for each frame type are also already specified. If there are gaps, let me know.

It'd be good to specify upfront what all of the Flags might be. I perused the entire document and I think there are only 3. IGNORE, METADATA, and LEASE Is this correct?

The reason I ask is I see in the Java implementation, there are FLAGS_RESPONSE_C and FLAGS_RESPONSE_F values that are not specified in the protocol doc. https://github.com/ReactiveSocket/reactivesocket-java/blob/master/src/main/java/io/reactivesocket/FrameFlyweight.java#L51-L54

There's now 4 flags: IGNORE, METADATA, LEASE and STRICT.
IGNORE, METADATA are used in every frame
LEASE and STRICT can only be used with the SETUP frame.
Note: The java implementation is not complete yet.

@stevegury What about the FOLLOWS and COMPLETE flags here: https://github.com/ReactiveSocket/reactivesocket/blob/master/Protocol.md#response-frame

You're right, my bad. I only looked at the message a Requester is sending.

They also seem to collide with the LEASE frame, which is also 0b00100000

Oh I see, you want all the flags to be unique.
I don't think we need to ensure that (this is even too restrictive). The interpretation of a flag depends on the type of message you're reading.

That's cool with me.


can you add Let's Encrypt

Our project is very nice.
It is possible to add the Let's Encrypt Script to get free and automated SSL-certificates for each container?
Look here: https://github.com/letsencrypt/letsencrypt/tree/master/letsencrypt-nginx
That can be the whipped cream. ;)
P.S. Let's Encrypt should be availability for production in  week of November 16, 2015
https://letsencrypt.org/2015/08/07/updated-lets-encrypt-launch-schedule.html

I found the solution: https://letsencrypt.readthedocs.org/en/latest/using.html#running-with-docker

"production working" does not mean generally available. They won't issue certs until at least Nov 16. I wouldn't be surprised if that schedule slips yet again.

@AdminJack It's not going to be possible to run those instructions in a straightforward way with this image. Those instructions involve running a Docker image called quay.io/letsencrypt/letsencrypt. That image appears to start its own Python-based web application that listens on port 443 and port 80, which would conflict with the Nginx daemon running in the nginx-proxy image.
In order to do what you suggest, this image would somehow have to run the letsencrypt app while waiting to verify control of the domain and getting the certificate, then switch over to running Nginx for nginx-proxy purposes. It would also presumably have to do this any time a new virtual host is detected.
There's also the fact that once this certificate was retrieved and installed, it would have to be stored in a volume to avoid being lost when the nginx-proxy Docker container is replaced (say to pull in an updated version of the nginx-proxy image).
It's also going to be problematic to run quay.io/letsencrypt/letsencrypt on the same host as nginx-proxy at the same time, since they both need to be exposed on port 443. As far as I could see, there isn't a way to run the quay.io/letsencrypt/letsencrypt Docker image on an alternate port.

All that being said, there may actually be some way to run quay.io/letsencrypt/letsencrypt behind nginx-proxy with VIRTUAL_HOST set that could actually work. I'm not sure what quay.io/letsencrypt/letsencrypt is doing and how the validation client is connecting back to it to know for sure.

letsencrypt (not the docker image) will put the files here:
/etc/letsencrypt/live/my.domain.com/
--> cert.pem
--> privkey.pem
--> chain.pem
--> fullchain.pem

so, if I create a dir
/etc/proxy-ssl
--> my.domain.com.crt --> /etc/letsencrypt/lime/my.domain.com/cert.pem
--> my.domain.com.key --> /etc/letsencrypt/lime/my.domain.com/privkey.pem

and run my container with -v /etc/proxy-ssl:/etc/nginx/certs  it should work, right?
Have I missed something?

@dropfen That won't work unless you also mount /etc/letsencrypt/mydomain.com into the container since those symlinks will be dangling otherwise.

@md5 right. So we actually know the naming convention of letsencrypt. The best scenario would be to mount it as /etc/nginx/certs and use their naming conventions their

Which language is the nginx.tmpl file written?

It use Go's text.template package. There are additional custom functions
added in the docker-gen command.

Using the ports 80 or 443 is one way to authenticate to the let's encrypt service. The documentation lists other possibilities besides standalone.
I managed to get it working with the webroot option. It creates a folder called .well-known under your web root. You can use the Per-VIRTUAL_HOST configuration to point it to a volume.
Another option that might work better in the long term is the nginx option. But I haven't looked into it yet because it is marked experimental.

I'm interested in this too. Ideally, nginx-proxy could validate new certificates without affecting the backend.

If shutting down nginx-proxy while the certificate is generated is an option you may use the following commands. Assuming that:

The nginx-proxy docker container is called nginx-proxy.
The certificate files for nginx-proxy are locate at /root/nginx-proxy-certs/.
The domain is called domain.tld

docker stop nginx-proxy
./letsencrypt  --agree-tos --agree-dev-preview certonly -d domain.tld
cp /etc/letsencrypt/live/domain.tld/privkey.pem /root/nginx-proxy-certs/domain.tld.key
cp /etc/letsencrypt/live/domain.tld/fullchain.pem /root/nginx-proxy-certs/domain.tld.crt
docker start nginx-proxy

I created a fork with Let's Encrypt support. See the pull request. #300.
I used the simp_le Let's Encrypt client and a bash script.

I made a fork of nginx-proxy for out of the box support of Let's Encrypt. Not ready to go upstream but it's a start

You might want to have a look at mbrugger/letsencrypt-nginx-docker – this is a prime candidate for integration with nginx-proxy.

Thanks for your liink! Unfortunatly, this nginx is not fully automated. In my case, I don't have a cron and everything needs to be working. Don't get me wrong, my solution is working, but it's not really clean. I posted here if someone is looking for a kickstart to do a better code than mine ;)

I have a solution with the original letsencrypt client integrated in nginx-proxy. I need some time to stabilize it...

@dropfen did you manage to get letsencrypt to work with this proxy? My browser does not trust the certs.

It's actually quite a bit easier than that ^^ I made a blog post about it if anyone cares to look here zettabyte.me
It's as easy as making a couple of symlinks:
sudo ln -s /etc/letsencrypt/live/example.com/privkey.pem /var/local/proxy-certs/example.com.key
sudo ln -s /etc/letsencrypt/live/example.com/cert.pem /var/local/proxy-certs/example.com.crt

Then running the following to start the proxy:
docker run -d -p 80:80 -p 443:443 --name=proxy --restart=always \
 -v /var/local/nginx/certs:/etc/nginx/certs -v /etc/letsencrypt:/etc/letsencrypt \
 -v /var/local/proxy-confs:/etc/nginx/vhost.d:ro \
 -v /var/run/docker.sock:/tmp/docker.sock:ro jwilder/nginx-proxy

@ambroisemaupate and @hrwebasst you are offtopic.
This issue is about integrated letsencrypt directly info the ngnix-proxy container to create and renew certs automatically.

@JrCs I don't think those comments are off-topic. This issue should be the place to discuss whether Let's Encrypt support should be built-in or whether it should be done through coordinating with other containers (and if so, what level of accomodation should be added to make it easy for other containers to add this support).

I thought a little about it, and I believe the best approach would be to have a separate docker container (letsencrypt-nginxproxy) that exposes one port for domain validation and another one for obtaining the certificates (to store them inside nginx-proxy).
Since nginx-proxy already listens to new containers being started, the letsencrypt-nginxproxy container could have a special env variable that tells nginx-proxy that it is time to pull in the new certificates when started.  letsencrypt-nginxproxy should terminate once done.
letsencrypt-nginxproxy should look for VIRTUAL_HOST envs just like nginx-proxy and request certificates for all given domains.

I don't see why we need to divide into smaller containers. LE is in beta for now, but they will be "production ready" some day, and at that time, registration will be easier I think (no google form). That means that Traefik could generate dynamically generate a new cert for each new frontend.
nginx-proxy is only working with Docker for example. If we need to make a LE generator, we need to find a way to trigger the execution. I'm currently using a fork version of nginx-proxy with Let's Encrypt, here's what I do:

Start LE official container and expose volume /etc/letsencrypt
Use --volumes-from option on my fork of nginx-proxy

It's not dynamic (e.g I need to restart my containers to generate a new certs), and I was hoping replace nginx-proxy with Traefik with Let's Encrypt support in a easy mode. Something like:

Declare "letsencrypt = true" on config file
That's it! It will generate on the fly certificates

I've done a bit of thinking about this and feel there needs to be a no downtime solution.
At home I've got a cron job which stops nginx, runs lets encrypt to generate certs and then starts nginx again. This results in a few seconds of downtime, which is fine for my home server.
However in a production environment those seconds of downtime would be best avoided. Therefore it may be better to use the webroot authentication method instead of the standalone. Webroot adds a file to a location accessible on your server to authenticate against, while leaving your server running, and then removes it again.
I wonder if it would be possible to have a lets encrypt container running a simple web server which implements the webroot method. Then have the proxy detect that this container is running and proxy just that file location to the lets encrypt container while leaving the rest of the proxy pointing at the service.
Then when the lets encrypt container exits remove that single file proxy.

@jacobtomlinson my letsencrypt-nginx-proxy-companion container allow to create/renew Let's Encrypt certificates with no downtime at all.

@JrCs Neat! What authentication method does it use to get no downtime?

@jacobtomlinson It use webroot with nginx-proxy has webserver

@JrCs Perfect! That's pretty much what I was going to make, you've saved me a lot of time there, thanks!

Awesome @JrCs, I'll have to set that up before my certs expire :)
and thanks @hrwebasst for the clear explanation of how-to

I made an image that obtains certificates from Let's Encrypt, and it uses docker-gen to do container discovery in a similar way as Nginx-proxy.
https://github.com/SteveLTN/https-portal

We have integrated letsencrypt with nginx-proxy. It's a new image that has jwilder/nginx-proxy:0.2.0 as base image. It will auto generate ssl certs for all vhosts. Only when you create the container for the first time, it might take up to 10 minutes before the ssl cert actually shows up. That's due to the update process letsencrypt runs at startup. Have a look here and let me know what we can do! We are happy to make it better for all!
https://hub.docker.com/r/eforce21/letsencrypt-nginx-proxy/
https://github.com/eforce21/letsencrypt-nginx-proxy

I have turned the setup from @JrCs into a small, opinionated docker-compose example at docker-nginx-gen-letsencrypt.

There is also an alternative implementation: https://github.com/tozd/docker-nginx-proxy

And another https://github.com/sashee/letsencrypt-proxy-docker
On 31 May 2016 at 12:41, gw0<EMAIL_ADDRESS>wrote:

There is also an alternative implementation:
https://github.com/tozd/docker-nginx-proxy
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/jwilder/nginx-proxy/issues/274#issuecomment-222653325,
or mute the thread
https://github.com/notifications/unsubscribe/ABka_MaX8O-GtBYZWAEIEkmFgHyPPOMPks5qHBBjgaJpZM4GSt7s
.

And another: https://github.com/indiehosters/nginx (forked from @almereyda )
Yes @almereyda you can now choose between haproxy and nginx with LibrePaaS \o/ :)
@jwilder I really think it would be nice this repo gives a basic docker-compose file with letsencrypt capabilities.

I can confirm that @SteveLTN 's solution mentioned above works really well and allows containers to be added and removed on the fly with no downtime!
https://github.com/SteveLTN/https-portal
I have been really struggling for a couple weeks now to get nginx-proxy + letsencrypt to work with multiple containers (although that's most likely due to my lack of knowledge as much as anything else ;)

https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion resolves this.


The "Weight Equally" checkbox should be enabled by default

Describe the bug
The "Weight Equally" checkbox for the Conditions table should be enabled by default because experiments with equally weighted conditions will be more common.
To Reproduce
Steps to reproduce the behavior:

Go to the Design step in the stepper and see the "Weight Equally" checkbox is disabled by default

Expected behavior
The "Weight Equally" checkbox for the Conditions table should be enabled by default
Screenshots

Current design:

Expected design:

This issue is combined with issue #422 and merge with PR #436


Merge branch 'dev' into 'master'

Pull Request Test Coverage Report for Build 488

124 of 124 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.9%) to 93.695%

Totals

Change from base Build 483:
0.9%

Covered Lines:
951

Relevant Lines:
1015

💛  - Coveralls


Vault Transaction: Slow Send

Task Description
Sending/Signing a transaction from Vault is taking a lot of time. Need to check which operations are taking time and optimize the same.
Time taken while spending from a 3/5 Vault:
https://github.com/bithyve/bitcoin-keeper/assets/17980466/776965ea-269d-474a-89fd-acd2548bd286

Verified  this issue on keeper app v 1.1.8(265)


Atom for Mac m1

I just want to know when atom's native version for Mac m1 will be available

see #21688


Importer cleanup certificate handling

[x] allow for global activation/deactivation of cert checking --> introduce default config settings (which may be overwritten by command line switch) for

suppressing cert warnings
verification of ceriticates

[x] set default value to False to make default installation work
[ ] try to set default value to True and add self signed certs as valid?

cp certificate to /usr/local/share/ca-certificates/
sudo dpkg-reconfigure ca-certificates -f noninteractive
connect to hostname (not localhost)

connect to sting with validated cert:
fworch@debian10:~/importer$ ./import-mgm.py -m5  -v
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:849: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
2022-05-23T14:03:34+0100 [INFO ] [common.py :import_man: 160] starting import of management sting(5), import_id=966
/usr/lib/python3/dist-packages/urllib3/connection.py:362: SubjectAltNameWarning: Certificate for <IP_ADDRESS> has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python3/dist-packages/urllib3/connection.py:362: SubjectAltNameWarning: Certificate for <IP_ADDRESS> has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python3/dist-packages/urllib3/connection.py:362: SubjectAltNameWarning: Certificate for <IP_ADDRESS> has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
2022-05-23T14:03:39+0100 [INFO ] [common.py :complete_i: 345] import_management: import no. 966 for management sting (id=5) successful, change_count: 0, duration: 5s
fworch@debian10:~/importer$


disable/ enable edit and delete event based on the callback func

Add read_only prop to the event and don't show edit/ delete if it is true

Tooltip for the disabled event:


Append multiple agents to TrajectoryData objects

Problem
Refer to #153
Solution

updated buffer size for single agents
enforce integer datatypes for indices

Type of change

Bug fix (non-breaking change which fixes an issue)

Steps to Verify:
Refer to this notebook which creates a single simularium file from multiple cytosim simulations
Screenshots:
Simularium trajectory with multiple actin filaments visualized simultaneously

Keyfiles:
simulariumio/data_objects/trajectory_data.py

Looks good to me!

I see broken tests.. floating point comparisons.. and combined with the int casting in the code change, I'm not sure what happened.  But the precision for the float comparisons seems way too high.  Usually in a unit test you would never test for equality with floating point numbers that possibly had math run on them, but use an epsilon.

I see broken tests.. floating point comparisons.. and combined with the int casting in the code change, I'm not sure what happened. But the precision for the float comparisons seems way too high. Usually in a unit test you would never test for equality with floating point numbers that possibly had math run on them, but use an epsilon.

Yup, looks like very small floating point errors are causing the tests to fail in an OS-dependent way. I'm not sure what the best way to deal with this is, would be happy to get some input. Any thoughts @ascibisz?

Looked into the bug a bit, currently there's an equality assertion between two tuples containing a dictionary and a float array in a few tests. Pytest does have an approx function that handles float array comparison with an epsilon (https://docs.pytest.org/en/7.1.x/reference/reference.html#pytest-approx) that might work if we compare the elements of the tuple?
Something like:
# test_cellpack_convert.py, 156
assert expected_bundleData_data[0] == bundleData["data"][0] && expected_bundleData_data[1] == pytest.approx(bundleData["data"][1]

Also relevant stackoverflow thread if this is happening elsewhere: https://stackoverflow.com/questions/56046524/check-if-python-dictionaries-are-equal-allowing-small-difference-for-floats

Looked into the bug a bit, currently there's an equality assertion between two tuples containing a dictionary and a float array in a few tests. Pytest does have an approx function that handles float array comparison with an epsilon (https://docs.pytest.org/en/7.1.x/reference/reference.html#pytest-approx) that might work if we compare the elements of the tuple?
Something like:
# test_cellpack_convert.py, 156
assert expected_bundleData_data[0] == bundleData["data"][0] && expected_bundleData_data[1] == pytest.approx(bundleData["data"][1]

Also relevant stackoverflow thread if this is happening elsewhere: https://stackoverflow.com/questions/56046524/check-if-python-dictionaries-are-equal-allowing-small-difference-for-floats

I'm trying out a fix using numpy isclose which might work. Will keep you posted!

Codecov Report
Patch coverage: 100.00% and no project coverage change.

Comparison is base (b2ab7fe) 92.55% compared to head (eb01b78) 92.56%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #155   +/-   ##
=======================================
  Coverage   92.55%   92.56%
=======================================
  Files         101      101
  Lines        4542     4546    +4
=======================================
+ Hits         4204     4208    +4
  Misses        338      338

Files Changed
Coverage Δ

simulariumio/data_objects/trajectory_data.py
96.87% <100.00%> (+0.15%)
:arrow_up:

...riumio/tests/converters/test_cellpack_converter.py
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


misc: move to std::print

conflict tho

wouldn't it be better if u replace cerr to print(stderr, ...)

@littleblack111
I think help should not be printed to stderr.
But I can change it back. Do not have a strong opinion on it.


Line 432 of PatientLevelPrediction chapter

In the line 432 of PatientLevelPrediction chapter,

Measurement: Construct covariates for each measurement concept ID and time interval selected and if a patient has the concept ID recorded during the specified time interval prior to the cohort start date in the measurement table, the covariate value is 1, otherwise 0.
Measurement Value: Construct covariates for each measurement concept ID with a value and time interval selected and if a patient has the concept ID recorded during the specified time interval prior to the cohort start date in the measurement table, the covariate value is the measurement value, otherwise 0.

To my knowledge, the measurement value is 'continuous value' not a binary value. How do you think @PRijnbeek  @jreps  @schuemie ?

Sorry, I misunderstood this. I think this issue had been fixed before.


Release Troubles

The v3.1.0 release is all ready to go, but there are a couple of issues stopping me from publishing it:

It looks like I was added as an npm collaborator to the old addon by mistake, so I can't publish to NPM yet. Would someone mind adding me to the new one?
ember-cli-release is configured to use HubFlow, but part of its process involves pushing some commits to develop directly, which I can't do because the develop branch is protected on GitHub (it's enforcing a Travis status check). I lack permissions to temporarily relax the constraint, and that seems really un-ideal anyway. Any advice on this?

Paging @seawatts and @aaguiarz for assistance. Thanks in advance!

So who do I need to pester to help get this resolved?  I might not be the best coder, but man can I be a squeaky wheel...

@XaserAcheron I apologize.. i missed this. You are an admin of the repo now, let me know if that works for you.

Thanks @aaguiarz ! @XaserAcheron you should be good to go.

Thanks! That's one block removed, though I still lack npm publish permissions. Are you able to take a look at that while you're here, @aaguiarz?

On it

@XaserAcheron what's your npm user?

https://www.npmjs.com/~xaser <-- pretty barebones, but I exist. :P

You should are a collaborator on https://www.npmjs.com/package/ember-simple-auth-auth0 now

All good now. Thanks again!
v3.1.0 is live now, so I suppose this case is closed. ;)


added workflow for autocommenting on merged PR

Fixes #50

@hasmithagunda123 please review my PR and merge it if it is good to go


Added more options to start.sh

The VMs supplied by modern.ie are currently 32b systems.  I was
having issues running these with the packaged version of
qemu-system-x86_64 on my distro (Ubuntu 16.04) and wanted to be
able to have a flag to use qemu-system-i386.  In doing so, I
also added a few more options that made start.sh more flexible.
--32bqemu
use qemu-system-i386
-m
RAM for system in MB
--pcnet
Use pcnet network adaptor.
--image imagename
use imagename for qemu-system
--help/-?
A (probably excessively) terse help output.
This patch does admittedly break the current "just pass an image
name" on the command line.  I don't know how to reconcile that.

This patch does admittedly break the current "just pass an image name" on the command line. I don't know how to reconcile that.

Looks easy enough to achieve though, am I missing something? Insisting the image name is at the end of the options is reasonable.
http://stackoverflow.com/questions/11742996/shell-script-is-mixing-getopts-with-positional-parameters-possible
I'm not convinced it's worth having both the positional image parameter option and a --image, however if you really wanted to you could specify that --image wins over the positional parameter, which wins over the default.


Mast: Watch (?-i:5774b06d7bb11a6c355c214cd673d5da)(?#apnews.com)

Mast requests the watch of the watch_keyword (?-i:5774b06d7bb11a6c355c214cd673d5da)(?#apnews.com). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

(?-i:5774b06d7bb11a6c355c214cd673d5da)(?#apnews.com) has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by tripleee in Charcoal HQ


Unknown identifier : FAnonymous

Trying to build at Ubuntu, I get this error:
/usr/lib/haxe/lib/lime/7,6,3/src/lime/_internal/macros/AssetsMacro.hx:361: character 73 : Unknown identifier : FAnonymous
/usr/lib/haxe/lib/lime/7,6,3/src/lime/_internal/macros/AssetsMacro.hx:361: character 73 : For function argument 'name'
/usr/lib/haxe/lib/openfl/8,9,2/src/openfl/text/Font.hx:23: character 1 : Build failure
Command exited with non-zero status 1

and this error:
/usr/lib/haxe/lib/lime/7,6,3/src/lime/_internal/macros/EventMacro.hx:153: character 48 : Too many arguments
/usr/lib/haxe/lib/lime/7,6,3/src/lime/ui/Window.hx:82: characters 42-67 : Unknown<0> cannot be constructed
Command exited with non-zero status 1

using haxe 4.0.5

I'm not getting any errors in Haxe 4.2.4, but I can confirm that there's a problem with Image embedding. I'm testing the DisplayingABitmap sample, with a few changes:
@:bitmap("Assets/openfl.png") class OpenFLGraphic extends openfl.display.BitmapData {}
@:image("Assets/openfl.png") class LimeGraphic extends lime.graphics.Image {}
//...
var bitmap = new Bitmap(new OpenFLGraphic(512, 200));
addChild(bitmap);

bitmap = new Bitmap(BitmapData.fromImage(new LimeGraphic()));
bitmap.y = 200;
addChild(bitmap);

The OpenFL one works fine except that its width and height are both 0 in HTML5. The Lime one is invisible in HTML5. Both work correctly on other targets. I didn't spot any major differences between the two AssetsMacro classes. No idea what's going on.


Get 'sources' files from S3

We've been informed that the /tmp folder (where we're looking now for S3 files) is not reliable and may not always contain the files our code will look for, possibly especially for ALKilnInThePlayground (I think that the age of the Project might have something to do with it?). Notes:

If the files are used in the interview, they will be present in /tmp when the interview is run. It's probably unreasonable for us to have the interview authors include those files on their page (only in server dev mode, of course) specifically to make sure they're in /tmp. A note: "The filesystem path can be obtained with path_and_mimetype() or path()" <- not sure if that refers to the /tmp directory or not.
For getting the files from S3 itself we'll need credentials. They're in the docassemble config, which I think we can get in node through the da API. We could also probably get it in the YAML of the ALKiln interview, but I'm not sure how that would help. "The only time S3 does not require credentials is when there is an IAM role for the EC2 instance that has permissions already" <- not quite sure what that means or whether the ALKiln interview, running on the same server as the Project, would have access to that. If it does, I suppose there might be an option to send all the files instead of the file path, but that seems less than ideal.
Not sure how convoluted the S3 API is. s4cmd is a possible Python candidate for using the S3 API and there are potentially a couple node packages that do the same kind of thing. We can look for s3cmd or just S3.

As per our discussion in deep dive, we can force the files to be cached in the /tmp' folder with [code like that in the AL dashboard](https://github.com/SuffolkLITLab/docassemble-ALDashboard/blob/main/docassemble/ALDashboard/create_package.py#L14-L17). The arg fix` is apparently the key to that. Then the files will be there for at least 2 hours. We might want to add that limitation to the docs at some point, though I'd be surprised if anyone actually runs into that limitation.


Some tweets in timeline do not contain Media

Some tweets in the timeline do not contain Media although the tweet does contain an image.
The  attached demo console application should simulate a client accessing twitter. I want to display the text and the image of every tweet. Some tweets do not contain an Image in media although twitter display an image.
DemoProgram.zip

I am also experiencing this issue.
Further more, some tweets missing Media will also missing Hashtag, although the tweet contains both image and Hashtag.

Hello there.
Could you please send me the Tweet ID that you consider should contain a Media/Hashtag so that I can test and see if the problem comes from Tweetinvi or from Twitter.

864284541057355777
865740426023653376
Both two should contains hashtag and image

Your problem is that this is an extended Tweet and I guess you did not configure your application to use extended Tweets.
Please use the following :
TweetinviConfig.CurrentThreadSettings.TweetMode = TweetMode.Extended;
TweetinviConfig.ApplicationSettings.TweetMode = TweetMode.Extended;


Download does not automatically recover when the connection times out

I set youtube-dl to download a youtube playlist overnight. During the night the connection times out and youtube-dl sits idle for a long time until eventually the connection times out.
This might be due to some minor packet loss on my line:
Ping statistics for <IP_ADDRESS>: Packets: Sent = 200, Received = 193, Lost = 7 (3% loss),
I ran it last night with --verbose and got the following output:
D:\youtube-dl>youtube-dl -U
youtube-dl is up-to-date (2014.11.16)

D:\youtube-dl>youtube-dl.exe --verbose --retries 999 --download-archive _test10202.txt
 https://www.youtube.com/playlist?list=PLkT4HgHxrZK9YInmLpeylxsTp5zM3IJHR
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', '--retries', '999', '--download-archive', '_test10202.txt', 'https://
www.youtube.com/playlist?list=PLkT4HgHxrZK9YInmLpeylxsTp5zM3IJHR']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2014.11.16
[debug] Python version 2.7.8 - Windows-7-6.1.7601-SP1
[debug] exe versions: none
[debug] Proxy map: {}
[youtube:playlist] PLkT4HgHxrZK9YInmLpeylxsTp5zM3IJHR: Downloading webpage
[download] Downloading playlist: test10202-deletable
[youtube:playlist] playlist test10202-deletable: Collected 46 video ids (downloading 46 of them)
[download] Downloading video #1 of 46
[youtube] Confirming age
[youtube] MtY2WGfquDk: Downloading webpage
[youtube] MtY2WGfquDk: Downloading video info webpage
[youtube] MtY2WGfquDk: Extracting video information
[youtube] MtY2WGfquDk: Downloading DASH manifest
[debug] Invoking downloader on u'https://r1---sn-woc7en7z.googlevideo.com/videoplayback?ms=au&initcwndbps=1387
50&mv=m&sparams=id%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmm%2Cms%2Cmv%2Cratebypass%2Crequiressl%2Csource%2Cupn%
2Cexpire&source=youtube&sver=3&mt=1416350398&fexp=904846%2C907259%2C916614%2C919330%2C927622%2C931350%2C932404
%2C936117%2C938692%2C942491%2C943909%2C947209%2C947215%2C947221%2C948116%2C948124%2C952302%2C952605%2C952901%2
C953912%2C957103%2C957105%2C957201&itag=22&upn=SDjNjywZtrg&mm=31&signature=48E6D15D502BCE56EB165E56E4CF65C2E24
4F5EB.B9DDB2DBC9A6A4D3FADAD814119C0C6764AE62B2&ipbits=0&expire=1416372101&id=o-AMdzPK1EhlLqc24grYRSJpujyJc3TCX
5cSj_pVufKbEv&key=yt5&ratebypass=yes&requiressl=yes&ip=<IP_ADDRESS>'
[download] Destination: Classics - Planescape Torment (Windows)-MtY2WGfquDk.mp4
[download] 100% of 283.36MiB in 40:55
[download] Downloading video #2 of 46
[youtube] 72us6pnbEvE: Downloading webpage
[youtube] 72us6pnbEvE: Downloading video info webpage
[youtube] 72us6pnbEvE: Extracting video information
[youtube] 72us6pnbEvE: Downloading DASH manifest
[debug] Invoking downloader on u'https://r2---sn-woc7en7z.googlevideo.com/videoplayback?id=o-AEwsEhH1Bm5VbPH46
TqhipIHki8_Lau33ym5H6tFyNWY&upn=YdBIgQ738Eo&requiressl=yes&ms=au&mt=1416352917&ratebypass=yes&mv=m&sparams=id%
2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmm%2Cms%2Cmv%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&source=you
tube&mm=31&signature=D135764C334B4E017270E45AFDEB8F1CA952EF39.93D41F6D24EB7729DD376979E37B89B7DC75A7EA&key=yt5
&fexp=904846%2C907259%2C916614%2C919330%2C927622%2C931350%2C932404%2C936117%2C938692%2C942491%2C943909%2C94720
9%2C947215%2C947221%2C948116%2C948124%2C952302%2C952605%2C952901%2C953912%2C957103%2C957105%2C957201&sver=3&it
ag=18&expire=1416374574&ip=<IP_ADDRESS>&ipbits=0&initcwndbps=70000'
[download] Destination: Richard P Feynman - Quantum Mechanical View of Reality 1 (Part 1)-72us6pnbEvE.mp4
[download] 100% of 78.68MiB in 11:31
[download] Downloading video #3 of 46
[youtube] cQBuB-J4nI4: Downloading webpage
[youtube] cQBuB-J4nI4: Downloading video info webpage
[youtube] cQBuB-J4nI4: Extracting video information
[youtube] cQBuB-J4nI4: Downloading DASH manifest
[debug] Invoking downloader on u'https://r2---sn-woc7en76.googlevideo.com/videoplayback?requiressl=yes&ms=au&m
v=m&ipbits=0&initcwndbps=82500&ip=<IP_ADDRESS>&source=youtube&id=o-ABZcxRiPRjSWGGO9oj2IqW6sBo83JKja0xqWDmsTq
ovz&expire=1416375274&key=yt5&mm=31&fexp=904846%2C907259%2C916614%2C919330%2C927622%2C931350%2C932404%2C936117
%2C938692%2C942491%2C943909%2C947209%2C947215%2C947221%2C948116%2C948124%2C952302%2C952605%2C952901%2C953912%2
C957103%2C957105%2C957201&itag=22&mt=1416353610&upn=1ClEWtFQa30&sver=3&ratebypass=yes&sparams=id%2Cinitcwndbps
%2Cip%2Cipbits%2Citag%2Cmm%2Cms%2Cmv%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&signature=0C6DFA191C588
69927B641FA23BFF73FE1BC6F0D.F93D8AD6D964B8B044A9BA0FEA9A3AB04EAFC29C'
[download] Destination: Dota 2 Song of the Siren Roshan Steal (XMG Captains Draft 2.0)-cQBuB-J4nI4.mp4
[download] 100% of 24.39MiB in 04:46
[download] Downloading video #4 of 46
[youtube] 0KBFxnK8DEw: Downloading webpage
[youtube] 0KBFxnK8DEw: Downloading video info webpage
[youtube] 0KBFxnK8DEw: Extracting video information
[youtube] 0KBFxnK8DEw: Downloading DASH manifest
[debug] Invoking downloader on u'https://r1---sn-woc7en76.googlevideo.com/videoplayback?key=yt5&ip=197.214.116
.29&mm=31&itag=22&id=o-AHoWBfHsQ3OS366FvxwwqMNoogvmHJPuNXDFGNLCF31y&ms=au&mt=1416353883&initcwndbps=102500&mv=
m&expire=1416375566&signature=971A3255D8BE0EEE20AE54407311D8ABC3B1A5F4.6D5C71650F42D47225740C0847866E6B204867E
2&upn=w2HpZqhi5YY&sparams=id%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmm%2Cms%2Cmv%2Cratebypass%2Crequiressl%2Csou
rce%2Cupn%2Cexpire&ipbits=0&requiressl=yes&ratebypass=yes&fexp=904846%2C907259%2C916614%2C919330%2C927622%2C93
1350%2C932404%2C936117%2C938692%2C942491%2C943909%2C947209%2C947215%2C947221%2C948116%2C948124%2C952302%2C9526
05%2C952901%2C953912%2C957103%2C957105%2C957201&sver=3&source=youtube'
[download] Destination: Dota 2 Headshot v58.0-0KBFxnK8DEw.mp4
[download] 100% of 60.47MiB in 11:23
[download] Downloading video #5 of 46
[youtube] gOSohcDcws0: Downloading webpage
[youtube] gOSohcDcws0: Downloading video info webpage
[youtube] gOSohcDcws0: Extracting video information
[youtube] gOSohcDcws0: Downloading DASH manifest
[debug] Invoking downloader on u'https://r1---sn-q4f7dnez.googlevideo.com/videoplayback?ip=<IP_ADDRESS>&ms=a
u&mv=m&source=youtube&key=yt5&id=o-ADmSn_Hn6qx-mK2VjeOae6DpPx9NNubltVFQgEgESDk5&fexp=904846%2C907259%2C916614%
2C919330%2C927622%2C931350%2C932404%2C936117%2C938692%2C942491%2C943909%2C947209%2C947215%2C947221%2C948116%2C
948124%2C952302%2C952605%2C952901%2C953912%2C957103%2C957105%2C957201&mt=1416354587&ipbits=0&mm=31&expire=1416
376257&initcwndbps=122500&sver=3&requiressl=yes&sparams=id%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmm%2Cms%2Cmv%2
Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&upn=d7ySmZ3-Dms&itag=22&signature=06863E9FB2DFD9F4382A392E71D
F136C0E761CD8.5B7C212F3745096415B2C6305A996D85A9F79AD3&ratebypass=yes'
[download] Destination: Dota 2 - XMG Captains Draft 2.0 - Evil Geniuses vs Team Secret - Game 1-gOSohcDcws0.mp4
[download]   2.0% of 1.42GiB at 52.62KiB/s ETA 07:40:47 ERROR: unable to download video data: The read operation timed out
Traceback (most recent call last):
  File "youtube_dl\YoutubeDL.pyo", line 1064, in process_info
  File "youtube_dl\YoutubeDL.pyo", line 1040, in dl
  File "youtube_dl\downloader\common.pyo", line 292, in download
  File "youtube_dl\downloader\http.pyo", line 140, in real_download
  File "socket.pyo", line 380, in read
  File "httplib.pyo", line 567, in read
  File "socket.pyo", line 380, in read
  File "ssl.pyo", line 246, in recv
  File "ssl.pyo", line 165, in read
SSLError: The read operation timed out

I captured another instance, verbose output trimmed to the stack trace and single video:
[download] Downloading video #18 of 46
[youtube] kf6rTOQdlLg: Downloading webpage
[youtube] kf6rTOQdlLg: Downloading video info webpage
[youtube] kf6rTOQdlLg: Extracting video information
[youtube] kf6rTOQdlLg: Downloading DASH manifest
[debug] Invoking downloader on u'https://r1---sn-woc7en76.googlevideo.com/videoplayback?id=o-AO8vuBF_tLYuxwlGI
sy3RGIWUQjxK4Xd1J65mh2k7um6&signature=975549B152F69BC0F8B9A8C8A0A99055C46CA706.7D97DA8335EC70CA1532AB85AC6EDE0
0BB91A7CB&key=yt5&mm=31&fexp=907259%2C914075%2C924637%2C927622%2C932404%2C936118%2C937432%2C938689%2C943909%2C
947209%2C947215%2C948124%2C952302%2C952605%2C952901%2C953912%2C957103%2C957105%2C957201%2C959201&ip=197.214.11
6.29&requiressl=yes&ms=au&source=youtube&mv=m&ipbits=0&initcwndbps=212500&ratebypass=yes&sver=3&upn=f-ocBNeD8a
I&expire=1416417705&sparams=id%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmm%2Cms%2Cmv%2Cratebypass%2Crequiressl%2Cs
ource%2Cupn%2Cexpire&mt=1416396074&itag=22'
[download] Destination: Dota 2 Symphony of Skills 55-kf6rTOQdlLg.mp4
[download]  44.8% of 47.30MiB at 23.91KiB/s ETA 18:39 ERROR: unable to download video data: The read operation timed out
Traceback (most recent call last):
  File "youtube_dl\YoutubeDL.pyo", line 1064, in process_info
  File "youtube_dl\YoutubeDL.pyo", line 1040, in dl
  File "youtube_dl\downloader\common.pyo", line 292, in download
  File "youtube_dl\downloader\http.pyo", line 140, in real_download
  File "socket.pyo", line 380, in read
  File "httplib.pyo", line 567, in read
  File "socket.pyo", line 380, in read
  File "ssl.pyo", line 246, in recv
  File "ssl.pyo", line 165, in read
SSLError: The read operation timed out

Can confirm the issue.

I was just searching to see if this issue already existed before reporting it. I can confirm it happens for me as well
Owner@sehrbook /cygdrive/e/Videos
$ youtube-dl.exe --retries="infinite" --verbose 'https://www.facebook.com/100006878506221/videos/1679312832307984/'
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--retries=infinite', u'--verbose', u'https://www.facebook.com/100006878506221/videos/1679312832307984/']
[debug] Encodings: locale cp1252, fs mbcs, out None, pref cp1252
[debug] youtube-dl version 2016.01.09
[debug] Python version 2.7.10 - Windows-8-6.2.9200
[debug] exe versions: none
[debug] Proxy map: {}
[facebook]<PHONE_NUMBER>307984: Downloading webpage
[debug] Invoking downloader on u'https://video-iad3-1.xx.fbcdn.net/hvideo-xfp1/v/t43.1792-2/12451028_953909387979006_2090296038_n.mp4?efg=eyJ2ZW5jb2RlX3RhZyI6InN2ZV9oZCJ9&oh=f18edad53ab719c011075dd496430c07&oe=5691EE97'
[download] Resuming download at byte 2167250
[download] Destination: Un p'tit Jive, ce rock Am▒ricain tr▒s vivant, et en plein Paris svp, sur l'es...-1679312832307984.mp4
[download]  31.5% of 9.90MiB at  8.38KiB/s ETA 13:48   ERROR: unable to download video data: ('The read operation timed out',)
Traceback (most recent call last):
  File "youtube_dl\YoutubeDL.pyo", line 1609, in process_info
  File "youtube_dl\YoutubeDL.pyo", line 1551, in dl
  File "youtube_dl\downloader\common.pyo", line 342, in download
  File "youtube_dl\downloader\http.pyo", line 162, in real_download
  File "socket.pyo", line 384, in read
  File "httplib.pyo", line 612, in read
  File "socket.pyo", line 384, in read
  File "ssl.pyo", line 734, in recv
  File "ssl.pyo", line 621, in read
SSLError: ('The read operation timed out',)

This is exactly the sort of situation retrying is supposed to handle. However, it appears that http.py isn't expecting SSLError to be recoverable, when often they are. Not all SSLError will be cryptographic problems, but it appears the only nod to this is that the catch of socket.error (SSLError's superclass) retries ECONNRESET.
I would recommend handling SSLError separately, since only the subtype CertificateError is unrecoverable (i.e. not network-based). Rather than handling SSLError as a socket.error, which assumes that the only recoverable socket error is ECONNRESET, the fact that many SSLError are transient network problems should allow a more nuanced handling of of SSLError.
Specifically, the OpenSSL docs state explicitly that the following errno are retriable ("The TLS/SSL I/O function should be called again later."):
SSL_ERROR_WANT_READ, SSL_ERROR_WANT_WRITE, SSL_ERROR_WANT_CONNECT, SSL_ERROR_WANT_ACCEPT, SSL_ERROR_WANT_X509_LOOKUP, SSL_ERROR_WANT_ASYNC
SSL_ERROR_SYSCALL is possibly retriable, but the Python stdlib provides no way to inspect the error more deeply. Still, this should probably be retried by default as long as retries != infinite, since if it's not retriable (say EMFILE at the OS level), it will fail quickly and run through the retries with no harm done.
SSL_ERROR_SSL is the only one that is probably not retriable, since it would likely be a crypto problem due to mismatched certs or a misconfigured server. Still, the error has enough detail that it is (I believe) broken out into a couple different exception types in Python, one of which being CertificateError.

youtube-dl obviously has a retry mechanism.
[download]  79.2% of ~9.08GiB at  1.68MiB/s ETA 03:12[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 10 of 10)...
[download] Skipping fragment Frag6952...

It just doesn't catch everything
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/hls.py", line 115, in real_download
    success = ctx['dl'].download(frag_filename, {'url': frag_url})
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/youtube_dl/utils.py", line 1034, in https_open
    req, **kwargs)
  File "/usr/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 734, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 621, in read
    v = self._sslobj.read(len or 1024)
SSLError: ('The read operation timed out',)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/hls.py", line 115, in real_download
    success = ctx['dl'].download(frag_filename, {'url': frag_url})
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/youtube_dl/utils.py", line 1034, in https_open
    req, **kwargs)
  File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
URLError: <urlopen error timed out>

For now, as a workaround, you can do something like the following in the terminal. The loop attempts to run the command in the getvids function up to 10 times or until it finishes successfully.
`
function getvids() {
youtube-dl --playlist-items 1-3 https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re
}
for i in {1..10}; do getvids && break || sleep 15; done
`

jeffymahoney how to run this loop? I got SyntaxError: invalid syntax

@daewizal That's not Python that he's using. If you're on windows, you can create a batch script to "retry". For example, put the following in a file called youtube-dl-retry.cmd
:retry youtube-dl.exe --verbose --retries 999 https://www.youtube.com/watch?v=YkygAoXqP4A IF NOT "%ERRORLEVEL%"=="0" GOTO :retry @echo COMPLETE, ERRORLEVEL = %ERRORLEVEL% pause
This will keep on running the youtube-dl command until it succeeds.

@IGTHORN thank you!

@IGTHORN : could you also please provide shell script for same youtube-dl-retry.sh?

@nipunsadvilkar The script posted by jeffymahoney is a shell script. You can create a file called youtube-dl-retry.sh and paste in his script. Just change the line with youtube-dl so that it contains the video you want to download. Jeff's script will only retry 10 times, so if you want it to retry more than that, change the for loop line to: for i in {1..1000}; do getvids && break || sleep 15; done

@IGTHORN , @jeffymahoney : Thank you guys! :+1:


Replace pip3 to python3 -m pip

Here: https://github.com/docker-library/python/blob/master/Dockerfile-alpine.template#L76

We refactored the linked section with https://github.com/docker-library/python/pull/194.

Currently I use my custom version of Python Docker file (https://dpaste.de/8Xps), I try to stay close to original version..

Alpine 3.5
upgrade packages before processing
install and configure flake8 and some tools like pylint, mypy and sphinx,..

I fixed some bug with wget: can't execute 'ssl_helper': No such file or directory, as I see, it's similar too https://github.com/Yelp/dumb-init/issues/73 ... and previous wget -0 ... completed without errors, so I added get-pip.py to 46 line close to other wget commands, before deleting .fetch-deps and it works now... https://dpaste.de/8Xps#L46,74
For some reason your code works in your version but doesn't in my. May be because of modern Alpine version or because of it is in different RUN block...
I use one single RUN block to minimize count of layers of image...
Not sure about replacing && to ;, in future I going to use Ansible for building images (https://www.ansible.com/ansible-container), but not sure, it will work with Docker Cloud/Hub build system (but it may work with https://docs.docker.com/docker-cloud/builds/advanced/#override-build-test-or-push-commands)...

wget: can't execute 'ssl_helper': No such file or directory

This is indeed due to the Alpine version -- the newer Alpine releases no longer require libressl or CA certificates installed in order to fetch SSL web content.
Given that this doesn't appear to be an issue with the image we maintain (and thus not something we can really fix), I'm going to close. :+1: