Do not erase signature constraints when calculating variance

Fixes #55671
As noted in the linked issue, this is, unfortunately, probably in-use to mask a bunch of real issues (or as a workaround for other issues) in the wild, since it was an easy to make a type that would "accept" any assignments between instances of the same alias, even though once you did a structural comparison it only accepted near-exact matches. Even in our test suite it comes up a couple times in some examples we've pulled from the wild - I expect DT and top100 to have more.

@typescript-bot pack this
@typescript-bot user test this
@typescript-bot test this
@typescript-bot run dt
@typescript-bot perf test this

DT reported as "a lot of problems", but it's just a lot of packages depending on immutable (which has the same extends error as in the test here - so I should probably figure out where an any needs to be a never (or vice-versa) and fix it), and dependencies on fp-ts (which, just like with the user suite, is an array subtype we now measure as invariant instead of covariant (which you can work around with an out annotation on the type parameter, as I've done in our own API). There's also a few instances of the propTypes circularity that's now found in the tests (usually via styled-components, which is where the test comes from), too. Not really sure what the workaround for that one should be yet, since the types are quite complicated. And lastly, there's some dependencies on a package called clownface, which also is detected as having an incorrect extends now (albeit much less deeply nested than immutable!). There's a few others, too (which are usually array or collection subtyping issues caused by more accurate variance calculation - zod and bluebird come to mind), but those are the ones that infect a lot of packages on DT by proxy.
I think this is a change we can take, once we get those big offenders fixed. Though the array subtype issue might warrant some special-casing in code, even though you can work around the issue with an annotation; arrays are forced to be covariant, whereas if we structurally inspect them (like we do their subtypes), they're measured as invariant (which is more correct! just.... harder to use). It's probably worth detecting array subtypes and short-circuiting their variance calculations, so subtypes have the same hardcoded-covariant behavior array itself does. That change can probably stand on its' own, too, since it's likely performance-positive w.r.t. variance calculation for array subtypes (unlike this change, which can only be bad for perf, since it forces us to make more comparisons with a stricter variance result).

It’s interesting to me that method bivariance alone isn’t enough to keep arrays covariant (though it does help with user-defined arraylikes, as pointed out by Ryan in the array subtype covariance PR)


Act now date range text color is confusing

summary:
The act now date ranges are bolded in orange. The hospital overload is also orange. User are seeing the two and expecting the topic and are confused when the highlight different sets of dates.
expected result:
Same colors address same data point/ topic.
actual result:
Confusion due to different topics using the same color
environment (device, browser, mobile / web):
Web
link:
All states but specifically MO
https://covidactnow.org/state/MO
screenshot (you can copy/paste images directly in):

agreed - same as this one:
https://github.com/covid-projections/covid-projections/issues/367

closing as tracked in the other issue


nothing about filtering groups

there's nothing in these docs about how filtering groups work and how to construct an and over or type setup

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: aeeb3695-9131-3598-c00b-0ae63fa6f33b
Version Independent ID: 203d91ee-3831-0149-19e2-118de41ca5ae
Content: Get started with notifications in Azure DevOps - Azure DevOps
Content Source: docs/notifications/about-notifications.md
Product: devops
Technology: devops-collab
GitHub Login: @chcomley
Microsoft Alias: chcomley

@drdamour  Thank you for the feedback, assigning this to the author for review.
In the meantime, please refer to Notifications and Subscriptions general description on the Events and notifications page.

Nothing on that page about filtering

Hi @drdamour, I'm going to assume that you may have moved on at this point, but wanted to leave a link to more information, just in case! Please reference the following article, Manage notifications for a team or group and you can see how to Follow a user story, bug, issue, or other work item or pull request. Thank you and please @mention me if you have any further questions.

@chcomley no havent moved on. Neither of those links describe how filter groups function from what i can tell. So still open issue

@drdamour, I've just followed up with the PM, so I hope to be in touch shortly. Thanks for your patience. Hope you're having a great day.

Hi @drdamour, Can you please further clarify what you're trying to accomplish? Thank you.

@chcomley in the UX there's this option about grouping filters..i assume it has something to do with and's and ors, but i can't figure it out

went to read the docs on it...but nothing there

@chcomley in the UX there's this option about grouping filters..i assume it has something to do with and's and ors, but i can't figure it out

went to read the docs on it...but nothing there

Hi @drdamour, This works the same as a query. You should be able to find the information you need here: https://docs.microsoft.com/en-us/azure/devops/boards/queries/using-queries?view=azure-devops#group-clauses.

Hi @drdamour, This works the same as a query. You should be able to find the information you need here: https://docs.microsoft.com/en-us/azure/devops/boards/queries/using-queries?view=azure-devops#group-clauses.

indeed...seems like this page https://docs.microsoft.com/en-us/azure/devops/notifications/about-notifications?view=azure-devops or maybe more appropriately https://docs.microsoft.com/en-us/azure/devops/notifications/concepts-events-and-notifications?view=azure-devops#notifications should have "learn more about filtering" links in them to the page you just linked me.
thanks!

indeed...seems like this page https://docs.microsoft.com/en-us/azure/devops/notifications/about-notifications?view=azure-devops or maybe more appropriately https://docs.microsoft.com/en-us/azure/devops/notifications/concepts-events-and-notifications?view=azure-devops#notifications should have "learn more about filtering" links in them to the page you just linked me.
thanks!

@drdamour , I'll update the article with links. Thank you for your feedback, much appreciated.

@drdamour , I'll update the article with links. Thank you for your feedback, much appreciated.


➖ Remove unused dependencies

#118  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @mingi3314 and the rest of your teammates on  Graphite


Relax lifetime requirements for FdSet::{insert, remove, contains}

Fixes #2130

cc @SteveLauC

Seems that these functions should also take a BorrowedFd, just like #2134


stripe namespace

with stripe library 2.1.1 the namespace causes in lib/Stripe.php causes the Stripe class to not be declared and throws an exception. Further more, if you stop using App::import and include the file it then errors saying you can not redeclare the Strip class. I've solved this by removing the namespace, I do not know if this is the correct method of fixing this issue, but it works.

@jijarine : Same way I have resolved integration issue, Have you tried 'retrieveEvent();'??

No, can you explain more?

After trying to use this component I found it was simpler to just include the Stripe library and execute the API directly instead of using the component. With doing this I no longer have issues with namespaces.

I just updated the component for support Stripe API 3.40 / 2015-10-16, so, now the namespace issue should be resolved.
https://github.com/hashmode/CakePHP-Stripe-Component-Full/commit/8adadf380834c6e84d2f74b2f82ead19e0ee5307


[Hub Generated] Review request for Microsoft.Consumption to add version stable/2019-10-01

This is a PR generated at OpenAPI Hub. You can view your work branch via this link.
Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Please follow the link to find more details on PR review process.

/azp run automation - sdk

/azp run automation - sdk

@zachraMSFT
LGTM. You might want to check why .NET SDK fails.
Then let me know when you want it merged.

/azp run automation - sdk

@weidongxu-microsoft You can merge now. Made a whitespace change to rerun because build was too old and it passed now, so looks like a temporary error. Same thing happening for Trenton now, it's a 401.


Support Swift Package Manager

#77

Bump on this PR. Is any able to review and merge this? Maybe @swolchok or @Megra? Thanks!


Please confirm these before moving forward

[x] I have searched for my issue and not found a work-in-progress/duplicate/resolved issue.
[x] I have not been informed if the issue is resolved in a preview version of the winget client.

Category of the issue
Installer hash mismatch.
Brief description of your issue
Hash verification fails.  Running as admin.
[CLI ] Package hash verification failed. SHA256 in manifest [d5efbcb34f1dee32e6912251678e0177415181bc7c4707b76d466c232b01fabd] does not match download [75a7ac72bbf192a70606ce92f3a2fc3f61f52b1e7ca572e76b3199da20fe41e7]
Steps to reproduce
winget install --id Microsoft.PowerAutomateDesktop --silent --exact --accept-source-agreements --accept-package-agreements
Actual behavior
The file downloads but fails hash verification.  This was working yesterday and fails today, though the package is still the same version as yesterday.
Expected behavior
Ideally it would pass the hash verification :)
Environment
Windows Package Manager v1.8.1911
Copyright (c) Microsoft Corporation. All rights reserved.
Windows: Windows.Desktop v10.0.22631.4169
System Architecture: X64
Winget Directories
Logs                               %TEMP%\WinGet\defaultState
User Settings                      %LOCALAPPDATA%\Microsoft\WinGet\Settings\defaultState\settings.json
Portable Links Directory (User)    %LOCALAPPDATA%\Microsoft\WinGet\Links
Portable Links Directory (Machine) C:\Program Files\WinGet\Links
Portable Package Root (User)       %LOCALAPPDATA%\Microsoft\WinGet\Packages
Portable Package Root              C:\Program Files\WinGet\Packages
Portable Package Root (x86)        C:\Program Files (x86)\WinGet\Packages
Installer Downloads                %USERPROFILE%\Downloads
Links
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale
Admin Setting                             State
LocalManifestFiles                        Disabled
BypassCertificatePinningForMicrosoftStore Disabled
InstallerHashOverride                     Disabled
LocalArchiveMalwareScanOverride           Disabled
ProxyCommandLineOptions                   Disabled
DefaultProxy                              Disabled
Screenshots and Logs
No response

Fixed by #180840.

The winget publishing pipeline has been having issues the past few days. See #180404. It should be working again soon.

Verified this is working again:
Downloading https://download.microsoft.com/download/3/e/4/3e401e87-e96f-4527-bfdf-bf25e370f15a/Setup.Microsoft.PowerAutomate.exe
  ██████████████████████████████   313 MB /  313 MB
Successfully verified installer hash
Installer downloaded: C:\Users\...\Downloads\Microsoft.PowerAutomateDesktop_<IP_ADDRESS>78\Power Automate for desktop_<IP_ADDRESS>78_Machine_X86_burn_en-US.exe

Fixed by Automatic update of Microsoft.PowerAutomateDesktop from <IP_ADDRESS>61 to <IP_ADDRESS>78 #180840.

The winget publishing pipeline has been having issues the past few days. See #180404. It should be working again soon.

Since this version appears to have become published, I'll assume this Issue is complete. If this is in error, please @ me and I"ll reopen.
PS C:\ManVal> find-WinGetPackage Microsoft.PowerAutomateDesktop

Name                       Id                             Version        Source
----                       --                             -------        ------
Power Automate for desktop Microsoft.PowerAutomateDesktop <IP_ADDRESS>78 winget


How do you symbolicate Dart AOT-compiled binary? / How do you draw a Dart (FFI) flamegraph?

Hi thanks for the language and SDK! I am trying to profile the code (across FFI boundary) and draw a flamegraph.
I did a simple experiment using https://github.com/flamegraph-rs/flamegraph, with AOT binary (dart --enable-experiment=native-assets build) and get something like:

As you can see, the Rust symbols are good (the ones in libfrb_example_pure_dart.dylib). However, most symbols are missing, which I guess are the symbols that should be in the Dart AOT binary.
Since profiling some Dart AOT code is surely a common need when developing Dart itself, and I cannot find doc about this, I wonder what do you do to symbolicate Dart AOT-compiled binary? Or, how do you profile and draw a Dart (FFI) flamegraph? Thanks for any suggestions!
P.S. Also no luck if using aot compile: dart --enable-experiment=native-assets compile aot-snapshot, sudo flamegraph dartaotruntime benchmark/simple_benchmark.aot

The dart compile aot-snapshot should produce an ELF file that has static symbols in it. When I build the SDK at main and run
% cat hello.dart
main() {
  while (true) {}
  print("hello");
}

% dart compile aot-snapshot hello.dart
Generated: /.../hello.aot

% perf record -g dartaotruntime hello.aot
...
<ctrl-c>
% perf report --no-children
-   99.99%  DartWorker       hello.aot             [.] main                                                                                ▒
   - main                                                                                ◆
      - _Closure.call                                                                                ▒
        _delayEntrypointInvocation.<anonymous closure>                                                                                ▒
        _Closure.call                                                                                ▒
        _RawReceivePort._handleMessage                                                                                ▒
        stub InvokeDartCode                                                                                ▒
        0x5616e3df5d32                                                                                ▒
        0x5616e3df74df

This won't work for dart compile exe because the produced executable ELF file has another ELF file embedded inside it (the app isn't statically linked together with the aot runtime).

Thank you very much! It successfully works on Linux.
P.S. MacOS still does not work yet, checking what is going on (maybe dtrace system integrity protection problem)
(base) ➜  hello_dart dart compile aot-snapshot bin/hello_dart.dart
Generated: /Users/tom/temp/hello_dart/bin/hello_dart.aot
(base) ➜  hello_dart sudo flamegraph -- ~/temp/dartaotruntime bin/hello_dart.aot
Password:
dtrace: system integrity protection is on, some features will not be available

dtrace: description 'profile-997 ' matched 1 probe

So, has anyone succeeded in profiling and getting symbols in MacOS before? Or, is there any ways to let .aot file generate symbols suitable for MacOS?
Thanks!

So, has anyone succeeded in profiling and getting symbols in MacOS before? Or, is there any ways to let .aot file generate symbols suitable for MacOS?

The way to achieve this is to tell our AOT compiler to emit assembly and then compile that manually to a MachO file.
To avoid requiring users to have an assembler/compiler/linker to be installed, the dart build / dart compile tools use ELF on all platforms and there doesn't seem to be an option to change this behavior.
Though all tools are there. On linux I can do that with
% export DART_SDK=<path-to-dart-sdk>
% $DART_SDK/bin/dartaotruntime $DART_SDK/bin/snapshots/gen_kernel_aot.dart.snapshot --platform=$DART_SDK/lib/_internal/vm_platform_strong.dill --aot --tfa -o hello.dill hello.dart
% $DART_SDK/bin/utils/gen_snapshot --snapshot-kind=app-aot-assembly --assembly=hello.S hello.dill
% perf record -g $DART_SDK/bin/dartaotruntime hello.so
% perf report --no-children

=> On MacOS this should also work by using the mac compiler/linker to generate a dylib file

Thank you! I will try that assembly approach later.

Although one can kind of work around the issue manually, this is a valid feature request. Filed https://github.com/dart-lang/sdk/issues/54254 to track this.

Thank you!


Visual Issue with Ways

I found this visual issue that seems to randomly appear. I have no idea how to recreate this issue, it happens at random. Dragging the map seems to fix it.

Yes, I've been seeing a lot of this too, both in iD and RapiD.  I'm pretty sure it's a rendering bug that showed up in Chrome 94.

@bhousel do you think this is related to the fact that icons are occasionally rotated by 90deg, or is that a separate bug?

@bhousel do you think this is related to the fact that icons are occasionally rotated by 90deg, or is that a separate bug?

I don't know -  I haven't seen that icon rotation bug yet 😆
They're really different places in the code, so my gut says the issues are not related.

I'm having a similar problem with green areas randomly showing up in the editor. I can't reproduce it in Firefox though. So it's pretty likely that Chrome is the culprit.
https://www.openstreetmap.org/#map=17/39.06914/-122.17400

More info, these issues are present on both Chrome and Edge, so it's something to do with the Blink rendering engine.
The rotated icon (use path) appears to be this issue:
https://bugs.chromium.org/p/chromium/issues/detail?id=1257978
I can't find an open issue for the road fading effect, so I opened one here:
https://bugs.chromium.org/p/chromium/issues/detail?id=1260092

I'm closing this as it seems that the fix made it into Chrome 96.


Capture next / previous mouse buttons with JS

Currently if I use the next / prev mouse buttons it will reload the page. Ideally I'd like the javascript level to catch this and go to the previous state. Such as from the current queue to album overview where I was last at.

If any developer wants to do this, this can be done using window.history.pushState

Nope, that would be vue-router. I don't think this is necessary though - when was the last time you navigated back/forth in iTunes, if at all?

I use google music and this can come in handy actually.

I requested it because this is what I actually did and found myself sent to a previous url, which means the queue was cleared and music stopped. More exactly:

Navigated to artists, clicked play on "Hybrid Minds".
Found myself in the current queue.
Pressed previous because I wanted to to add "Ivy Lab" as well.
Lost current queue and went to previous page in browser history.

Spotify also has support for the next and previous button, both in the web player and the desktop client.

Sorry guys, for keeping you waiting (if you do at all!) But the feature is now available on v3.1.0. Cheers!


Can't use .card when I use npm

Hi, I use npm to include wingcss, but .card class can't be used. I find the source code in node_modules/wingcss/dist/wing.css, I can't see any class about .card.

This is because cards will be released in v1.0.0, for now, you can copy the files from this repo's dist folder.


⚠ SceneTime has degraded performance

In 22b0d21, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1038 ms

Resolved: SceneTime performance has improved in 5d446a4 after 23 minutes.


DASHBUILDE-75: Data set API support for non-CDI containers

This PR is a refactor of the data set subsystem in order to make it runnable under non-CDI containers just as any other plain java library. This includes the ability to register data set definitions as well as to execute data lookup calls against those definitions. This makes possible to embed the data set engine into other java applications and runtimes without the need of a CDI environment.
(This non-cdi support has been required by the KIE execution server)
Instructions on the Dataset API usage can be found here. I've also prepared a project template that can be helpful for those who want to play a bit with the API (do not forget to build dashbuilder from this PR first, otherwise the template project build will fail).

looks perfect to me. Good and clear separation and the code module (api and shared) has very few dependencies and no cdi deps for core.
+1

We agreed with @mbiarnes  to hold on the headers replacement until this PR is fixed in order to avoid too many conflicts. That means it's not necessary to set the headers correctly as part of this PR since that will be entirely fixed later on.


Fixing checkstyle issues in cosmos package

Added suppressions for checks that are not handled at the moment
Addressing #8681 https://github.com/Azure/azure-sdk-for-java/issues/8681
Enabling failsOnError and failOnViolation on cosmos pom
Adding basic javadoc at places, this has to be redone with proper docs

/azp run java - cosmos - tests

/azp run java - cosmos - tests


adding new feature : pagination

v0.3.1
Added New Feature:

Pagination Support flag = --pages, takes current terminal viewport into consideration.

v0_3_1


docker: remove docker- prefix from containerd and runc

Motivation for this change
Latest Docker changed the required filenames for the containerd and runc binaries as can be seen at docker/docker-ce@f30fc76.
This commit drops the docker- prefix
I have run locally the docker test and finishes successfully again.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

Seems to be working fine now, built and tested locally.

Looks like it's in master at 68c3e5f38bc99c7e87b721ea9b11a24403a85204 now?

Indeed it is.


bug: topic writer stops connections retry on GENERIC_ERROR

Bug Report
YDB GO SDK version:

Environment

Current behavior:

Writer dead and does not try to reestablish connection
failed to read grpc message from writer stream: operationStatus error: GENERIC_ERROR: [{#500100 pipe to partition's tablet is dead}] at `github.com/ydb-platform/ydb-go-sdk/v3/internal/conn.(*grpcClientStream).RecvMsg(grpc_client_stream.go:113)` at `github.com/ydb-platform/ydb-go-sdk/v3/internal/grpcwrapper/rawtopic/rawtopicwriter.(*StreamWriter).Recv(streamwriter.go:45)` at `github.com/ydb-platform/ydb-go-sdk/v3/internal/topic/topicwriterinternal.(*SingleStreamWriter).receiveMessagesLoop(writer_single_stream.go:179)` at `github.com/ydb-platform/ydb-go-sdk/v3/internal/topic/topicwriterinternal.(*WriterReconnector).Write(writer_reconnector.go:197)`

Expected behavior:

Errors should be retried
Steps to reproduce:

Related code:

insert short code snippets here

Other information:

It is expected behavior from SDK: GENERIC_ERROR is non retriable error: https://ydb.tech/en/docs/reference/ydb-sdk/error_handling.
But it is not expected from server side - tablet die must raise retriable error.
Server side guys working for fix it in internal ticket LOGBROKER-7989


Wave V Icons

I'll get started soon on these....

Thanks!

x-wing-ships-SLK-OctWave5.ttf.zip
Nantex is semi-colon ; and BTL-B is the colon :

I'm not sure how quickly they can be done, but I am guessing Gold Squadron uses this font for the stream ship icons.. it bugged me onstream that these were missing.  LOL

Have updated stream.tabletop.to - so GSP should have them on stream now. Thanks!

Thank you much!

You bet.  Thanks for getting that pushed out so quickly.   Looks good onstream.  If you want any additional glyphs for the app let me know.

Dbxau, would you be able to have the overlay app place maneuver template shapes or stress/focus/tractor icons during their commentary? I can make glyphs to help with that.   Email me if I can help in any way<EMAIL_ADDRESS>

CSS updated in ba60fe035b2857d4cc8e01bb0ba68d009ff7c21f


std::list didn't compile in C++Builder.

The class QPDFOutlineObjectHelper contains a private member class storing some data and containing the following line:

std::list<QPDFOutlineObjectHelper> kids;

That line doesn't compile in my C++Builder 10.2 because the type is not fully defined at the time it's needed for the list. Not sure why this works elsewhere, but it's a known limitation in theory:
http://www.drdobbs.com/the-standard-librarian-containers-of-inc/184403814
http://codeverge.com/embarcadero.cppbuilder.cpp/e2450-undefined-structure/1053917
https://stackoverflow.com/a/45554279/2055163
Because you are using PointerHolder at the same place for parent already, I simply suggest moving to that for the std::list as well. Because things get pretty long and difficult to maintain this way, I introduced two new typedefs shortening names a lot and making future replacements easier. At some places method signatures now even fully align! ;-)
I didn't care about breaking API/ABI-changes, but didn't get anything else to work as well. So if you have another solution, please let me know. Thanks!

Yes, I can't take this because of the API/ABI breakage. I'll look into it a little more and see if I'm doing something that's actually wrong.

I'm going to go ahead and close this, but if you find a good solution, please feel free to report. Thanks.

Yes, I can't take this because of the API/ABI breakage.

Just recognized that you are intending to release QPDF 9 as new major release next: Why is breaking the API/ABI a problem with a new major release for you? What's your overall policy on this, never change anything to become backwards incompatible at all?

My policy is that breaking ABI causes a major release (basically semantic versioning). I like to do that as little as possible because it is disruptive to packagers as it makes people have to relink, but when necessary, I will break ABI and do a major release. qpdf 9 breaks ABI, so now is an excellent time for ABI-breaking changes, if any.
My policy for API is that I try to avoid ever breaking the API because that causes work for everyone who uses qpdf. In the past, I have broken API only when there was no good way to avoid it, and most of the time, API breakage has been on minor or advanced parts of the API.
There may come a time when I drop support for non-C++11 and consider something radical like dumping PointerHolder in favor of C++11 auto pointers, but I haven't quite gotten that far. It's also conceivable that I could break API for performance reasons. For example, there are methods right now that allow exposing an Array QPDFObjectHandle's elements as a std::vector and a Dictionary QPDFObjectHandle's elements as a std::map. If I change the underlying representations for performance reasons, which is a distinct possibility, I would necessarily remove those. Also, the worst API mistake in all of qpdf is returning a reference to the internal pages cache, and I may sometime break that. It won't be for qpdf 9 though most likely. When I break API, I always enumerate the changes in the release notes. If I break API in a big way, like killing PointerHolder, I will most likely provide some non-default approach to aid in the transition, such as allowing QPDF to be built with a preprocessor symbol that provides a backward-compatible API or something.
qpdf 9 has a few API breakages, but all for cases where the old API simply didn't work or for dark corners that normal users wouldn't touch. Breaking API for deep power user interfaces is less problematic because those people are better equipped to change their code and because it affects fewer people.

If ABI/API-changes are possible, then what's the reason of not accepting this PR for release 9? It surely sounds and looks more difficult than what it actually is. I only did two things:

Using PointerHolder where it wasn't used before to fix some compiler issue, which you seem to are using in most places already anyway.

This propagated up the call stack and slightly changed some method signatures. To make those things easier in future, additional typedefs have been introduced.

The main changes are the following two lines only:
typedef PointerHolder<QPDFOutlineObjectHelper> Ptr;
typedef std::list<QPDFOutlineObjectHelper::Ptr> List;

https://github.com/qpdf/qpdf/pull/297/commits/2fa043360b18941a51529d542a1b6f5f80892880#diff-6cebeb8348d32fdf02fb4cd2ed4da6bcR40
In my opinion, the code is easier to read and maintain now. Existing users don't need to change much as well, if at all, only some additional dereferences or such.
It would be great if you could reconsider your decision, so that I don't need to revert my changes and find another solution. Thanks!

I don't want the extraneous typedefs. They are inconsistent with the rest of the library. However, I'll look more closely at the changes and possibly select some of them and ask you to retest.

If the typedefs are just cosmetic, you would improve your odds by resubmitting without the List and Ptr typedefs so I can see the real changes. :-)

Will do so.

Removed the typedefs as requested. In the end, the compiler only fails at the following line. Using a PointerHolder there fixes the error, but leads to all the other signature changes. :-/
std::list<QPDFOutlineObjectHelper> kids;

https://github.com/qpdf/qpdf/commit/813a37f3bafa416688cc051d7593320f53078c91#diff-6cebeb8348d32fdf02fb4cd2ed4da6bcR116
The following commit implemented another approach avoiding the signature changes by copying kids into a backwards compatible structure, but not sure if it's worth it.
https://github.com/ams-tschoening/qpdf/commit/ffa8b82e369726016113ffbc4cacd9ec285b8ca5
I guess there's something wrong with the forward declaration and including of QPDFOutlineObjectHelper vs. QPDFOutlineDocumentHelper for my compiler, but I couldn't get anything else to compile.

I don't understand why this is necessary. The type of QPDFOutlineObjectHelper is fully defined, so the article you referenced doesn't seem to be relevant, and I really hate to use lists of PointerHolders when the whole notion of these helpers is they are copiable in this way. I wonder whether it works if I use vector instead of list. Have you tried that? At a glance, I don't see that it would change any of the code. Nothing is depending on any capability of list that isn't there in vector. It's possible that the queue initialization may change if queue.insert won't work with vector iterators, but that would just be making a for loop...
I'd rather see list -> vector than having to introduce another level of indirection.

I didn't test with std::vector because I wouldn't have expected to make any difference... But it does, std::vector fixes the problem as well for some reason. So I removed usage of PointerHolder once more and changed to std::vector wherever necessary. Doing that I saw you were using vectors already at multiple places, maybe for similar reasons?

I am good with this change. The PR fails CI because of fuzzer issues. However, I am willing to take the changes manually, fix the other issues, and update the docs. This is an API change, but it is worth doing to work around this issue. I will take it for 9.0.

I have incorporated your changes into my work branch. If everything goes well, I will push to master this evening and manually close this issue. Thanks for your work in investigating this and making my job easier.

Fixed in master.

Thanks for merging!


Add support for running docker-ce from a snap (Fixes: #9)

This adjusts the following:

Systemd unit dependencies
Location of docker daemon (/usr/bin/ -> /snap/bin)
Default data storage location
Configuration storage location

It allows optionally using both 'jq' and 'docker' from the snaps
provided by the Ubuntu server 18.04 installation process.

Why you add an x ?

I've seen this uses in autotools (configure.ac) and bash scripts in the past.  I suppose it's related to potentially having a bad expansion that includes something like a backspace character?  If you would prefer I remove it, i'll remove it.

I don't think with quota we run into a problem. Please remove it, so we use the same style over all files. Thanks

Please remove it, so we use the same style over all files. Thanks

OK adjusted, thanks.


Make the engine thread pool configurable

Ideally number of threads need to be calculated and spawned intelligently based on the environment and runtime status (CPU Usage, memory, etc).
A configuration parameter which is user configurable is also required.

Duplicate issue at #2709.


DNS Misconfigured or Unbound Problem

I'm using Unbound 1.13.1 with DNSSEC enabled, and using CloudFlare DNS servers.  I am not able to resolve www.cdc.gov
It fails because Missing DNSKEY RRset in response to DNSKEY query
When I dig +dnssec www..cdc.gov
`C:\Users\M>dig +dnssec www.cdc.gov
; <<>> DiG 9.16.8 <<>> +dnssec www.cdc.gov
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 9269
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;www.cdc.gov.                   IN      A
;; Query time: 0 msec
;; SERVER: <IP_ADDRESS>#53(<IP_ADDRESS>)
;; WHEN: Fri Feb 19 15:37:32 Mountain Standard Time 2021
;; MSG SIZE  rcvd: 40`
Looking at the Unbound log, I see this:
`
Feb 19 15:38:33 unbound[550:2] info: resolving www.cdc.gov. A IN
Feb 19 15:38:33 unbound[550:2] info: response for www.cdc.gov. A IN
Feb 19 15:38:33 unbound[550:2] info: reply from <.> 2606:4700:4700::1111#853
Feb 19 15:38:33 unbound[550:2] info: query response was CNAME
Feb 19 15:38:33 unbound[550:2] info: resolving www.cdc.gov. A IN
Feb 19 15:38:33 unbound[550:2] info: response for www.cdc.gov. A IN
Feb 19 15:38:33 unbound[550:2] info: reply from <.> 2606:4700:4700::1111#853
Feb 19 15:38:33 unbound[550:2] info: query response was ANSWER
Feb 19 15:38:33 unbound[550:2] info: validate(cname): sec_status_secure
Feb 19 15:38:33 unbound[550:2] info: resolving akam.cdc.gov. DS IN
Feb 19 15:38:33 unbound[550:2] info: validated DS akam.cdc.gov. DS IN
Feb 19 15:38:33 unbound[550:2] info: resolving akam.cdc.gov. DNSKEY IN
Feb 19 15:38:33 unbound[550:2] info: Missing DNSKEY RRset in response to DNSKEY query.
Feb 19 15:38:33 unbound[550:2] info: resolving akam.cdc.gov. DNSKEY IN
Feb 19 15:38:34 unbound[550:2] info: response for akam.cdc.gov. DNSKEY IN
Feb 19 15:38:34 unbound[550:2] info: reply from <.> 2606:4700:4700::1111#853
Feb 19 15:38:34 unbound[550:2] info: query response was nodata ANSWER
Feb 19 15:38:34 unbound[550:2] info: Missing DNSKEY RRset in response to DNSKEY query.
Feb 19 15:38:34 unbound[550:2] info: resolving akam.cdc.gov. DNSKEY IN
Feb 19 15:38:35 unbound[550:2] info: response for akam.cdc.gov. DNSKEY IN
Feb 19 15:38:35 unbound[550:2] info: reply from <.> 2606:4700:4700::1001#853
Feb 19 15:38:35 unbound[550:2] info: query response was nodata ANSWER
Feb 19 15:38:35 unbound[550:2] info: Missing DNSKEY RRset in response to DNSKEY query.
Feb 19 15:38:35 unbound[550:2] info: resolving akam.cdc.gov. DNSKEY IN
Feb 19 15:38:36 unbound[550:2] info: response for akam.cdc.gov. DNSKEY IN
Feb 19 15:38:36 unbound[550:2] info: reply from <.> <IP_ADDRESS>#853
Feb 19 15:38:36 unbound[550:2] info: query response was nodata ANSWER
Feb 19 15:38:36 unbound[550:2] info: Missing DNSKEY RRset in response to DNSKEY query.
Feb 19 15:38:36 unbound[550:2] info: resolving akam.cdc.gov. DNSKEY IN
Feb 19 15:38:37 unbound[550:2] info: response for akam.cdc.gov. DNSKEY IN
Feb 19 15:38:37 unbound[550:2] info: reply from <.> <IP_ADDRESS>#853
Feb 19 15:38:37 unbound[550:2] info: query response was nodata ANSWER
Feb 19 15:38:37 unbound[550:2] info: Missing DNSKEY RRset in response to DNSKEY query.
Feb 19 15:38:37 unbound[550:2] info: resolving akam.cdc.gov. DNSKEY IN
Feb 19 15:38:37 unbound[550:2] info: response for akam.cdc.gov. DNSKEY IN
Feb 19 15:38:37 unbound[550:2] info: reply from <.> 2606:4700:4700::1111#853
Feb 19 15:38:37 unbound[550:2] info: query response was nodata ANSWER
Feb 19 15:38:37 unbound[550:2] info: Missing DNSKEY RRset in response to DNSKEY query.
  But I have not changed my settings and they were working properly.  What am I missing?
Here's my config:
`#
server:
###########################################################################
# BASIC SETTINGS
###########################################################################
# Time to live maximum for RRsets and messages in the cache. If the maximum
# kicks in, responses to clients still get decrementing TTLs based on the
# original (larger) values. When the internal TTL expires, the cache item
# has expired. Can be set lower to force the resolver to query for data
# often, and not trust (very large) TTL values.
cache-max-ttl: 86400
# Time to live minimum for RRsets and messages in the cache. If the minimum
# kicks in, the data is cached for longer than the domain owner intended,
# and thus less queries are made to look up the data. Zero makes sure the
# data in the cache is as the domain owner intended, higher values,
# especially more than an hour or so, can lead to trouble as the data in
# the cache does not match up with the actual data any more.
cache-min-ttl: 300

# Set the working directory for the program.
#directory: "/etc/unbound/"

# RFC 6891. Number  of bytes size to advertise as the EDNS reassembly buffer
# size. This is the value put into  datagrams over UDP towards peers.
# The actual buffer size is determined by msg-buffer-size (both for TCP and
# UDP). Do not set higher than that value.
# Default  is  1232 which is the DNS Flag Day 2020 recommendation.
# Setting to 512 bypasses even the most stringent path MTU problems, but
# is seen as extreme, since the amount of TCP fallback generated is
# excessive (probably also for this resolver, consider tuning the outgoing
# tcp number).
edns-buffer-size: 1232

# Listen to for queries from clients and answer from this network interface
# and port.
interface: <IP_ADDRESS>@52
interface: ::@52

# Rotates RRSet order in response (the pseudo-random number is taken from
# the query ID, for speed and thread safety).
rrset-roundrobin: yes

# Drop user  privileges after  binding the port.
username: "unbound"

# May be set to yes if you have IPv6 connectivity
do-ip6: yes

###########################################################################
# LOGGING
###########################################################################

# Do not print log lines to inform about local zone actions
log-local-actions: no

# Do not print one line per query to the log
log-queries: no

# Do not print one line per reply to the log
log-replies: no

# Do not print log lines that say why queries return SERVFAIL to clients
log-servfail: no

# Further limit logging
#logfile: /dev/null
logfile: "/unbound.log"

# Only log errors
#verbosity: 0
verbosity: 1

# Log time in UTC
log-time-ascii: yes

###########################################################################
# PRIVACY SETTINGS
###########################################################################

# RFC 8198. Use the DNSSEC NSEC chain to synthesize NXDO-MAIN and other
# denials, using information from previous NXDO-MAINs answers. In other
# words, use cached NSEC records to generate negative answers within a
# range and positive answers from wildcards. This increases performance,
# decreases latency and resource utilization on both authoritative and
# recursive servers, and increases privacy. Also, it may help increase
# resilience to certain DoS attacks in some circumstances.
aggressive-nsec: yes

# Extra delay for timeouted UDP ports before they are closed, in msec.
# This prevents very delayed answer packets from the upstream (recursive)
# servers from bouncing against closed ports and setting off all sort of
# close-port counters, with eg. 1500 msec. When timeouts happen you need
# extra sockets, it checks the ID and remote IP of packets, and unwanted
# packets are added to the unwanted packet counter.
delay-close: 10000

# Prevent the unbound server from forking into the background as a daemon
do-daemonize: no

# Add localhost to the do-not-query-address list.
do-not-query-localhost: no

# Number  of  bytes size of the aggressive negative cache.
neg-cache-size: 4M

# Send minimum amount of information to upstream servers to enhance
# privacy (best privacy).
qname-minimisation: yes

###########################################################################
# SECURITY SETTINGS
###########################################################################
# Only give access to recursion clients from LAN IPs
access-control: <IP_ADDRESS>/32 allow
access-control: <IP_ADDRESS>/16 allow
access-control: <IP_ADDRESS>/12 allow
access-control: <IP_ADDRESS>/8 allow
access-control: ::1/128 allow
access-control: x:x:x:b141::0/64 allow
access-control: x:x:x:b142::0/64 allow
access-control: x:x:x:b143::0/64 allow
access-control: x:x:x:b144::0/64 allow
access-control: x:x:x:b145::0/64 allow
access-control: x:x:x:b146::0/64 allow
access-control: x:x:x:b147::0/64 allow
access-control: x:x:x:b148::0/64 allow
access-control: x:x:x:b149::0/64 allow
# access-control: fc00::/7 allow
# access-control: ::1/128 allow

# File with trust anchor for  one  zone, which is tracked with RFC5011
# probes.
auto-trust-anchor-file: "var/root.key"

# Enable chroot (i.e, change apparent root directory for the current
# running process and its children)
#chroot: "/etc/unbound/"

# Deny queries of type ANY with an empty response.
deny-any: yes

# Harden against algorithm downgrade when multiple algorithms are
# advertised in the DS record.
harden-algo-downgrade: yes

# RFC 8020. returns nxdomain to queries for a name below another name that
# is already known to be nxdomain.
harden-below-nxdomain: yes

# Require DNSSEC data for trust-anchored zones, if such data is absent, the
# zone becomes bogus. If turned off you run the risk of a downgrade attack
# that disables security for a zone.
harden-dnssec-stripped: yes

# Only trust glue if it is within the servers authority.
harden-glue: yes

# Ignore very large queries.
harden-large-queries: yes

# Perform additional queries for infrastructure data to harden the referral
# path. Validates the replies if trust anchors are configured and the zones
# are signed. This enforces DNSSEC validation on nameserver NS sets and the
# nameserver addresses that are encountered on the referral path to the
# answer. Experimental option.
harden-referral-path: no

# Ignore very small EDNS buffer sizes from queries.
harden-short-bufsize: yes

# Refuse id.server and hostname.bind queries
hide-identity: yes

# Refuse version.server and version.bind queries
hide-version: yes

# Report this identity rather than the hostname of the server.
identity: "DNS"

# These private network addresses are not allowed to be returned for public
# internet names. Any  occurrence of such addresses are removed from DNS
# answers. Additionally, the DNSSEC validator may mark the  answers  bogus.
# This  protects  against DNS  Rebinding
private-address: <IP_ADDRESS>/8
private-address: <IP_ADDRESS>/12
private-address: <IP_ADDRESS>/16
private-address: <IP_ADDRESS>/16
private-address: fd00::/8
private-address: fe80::/10
private-address: ::ffff:0:0/96

# Enable ratelimiting of queries (per second) sent to nameserver for
# performing recursion. More queries are turned away with an error
# (servfail). This stops recursive floods (e.g., random query names), but
# not spoofed reflection floods. Cached responses are not rate limited by
# this setting. Experimental option.
ratelimit: 1000

# Use this certificate bundle for authenticating connections made to
# outside peers (e.g., auth-zone urls, DNS over TLS connections).
tls-cert-bundle: /etc/ssl/certs/ca-certificates.crt

# Set the total number of unwanted replies to eep track of in every thread.
# When it reaches the threshold, a defensive action of clearing the rrset
# and message caches is taken, hopefully flushing away any poison.
# Unbound suggests a value of 10 million.
unwanted-reply-threshold: 10000

# Use 0x20-encoded random bits in the query to foil spoof attempts. This
# perturbs the lowercase and uppercase of query names sent to authority
# servers and checks if the reply still has the correct casing.
# This feature is an experimental implementation of draft dns-0x20.
# Experimental option.
use-caps-for-id: yes

# Help protect users that rely on this validator for authentication from
# potentially bad data in the additional section. Instruct the validator to
# remove data from the additional section of secure messages that are not
# signed properly. Messages that are insecure, bogus, indeterminate or
# unchecked are not affected.
val-clean-additional: yes

###########################################################################
# PERFORMANCE SETTINGS
###########################################################################
# https://nlnetlabs.nl/documentation/unbound/howto-optimise/

# Number of slabs in the infrastructure cache. Slabs reduce lock contention
# by threads. Must be set to a power of 2.
infra-cache-slabs: 4

# Number of incoming TCP buffers to allocate per thread. Default
# is 10. If set to 0, or if do-tcp is "no", no  TCP  queries  from
# clients  are  accepted. For larger installations increasing this
# value is a good idea.
incoming-num-tcp: 10

# Number of slabs in the key cache. Slabs reduce lock contention by
# threads. Must be set to a power of 2. Setting (close) to the number
# of cpus is a reasonable guess.
key-cache-slabs: 4

# Number  of  bytes  size  of  the  message  cache.
# Unbound recommendation is to Use roughly twice as much rrset cache memory
# as you use msg cache memory.
msg-cache-size:<PHONE_NUMBER>

# Number of slabs in the message cache. Slabs reduce lock contention by
# threads. Must be set to a power of 2. Setting (close) to the number of
# cpus is a reasonable guess.
msg-cache-slabs: 4

# The number of queries that every thread will service simultaneously. If
# more queries arrive that need servicing, and no queries can be jostled
# out (see jostle-timeout), then the queries are dropped.
# This is best set at half the number of the outgoing-range.
# This Unbound instance was compiled with libevent so it can efficiently
# use more than 1024 file descriptors.
num-queries-per-thread: 4096

# The number of threads to create to serve clients.
# This is set dynamically at run time to effectively use available CPUs
# resources
num-threads: 4

# Number of ports to open. This number of file descriptors can be opened
# per thread.
# This Unbound instance was compiled with libevent so it can efficiently
# use more than 1024 file descriptors.
outgoing-range: 8192

# Number of bytes size of the RRset cache.
# Use roughly twice as much rrset cache memory as msg cache memory
rrset-cache-size:<PHONE_NUMBER>

# Number of slabs in the RRset cache. Slabs reduce lock contention by
# threads. Must be set to a power of 2.
rrset-cache-slabs: 4

# Do no insert authority/additional sections into response messages when
# those sections are not required. This reduces response size
# significantly, and may avoid TCP fallback for some responses. This may
# cause a slight speedup.
minimal-responses: yes

# # Fetch the DNSKEYs earlier in the validation process, when a DS record
# is encountered. This lowers the latency of requests at the expense of
# little more CPU usage.
prefetch: yes

# Fetch the DNSKEYs earlier in the validation process, when a DS record is
# encountered. This lowers the latency of requests at the expense of little
# more CPU usage.
prefetch-key: yes

# Have unbound attempt to serve old responses from cache with a TTL of 0 in
# the response without waiting for the actual resolution to finish. The
# actual resolution answer ends up in the cache later on.
serve-expired: yes

# Open dedicated listening sockets for incoming queries for each thread and
# try to set the SO_REUSEPORT socket option on each socket. May distribute
# incoming queries to threads more evenly.
so-reuseport: yes

###########################################################################
# LOCAL ZONE
###########################################################################

# Include file for local-data and local-data-ptr

include: /etc/unbound/srv-records.conf
include: /etc/unbound/a-records.conf

###########################################################################
# FORWARD ZONE
###########################################################################
include: /etc/unbound/forward-records.conf

remote-control:
control-enable: no
`

Yes, there is misconfiguration, but it is in the cdc.gov zone.
https://dnsviz.net/d/www.cdc.gov/YC7hBQ/dnssec/
https://lists.dns-oarc.net/pipermail/dns-operations/2020-December/020779.html
https://gitlab.nic.cz/knot/knot-resolver/-/issues/662#note_188577
I have found that setting qname-minimisation: no in the Unbound configuration makes it more likely to resolve www.cdc.gov successfully but the problem lies in the cdc.gov nameservers.

The cdc.gov nameservers publish:

A signed alias of www.cdc.gov to www.akam.cdc.gov.

; <<>> DiG 9.16.12-Debian <<>> +dnssec +norec @ns1.cdc.gov www.cdc.gov
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39120
;; flags: qr aa; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;www.cdc.gov.			IN	A

;; ANSWER SECTION:
www.cdc.gov.		300	IN	CNAME	www.akam.cdc.gov.
www.cdc.gov.		300	IN	RRSIG	CNAME 7 3<PHONE_NUMBER>6133457<PHONE_NUMBER>2957 42473 cdc.gov. n1VRoI84Zp5l+GHq7t24N7Pau6fBPG6YZ71WixTpeCSsp0EM28t8EiL4 2UQJPutO2k7WkZV8etYsEp5p6FzQ0RUOPQfljzaLYv9e4AyLWfEfHiof ZVTy6aT4gJLxiYG8W9JuhCE8JX4ldeqM7FKkvxZL2yt7rVKzJ3GK+CG2 c64=
www.akam.cdc.gov.	3600	IN	CNAME	www.cdc.gov.edgekey.net.

  It is inconsistent for the cdc.gov nameservers to publish answer section records for www.akam.cdc.gov given that akam.cdc.gov is a delegation point and akam.cdc.gov is served by a completely different set of nameservers.

; <<>> DiG 9.16.12-Debian <<>> +dnssec +norec @ns1.cdc.gov www.akam.cdc.gov
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2625
;; flags: qr aa; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;www.akam.cdc.gov.		IN	A

;; ANSWER SECTION:
www.akam.cdc.gov.	3600	IN	CNAME	www.cdc.gov.edgekey.net.

;; Query time: 4 msec
;; SERVER: <IP_ADDRESS>#53(<IP_ADDRESS>)
;; WHEN: Sat Feb 20 13:44:44 EST 2021
;; MSG SIZE  rcvd: 82

Some resolvers are able to find a working resolution path due to the inclusion of auth00.ns.uu.net and auth100.ns.uu.net in the NS RRset for cdc.gov published by the gov nameservers, but these servers are not included in the NS RRset at the apex of the cdc.gov zone.   Thank you for looking into the issue and explaining it.  I will try to bring it to the attention of those in charge of the cdc.gov domain.
I'll go ahead and close this issue.


BUG: The output code is mixed with "In[ ]:" when more than one function is returned as code.

The output code is mixed with "In[ ]:" when more than one function is returned as code.
Here is an example output:
import requests
import json
import os
import re
from tqdm import tqdm
from collections import Counter
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split

 In[ ]:

from tqdm import tqdm
from collections import Counter
from sklearn.model_selection import train_test_split

 In[ ]:

def load_data(path):
    with open(path, 'r', encoding='utf-8') as f:
        data = f.read()
    return data

 In[ ]:

def tokenize(text):
    return re.findall(r'\w+', text)

 In[ ]:

def translate(text, model):
    url = 'https://api.davinci.ai/v1/translate'

    headers = {'Authorization': 'Bearer {}'.format(model)}

    params = {'src': 'en', 'tgt': 'zh', 'text': text}

    response = requests.get(url, headers=headers, params=params)

    return response.json()['translation']

 In[ ]:

def translate_chunks(chunks, model):
    translated_chunks = []

    for chunk in tqdm(chunks):
        translated_chunks.append(translate(chunk, model))

    return translated_chunks

 In[ ]:

def save_data(path, data):
    with open(path, 'w', encoding='utf-8') as f:
        f.write('\n'.join(data))

 In[ ]:

def main():
    path = os.path.join('..', 'data', 'shakespeare.txt')

    data = load_data(path)

    chunks = tokenize(data)

    translated_chunks = translate_chunks(chunks, model)

    save_data('shakespeare_translated.txt', translated_chunks)

This has been resolved. @Emasoft


Update phonecalloriginmanager_setcallorigin_742485117.md

Updated the documentation since some of the info is stale based on the new feature/design change.

@wilsonNg007 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Thanks for your contribution! I've merged your changes and they should be live soon.


Design System: Slightly hide anti aliasing on Pill

Context
It seems like the browser is trying to smooth out the curved border (anti-aliasing).

The pixels on the edges of the white box shadow are blurred slightly to make the white box shadow seem more curved

From the blurring of the white box shadow we see a little bit of blue come up from underneath it. This disappears when we remove the border radius because there is no curving that needs to be 'anti aliased':

Summary
It seems like the browser is trying to smooth out the curved border (anti-aliasing). Added a border to slightly cover the blurred pixels. It doesn't solve the problem all the way, but I think it's a good try.
If there's a way to disable anti aliasing for a box shadow - let me know!
Relevant Technical Choices
Trying to hide the anti aliasing using a border to cover the pixels that are slightly transparent.
To-do
n/a
User-facing changes

Before
After

Testing Instructions

Go to dashboard/storybook
Tab with keyboard until you find a pill
See that there is less of those lines on the border of the pill.

QA

[ ] This is a non-user-facing change and requires no QA

UAT

[x] UAT should use the same steps as above.

Reviews
Does this PR have a security-related impact?
no
Does this PR change what data or activity we track or use?
no
Does this PR have a legal-related impact?
no
Checklist

[x] This PR addresses an existing issue and I have linked this PR to it in ZenHub
[x] I have tested this code to the best of my abilities
[x] I have verified accessibility to the best of my abilities (docs)
[x] I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
[ ] This PR contains automated tests (unit, integration, and/or e2e) to verify the code works as intended (docs)
[x] I have added documentation where necessary
[x] I have added a matching Type: XYZ label to the PR

Fixes #6537

What an interesting problem! I played with a few other potential solutions but they were more intense or didn't really do the job.
One i thought was super interesting was using :before, :after w/ a box-shadow (https://dev.to/shadowscientist/comment/133f4) but that only works if it's a tight space.
Otherwise, yah seems like it's just a known issue that is super nuanced:

https://stackoverflow.com/questions/58282479/anti-aliasing-border-radius-issue-when-using-rgba-values
https://bugzilla.mozilla.org/show_bug.cgi?id=1400880

That said, I like your implementation.

Thanks for the extra investigation @BrittanyIRL. I guess for now we can move this to QA but if we want a more intense solution I'm happy to try to do something different!


Security alert on install from ethers, ws

Describe the bug
When installing the latest @semaphore-protocol/core (v4.7.2) & running npm audit, the following alert appears:
ws  8.0.0 - 8.17.0
Severity: high
ws affected by a DoS when handling a request with many HTTP headers - https://github.com/advisories/GHSA-3h5v-q93c-6h6q
No fix available
node_modules/ws
  ethers  6.0.0-beta.1 - 6.13.0
  Depends on vulnerable versions of ws
  node_modules/ethers
    @semaphore-protocol/proof  >=4.0.0-alpha
    Depends on vulnerable versions of ethers
    node_modules/@semaphore-protocol/proof
      @semaphore-protocol/core  *
      Depends on vulnerable versions of @semaphore-protocol/proof
      node_modules/@semaphore-protocol/core

To Reproduce

npm install @semaphore-protocol/core (Note some older versions may have this alert as well)
npm audit

Expected behavior
No security alert is present. This has been fixed in ethers >= v6.13.1 https://github.com/ethers-io/ethers.js/releases/tag/v6.13.1.
Note that since this is a server-side ws issue, it is unlikely have an impact barring some exceptional use cases (Using Semaphore's ether's ws version on  server)
Technologies (please complete the following information):

Node.js v20.18.1
NPM 10.8.2

Additional context
I will open PR to address by updating ethers version. Feel free to assign this issue to me.

Would it also be valuable to add checks either in CI or a yarn command/script that fails if it detects a Severity: high in a dep?

Would it also be valuable to add checks either in CI or a yarn command/script that fails if it detects a Severity: high in a dep?

Yes, I think it's definitely valuable!

@cedoor Done https://github.com/semaphore-protocol/semaphore/issues/922


"Mârțești", "Bănărești" are listed in the drop-down menu

Description:
When user writes "Mârțești", "Bănărești" the locations appear in the drop-down menu next to the comune and the county witch is a part according to the law
Precondition:
The website is up an running.

Step 1
Write in the search bar "Mârțești"

Expected results
The location appears in the drop-down menu as "Mârțești, Argeș (Săpata)"

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Bănărești"

Expected results
The location appears in the drop-down menu as "Bănărești, Argeș (Săpata)"


stopComplying and stopReplying are possibly not as selective as they need to be

The possible culprit
Q: do stopComplying and stopReplying work as intended right now? Or do they remove too much?

Todo:
Add tests for each combination of possible arguments. By my count, there are 8 combinations
a, b, c
undefined, undefined, undefined
a, b, undefined
a, undefined, c
undefined, b, c
a, undefined, undefined
undefined, b, undefined
undefined, undefined, c

You should write a single test in a loop that goes through each of those since it's difficult to track all of the variations.

So one related issue:
callbacks may be wrapped via the makeCallback function, so if your callback needs to be wrapped, you will not be able to call stopReplying as it won't have the wrapped callback for comparison.
https://github.com/marionettejs/backbone.radio/blob/master/src/backbone.radio.js#L331


Error when installing (PHP 5.3)

When installing with PHP 5.3 I get the error:
Fatal error: Can't use method return value in write context in /var/www/web597/html/bludit/bl-plugins/disqus/plugin.php on line 36
The error is caused by:
if( !empty($Site->homePage()) ) {
    $this->disable = true;
}

The following works with PHP 5.3:
$hp = $Site->homePage();
if (!empty($hp)) {
    $this->disable = true;
}

http://stackoverflow.com/questions/1075534/cant-use-method-return-value-in-write-context

It worked. thank you :+1:

Fixed, I used the Helper from Bludit.
if( Text::notEmpty($Site->homePage()) ) {
    $this->disable = true;
}


fix: add support for placing attributes on server functions

Adding instrumentation to server functions is not straightforward (currently requires calling a different instrumented function). This commit takes all attributes placed on the server function (any attributes between #[server] and the source function body) and places them on both the generated front end and back end functions. If those attributes are only desirable on one of the targets (wasm or server side), they can be conditionally included using cfg_attr.

Thanks! This looks great, and much appreciated.


Add ability to allow privileged containers via a manifest flag.

This PR adds an additional boolean manifest field systemOptions.allowPrivileged that enables privileged containers in the cluster. Requires https://github.com/gravitational/planet/pull/476. Closes https://github.com/gravitational/gravity/issues/619.

@knisbet Yeah, I was gonna do them in a separate PR but I guess here would be fine too.

@knisbet I've added the docs too.


Alert-naming-changes

We need to handle update scenario. Otherwise all the configured alert willl be stopped due to rename

We need to handle update scenario. Otherwise all the configured alert willl be stopped due to rename

Actually it will be very complicated, so we'll ask user to go through App setup page instead, and disable and reenable product. As this is 5.0.0 so I think it should be okay to suggest user on doing this instead. Unless @hardikhdholariya if you have any better suggestion to handle this.
@hardikhdholariya  - We need to discuss this next week so we can discuss all points regarding 5.0.0


Add missing functions to the gradient object.

Needed to fully access the values of radial and conic gradients.
All of the functions needed to read gradient.linear existed, but not radial or conic gradients.  This PR adds them.
The use would be a function to modify a gradient, such as lightening the colors.
#let lighten-grad(clr) = {
  let new-stops = clr.stops().map(it => (it.at(0).lighten(50%), it.at(1)))
  gradient.radial(
    ..new-stops,
    space: clr.space(),
    relative: clr.relative(),
    focal-center:clr.focal-center(),
    focal-radius:clr.focal-radius()
  )
}
#let g = gradient.radial(..color.map.rainbow)
#box(width:5in, height:3in, fill:g)
#box(width:5in, height:3in, fill:lighten-grad(g))

Currently fails with
Error: type gradient has no method `focal-center`
    ╭─[/main.typ:14:22]
    │
 14 │     focal-center:clr.focal-center(),
────╯

And this is obviously undesirable.

I agree it's non-ideal from my Rustacean PoV, but I think the ship has already kind of sailed for gradients. Not really because we can always change it more fundamentally, but I don't see how else we could implement this at the moment. We could perhaps make it an error instead of returning none, since the kind can be checked, and we already have something similar for length conversion methods IIRC.


Implemented find_outliers.py

Description

This pull request proposes to add a new file: find_outliers.py. From the terminal the user can find all of the "cells" in a JSON or CSV that are outliers in their category (column). That is to say: Z-score > 3. The information is then printed to the terminal.
Related Issues

This script addresses the first feature mentioned in the issue "Create a python script to determine the performance and accuracy of the datasets" #31 No other feature is implemented.
Changes Made

-Create a Python script in the Validation folder
Screenshots (if applicable)

Checklist
Please review and check the following before submitting your pull request:

[ X] I have followed the project's coding conventions and guidelines
[ X] I have tested my changes thoroughly
[ X] I have added/updated relevant documentation
[X ] My code follows best practices and is easy to understand
[ ] I have added necessary test cases (if applicable)
[ ] All existing tests are passing
[ ] My changes do not introduce any new warnings or errors

Additional Notes

Support for other file types can later be added without too many difficulties. Currently only CSV and JSON are supported.
The script could easily be modified to be used as a function in another script instead of printing the output to the terminal.

@rtiop
use
python```
if name == 'main':
main()

Done. I've seen it in many scripts, but I never understood what it was for. What's its utility?

Done. I've seen it in many scripts, but I never understood what it was for. What's its utility?

It is mainly used so we can make a module, You can refer here to get clarified.


Move MapboxGeocoder.swift to master

Moved the MapboxGeocoder.swift pod off the swift3 branch, which is going away, per https://github.com/mapbox/MapboxGeocoder.swift/pull/57#issuecomment-280905054. Feel free to replace the commit with a newer one or the next release.

Thanks for the PR @1ec5, I'm going to update the whole repo in the next 2 weeks with creating even better examples after using Mapbox iOS SDK for more than 5 months now. Hope I can also contribute to mapbox-gl-native in the next 15 weeks, picking a few issues and work on the while working on my Bachelor thesis which also uses Mapbox. 👍


Roll Engine from 1ef10b240e28 to 4908c4d3d0e1 (6 revisions)

https://github.com/flutter/engine/compare/1ef10b240e28...4908c4d3d0e1
2020-09-16<EMAIL_ADDRESS>Verify Flutter clang module, add hook for verifying consumer warnings (flutter/engine#21203)
2020-09-16<EMAIL_ADDRESS>Roll Skia from a195d101f96c to 3e72b3ff8ca7 (7 revisions) (flutter/engine#21205)
2020-09-16<EMAIL_ADDRESS>Add Linux Wayland support (flutter/engine#20629)
2020-09-16<EMAIL_ADDRESS>[darwin] Header #import hygiene (flutter/engine#21193)
2020-09-16<EMAIL_ADDRESS>Fix NPE in PlatformPlugin.getClipboardData() (flutter/engine#21189)
2020-09-15<EMAIL_ADDRESS>Roll Fuchsia Linux SDK from -iBBiPj1C... to sXR21ye3r... (flutter/engine#21197)
Also rolling transitive DEPS:
fuchsia/sdk/core/linux-amd64 from -iBBiPj1CeUA to sXR21ye3rKeL
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/master/autoroll/README.md

Commit queue failed; closing this roll.


MySQL providers for Nether.Web

Create MySQL providers for

Identity
Player Management
Leaderboard

Update deployment to WebApps to use the In-App MySQL feature in App Service. This would allow a fully free environment for Nether.Web APIs (excluding analytics) which could be useful for dev/test, demos, etc

Since Web Apps have a Web App + SQL offering with free SQLDB, we will use that instead. See #417


Do not mutate output of flattenStyle

Summary:
flattenStyle may return an object which is already frozen (in development), so it is incorrect to further mutate this.
Changelog: [Internal]
Differential Revision: D59515063

Also Fix for TextInput.js and TextOptimized.js

Also Fix for TextInput.js and TextOptimized.js

@meetdhanani17 feel free to send out a follow-up PR for that

Follow-up for this in https://github.com/facebook/react-native/pull/45348

Also Fix for TextInput.js and TextOptimized.js

@meetdhanani17 feel free to send out a follow-up PR for that

Follow-up for one remaining in https://github.com/facebook/react-native/pull/45932


Improve ERA tests using fundamental properties and error bounds

This turned out to be highly nontrivial.  We thought about using some transfer function math to do this, but there wasn't a good, standard transfer function module with all the necessary methods.  In the end we decided to go with simply matching the modred computation to a known (pre-computed) solution.


Support flash encryption

Flash encryption is supported by all chips from the ESP32 and newer. We can encrypt data on the host and flash it to the target, assuming we first burn the encryption key into the target's eFuse.
For more details:
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/security/flash-encryption.html

After some further discussion, it seems there's not really anything actionable here. I have written a short guide for using flash encryption in this gist:
https://gist.github.com/jessebraham/0c7d86f9866614dc602e5d8e0e97c37a
If there is any specific functionality we need in the future we can open issues for that specifically.

@jessebraham in your gist, you need to manually build a secure flash bootloader with esp-idf idf.py, and then manually pass that bootloader binary to espflash.
That seems fairly incomplete to me. Shouldn't espflash be able to prepare the bootloader for encryption?


Add an extra label to the build job to label the stage of the build

This PR adds a third label to kubernetes job objects created by kmmo for future proofing.
Currently jobs are labelled the module.name and target-kernel, and these are used to search for the job to ensure its creation, completion etc.
The manger searching for these jobs has no other way of determining which job it should be managing.
If we add multiple jobs into the build pipeline (e.g. a signing job) the build manager has no way of distinguishing between the different jobs, so when it searches for completed jobs it will detect finished signing jobs as if they were finished build (kaniko) jobs, and vice versa. This would lead to much confusion and the pipeline not running through its stages correctly.
This commit adds a third "kmm.node.kubernetes.io/build-stage" label that is set to "build" to make it explicit that this is a build job. For signing jobs it should be set to "sign" and for any other future jobs it would be set to a descriptive name. The build job manager then searches for all 3 labels (and future job managers should also search for their own build-stage label) and ignores any jobs with a different build-stage set.

Codecov Report
Base: 73.46% // Head: 73.47% // Increases project coverage by +0.01% :tada:

Coverage data is based on head (98e03df) compared to base (4c6a77d).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #97      +/-   ##
==========================================
+ Coverage   73.46%   73.47%   +0.01%
==========================================
  Files          17       17
  Lines        1771     1772       +1
==========================================
+ Hits         1301     1302       +1
  Misses        404      404
  Partials       66       66

Impacted Files
Coverage Δ

internal/build/job/manager.go
76.74% <100.00%> (+0.27%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

/ok-to-test
/lgtm

/retest

@chr15p Can we please update the commit message: build-stage --> build-type?

/lgtm

/approve


avoid expensive queries that do not change certain results

The mayIndex function, which checks whether a page can be indexed or not, is downloading the website content even if the robots.txt was already cached and disallowing that page.
With this PR no result is changed but in case the robots.txt denies a page, it won't even be fetched (as reading the page meta wouldn't change that result).
The results of the tests is exactly the same after this change, only that some unnecessary calls to (the expensive) file_get_contents are avoided.

Thanks!


Added the skeleton for the Bitcoin wallet sample dapp

This MR just creates the skeleton for the sample dapp showcasing the Bitcoin integration.
The code will follow in subsequent MRs. If the idea is to merge in only fully functional sample dapps, the MR can remain in draft status.

@THLO As discussed offline, should we perhaps close this PR or mark it as a draft until the example is fully functional?

@THLO As discussed offline, should we perhaps close this PR or mark it as a draft until the example is fully functional?


Feature/init array default values

allows initializing a mutable array of size n with default values
skips having to init an array of size n with unknown type values and then having to fill the array with values.
added some QOL features such as init array from a function by index and init array from hardcoded list
fixes #291

@phischu

@phischu done


Render react-bootstrap instead of Bootstrap markup

Prerequisites

[x] I have read the documentation

Description
I would like to render all fields with react-bootstrap instead of Bootstrap markup since I don't want to let jQuery and Bootstrap.js deal with DOM changes next to React. So, I plan to rework all pure components in order to support react-bootstrap.
I'm aksing myself if you would like this kind of changes too or if I fork this project?
Version
0.40.0

This library doesn't load bootstrap's javascript code at all. Also, we think react-bootstrap would introduce a lot of overhead and wouldn't bring enough benefit to the project. I'm closing this.


Schedule Generator Implementation

Could you explain in short how did you implement the schedule maker? What libraries did you used to draw the schedule etc? I'm trying to do my own implementation in Java/Kotlin

Hi, I straightaway use this package to make the timetable, https://pub.dev/packages/flutter_timetable_view. Digging the code, found that it uses CustomPainter to draw a custom layout, but not sure exactly how they calculate the size and position.
I've tried to do it before on my own, using Table, Column and Row, and others, but none of them seems to be working.
Anyway just found this Android library, not sure how you want to lay out the schedule but this one is pretty nice. https://github.com/tlaabs/TimetableView

Thanks for the tip. I'm currently in the process of porting the app to C#/.NET and Kotlin.

Great!

There's a couple of problems in the schedule maker.

The subject exists, but the error still shows

Thanks!
I can repro this on my side. I think it is caused by hardcoded data in request params. I'll look into it

Hi. Can you test with the latest release?
If you're using the web app, be sure to force reload to clear the pwa cache.

Arrangement of Days on timetable is not aligned

Thank you and noted for the bugs report. I'll make a new issue for it.


Connection pooling issue Unix

Environment

Python: 3.6.8
pyodbc: 4.0.30
unixodbc: 2.3.7
OS: Ubuntu 18.04
DB: SQL Server Azure
driver: ODBC Driver 17 for SQL Server

Issue
I am refactoring some old code to handle SQL connection in a more consistent manner. Now, I have read about connection pooling and I wanted to check whether this was supported. In order to check this, I have run the following code:
import pyodbc

def runQuery(con, query, parameters=None):
    result = list()

    if parameters is None:
        parameters = ()

    with con.cursor() as cur:
        cur.execute(query, parameters)

        column_names = [col[0] for col in cur.description]
        result = [
            dict(zip(column_names, row))
            for row in cur.fetchall()
        ]

    return result

conn1 = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};'
                            f'SERVER={server};'
                            f'DATABASE={database};'
                            f'UID={username};'
                            f'PWD={password}')

with conn1:
    result1 = runQuery(conn1, 'SELECT @@SPID')
    print(result1)

conn2 = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};'
                            f'SERVER={server};'
                            f'DATABASE={database};'
                            f'UID={username};'
                            f'PWD={password}')

with conn2:
    result2 = runQuery(conn2, 'SELECT @@SPID')
    print(result2)

which results in
[{'': 76}]
[{'': 79}]

Is this the correct way of testing whether connection pooling is being used? If so, does this in fact indicate that this is not the case here?
My odbcinst.ini file contains the following:
[ODBC]
Pooling=Yes

[ODBC Driver 17 for SQL Server]
Description=Microsoft ODBC Driver 17 for SQL Server
Driver=/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.4.so.1.1
UsageCount=1

Any help would be greatly appreciated. Thanks!

Related: #48

Checking the SPIDs is as good a way as any.  As @gordthompson hints at though, using a context manager does not close the connection when it exits (contrary to popular belief).  Try positively closing the connection after the context manager, e.g.:
with conn1:
    result1 = runQuery(conn1, 'SELECT @@SPID')
    print(result1)
conn1.close()

and then see if the SPIDs are the same.  It's possible they might still be different of course, pooling is after all more of a suggestion than a mandate.
FYI, just in case you hadn't seen it, pooling can also be controlled using the pooling pyodbc attribute, but only before any connections have been made.

When I explicitly close the first connection, the SPIDs are indeed identical. However, they seem to be identical regardless of whether pooling is enabled or not (through the pooling pyodbc attribute and/or the odbcinst.ini config file).
I realize that this might not be strictly within the scope of pyodbc issues, but is there a better way of testing whether this is working or not?

You can open another connection (or several) to the server from a different machine, between the close and open from Python, which will force a non-pooled connection to not reuse the same SPID (because one of those other connections will have taken it.)

When running the above example in a loop, and querying the DB for the active connections using this query:
SELECT
      s.login_name
    , s.host_name
    , s.program_name
    , s.host_process_id
    , COUNT(*) AS connection_count
FROM sys.dm_exec_sessions AS s
GROUP BY
      s.login_name
    , s.host_name
    , s.program_name
    , s.host_process_id;

I get this result:

Everything remains fairly constant, except for the single connection on row 6. This line is constantly being removed and being added again over consecutive executions of the query. Maybe my understanding of the query and/or the concept of connection pooling is completely wrong, but this doesn't seem like connection pooling to me?

I found out that the CPTimeout parameter was missing from my config. This changes the behavior: now the number of connections keeps growing (1 per iteration), presumably until the timeouts start to occur. Connections are seemingly not being reused.
Testing code:
def runQuery(conn, query, parameters=None):
    result = list()

    if parameters is None:
        parameters = ()

    with conn.cursor() as cur:
        cur.execute(query, parameters)

        column_names = [col[0] for col in cur.description]
        result = [
            dict(zip(column_names, row))
            for row in cur.fetchall()
        ]

    return result

for i in range(5000):
    print(i)
    conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};'
                                    f'SERVER={server};'
                                    f'DATABASE={database};'
                                    f'UID={username};'
                                    f'PWD={password}')

    result = runQuery(conn, 'SELECT @@SPID')
    conn.close()

The reason that I am asking is the following. I have run the above code in a loop and the timings for both cases (pooling enabled/disabled) seem very similar, whereas I would expect the pooling variant to be noticeably faster.

@JelleVE I tested the "connection speed" with the following code and the time difference between pyodbc.pooling = True and pyodbc.pooling = False is significant here on my Windows 11 machine. The former finishes in 3.67 seconds while without pooling it takes 20.95 seconds:
import pyodbc
import time

sql_server_url = "[...]"
admin_username = "[...]"
admin_password = "[...]"
database_name = "[...]"
enable_pooling = False

start = time.time()
pyodbc.pooling = enable_pooling
for i in range(0, 50):
    connection = pyodbc.connect(
        "DRIVER={ODBC Driver 17 for SQL Server};SERVER="
        + sql_server_url
        + ";PORT=1433;DATABASE="
        + database_name
        + ";UID="
        + admin_username
        + ";PWD="
        + admin_password,
        autocommit=True,
    )
    with connection.cursor() as cursor:
        cursor.execute(f"SELECT @@SPID")
        columns = [column[0] for column in cursor.description]
        result = [dict(zip(columns, row)) for row in cursor.fetchall()]
        print(result)
    connection.close()
end = time.time()
print(end - start)

Unixodbc ignores pyodbc's pooling logic:

https://github.com/mkleehammer/pyodbc/blob/master/src/pyodbcmodule.cpp#L316

    if (bPooling)
    {
        if (!SQL_SUCCEEDED(SQLSetEnvAttr(SQL_NULL_HANDLE, SQL_ATTR_CONNECTION_POOLING, (SQLPOINTER)SQL_CP_ONE_PER_HENV, sizeof(int))))
        {
            PyErr_SetString(PyExc_RuntimeError, "Unable to set SQL_ATTR_CONNECTION_POOLING attribute.");
            return false;
        }
    }

https://github.com/lurcher/unixODBC/blob/master/DriverManager/SQLSetEnvAttr.c#L148-L157

    /*
     * we may do someting with these later
     */

    if ( !environment_handle &&
            ( attribute == SQL_ATTR_CONNECTION_POOLING ||
              attribute == SQL_ATTR_CP_MATCH ))
    {
        return SQL_SUCCESS;
    }

The only way to control pooling witn unixodbc is to use odbcinst.ini:

Pooling : global
CPTimeout: driver
PoolMaxSize: global
PoolWaitTimeout: global

The other option to do pooling is to add pooling layer on python. e.g.  asyncodbc
ref:

http://www.unixodbc.org/doc/conn_pool.html
https://github.com/lurcher/unixODBC/issues/42

@jikuja this issue seems to be fixed in unixodbc 2.3.12, which can be seen if you follow your unixODBC repo link again.
A bit of problem is that as of now Debian package repositories still have 2.3.11 as the latest version, and we need to install 2.3.12 manually. Below is a demonstration
check_pooling.py:
import os
import sys
import pyodbc

SERVER = os.environ['SERVER']
DATABASE = os.environ['DATABASE']
USERNAME = os.environ['USERNAME']
PASSWORD = os.environ['PASSWORD']
CONNECTION_STR = f'DRIVER={{ODBC Driver 18 for SQL Server}};SERVER={SERVER};DATABASE={DATABASE};UID={USERNAME};PWD={PASSWORD}'
QUERY = "select MAX(login_time) as login_time from sys.dm_exec_sessions where login_name = ? and program_name like 'python%'"

pyodbc.pooling = sys.argv[1].lower() == 'true'

for i in range(0, 5):
    connection = pyodbc.connect(CONNECTION_STR)
    cursor = connection.cursor()
    cursor.execute(QUERY, USERNAME)
    row = cursor.fetchone()
    print(row[0])
    connection.close()

Dockerfile
FROM mcr.microsoft.com/azure-functions/python:4-python3.10-core-tools

WORKDIR /root

# Download and install unixodbc 2.3.12
RUN curl -sO https://www.unixodbc.org/unixODBC-2.3.12.tar.gz
RUN gunzip unixODBC*.tar.gz
RUN tar xvf unixODBC*.tar
WORKDIR /root/unixODBC-2.3.12
RUN ./configure
RUN make
RUN make install

WORKDIR /root
RUN apt-get update
# Install dependencies of msodbcsql18 except unixodbc
RUN apt-get install -y libc6 libstdc++6 libkrb5-3 openssl debconf odbcinst
RUN curl -s https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc
RUN curl -s https://packages.microsoft.com/config/debian/11/prod.list | sudo tee /etc/apt/sources.list.d/mssql-release.list
# Install msodbcsql18 ignoring unixodbc dependency
RUN apt-get download msodbcsql18
RUN ACCEPT_EULA=Y dpkg --ignore-depends=unixodbc -i msodbcsql18_<IP_ADDRESS>-1_amd64.deb

RUN pip install --upgrade pip --user
RUN pip install pyodbc

ENV PYTHONUNBUFFERED=TRUE
ADD check_pooling.py ./

(My Linux skills are quite sloppy, please let me know if there is more optimal way to install/update to 2.3.12.)
docker build -t pyodbc_cp .
docker run -e ... pyodbc_cp python check_pooling.py true
docker run -e ... pyodbc_cp python check_pooling.py false
You will see that connection timestamps will differ when changing the argument true/false. Also, if you are on a slow/remote connection, you will notice a connection delay with "false", indicating that pooling is disabled, whereas with "true" they are obtained instantly.
For comparison, on version 2.3.11, which is installed through the package manager,
FROM mcr.microsoft.com/azure-functions/python:4-python3.10-core-tools

RUN curl -s https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc
RUN curl -s https://packages.microsoft.com/config/debian/11/prod.list | sudo tee /etc/apt/sources.list.d/mssql-release.list

RUN apt-get update
RUN ACCEPT_EULA=Y apt-get install -y msodbcsql18

# Uncomment the follwing lines to enable pooling:
#RUN mv /etc/odbcinst.ini /etc/odbcinst.old
#RUN echo "[ODBC]\nPooling=Yes\n" | cat - /etc/odbcinst.old > /etc/odbcinst.ini

RUN pip install --upgrade pip --user
RUN pip install pyodbc

ENV PYTHONUNBUFFERED=TRUE

WORKDIR /app
ADD check_pooling.py ./

you will see that the value of pyodbc.pooling is ignored and pooling unfortunately never works. unixodbc will however respect the setting in odbcinst.ini allowing you to enable pooling, however, in this case, again pyodbc.pooling will be disregarded and pooling will be always on.
With that said, it looks like the current issue may be closed, because with release of version 2.3.12 the behavior of pyodbc.pooling aligns with the documentation (@mkleehammer please validate).

Related unixODBC thread, including discussion about multiprocessing
https://github.com/lurcher/unixODBC/issues/149

^ That's a useful thread for understanding how multiprocessing can interfere with pooling, but IMO it is different from the above issue where unixodbc essentially disobeyed pyodbc's instruction to use pooling.

Hello @gordthompson thanks for your analysis.
We tried it and it works, however, we use pyodbc with Flask, and each request uses a different thread. As a result, the connection is not reused among different requests.
I also tested using your script, and the connections are not re-used:
import os
import sys
import pyodbc
import threading

SERVER = os.environ['SERVER']
DATABASE = os.environ['DATABASE']
USERNAME = os.environ['USERNAME']
PASSWORD = os.environ['PASSWORD']
CONNECTION_STR = f'DRIVER={{ODBC Driver 18 for SQL Server}};SERVER={SERVER};DATABASE={DATABASE};UID={USERNAME};PWD={PASSWORD}'
QUERY = "select MAX(login_time) as login_time from sys.dm_exec_sessions where login_name = ? and program_name like 'python%'"

pyodbc.pooling = sys.argv[1].lower() == 'true'

def main_func(iteration=-1):
    connection = pyodbc.connect(CONNECTION_STR)
    cursor = connection.cursor()
    cursor.execute(QUERY, USERNAME)
    row = cursor.fetchone()
    print(row[0], iteration)
    connection.close()

for i in range(0, 5):
    th = threading.Thread(
        target=main_func,
        kwargs={
            "iteration": i,
        },
    )
    th.start()

Do you know how can we make this work?
In pyodbc source code we have a comment saying:

threadsafety
The integer 1, indicating that threads may share the module but not
connections.  Note that connections and cursors may be used by different
threads, just not at the same time.

However, we have not found a way of modifying it.
Thanks

The threadsafety is a read-only flag required by the DBAPI to indicate pyodbc can be used on multiple threads.  The driver needs to also be threadsafe too, but pyodbc itself is.

@aAEeLll if you wanted to tag me, I can respond.
You are creating 5 threads and launch them all simultaneously. What do you expect? As they all demand a connection at the same time, the driver will have to create a connection for each. It does not seem to me a problem with threading.
The connection can be reused from the pool if the previous consumer released it back to the pool.
I tried my test in a scenario close to yours. I used FastAPI, which creates threads, too, in case of synchronous methods. I don't think it will be much different from Flask.
import time
import pyodbc
import fastapi
import uvicorn
from threading import get_ident
from constants import CONNECTION_STR, QUERY, USERNAME

app = fastapi.FastAPI()

@app.get('/cp')
def cp():
    connection = pyodbc.connect(CONNECTION_STR)
    cursor = connection.cursor()
    cursor.execute(QUERY, USERNAME)
    row = cursor.fetchone()
    connection.close()
    time.sleep(3)
    return get_ident(), row[0]

if __name__ == '__main__':
    uvicorn.run(app, host='<IP_ADDRESS>')

Test:
 % curl 'http://localhost:8000/cp' &
[1] 23575
 % curl 'http://localhost:8000/cp' &
[2] 23580
 % curl 'http://localhost:8000/cp' &
[3] 23627
 % curl 'http://localhost:8000/cp' &
[4] 23634
 % [140467339183872,"2024-04-16T21:40:21.900000"]
[1]    done       curl 'http://localhost:8000/cp'
 % [140467347576576,"2024-04-16T21:40:22.360000"]
[2]    done       curl 'http://localhost:8000/cp'
 % [140467355969280,"2024-04-16T21:40:22.360000"]
[3]  - done       curl 'http://localhost:8000/cp'
 % [140467372054272,"2024-04-16T21:40:22.360000"]
[4]  + done       curl 'http://localhost:8000/cp'

I use a pause of 3 s to have time to issue new requests before the previous one ends, so that the framework has to create new threads to serve them (otherwise it will be one thread). I issue 4 request in parallel quite quickly with <Up arrow> and <Enter>.
Note the connection 22.360000, which was reused by threads<PHONE_NUMBER>76576,<PHONE_NUMBER>69280, and<PHONE_NUMBER>54272.
Timing is crucial, though, the repeated request after a release must be within quite a short timeframe, like around 0.5 s or less. Using my previous script posted in February I repeatedly get a new connection every time time if I insert a pause of 0.9 s or more. It looks like the connection timeout is very short in unixODBC.

Thanks both for the clarification! 😃
Yes, I can confirm the problem was the connection timeout in unixODBC

https://github.com/mkleehammer/pyodbc/wiki/Tips-and-Tricks-by-Database-Platform#unixodbc-connection-pooling


Is it possible to get bounding box of text, including leading/trailing spaces?

Consider this input file:
<svg width="500" height="500" viewBox="0 0 500 500" xmlns="http://www.w3.org/2000/svg">
    <line x1="100" y1="100" x2="100" y2="400" file="yellow" />
    <text x="100" y="200" font-family="monospace" xml:space="preserve">
        <tspan fill="yellow">Line with trailing spaces.                             </tspan>
    </text>
    <text x="100" y="300" font-family="monospace" xml:space="preserve">
        <tspan fill="yellow">  A little leading spaces.</tspan>
    </text>
    <text x="100" y="400" font-family="monospace" xml:space="preserve">
        <tspan fill="yellow">This text already trimmed.</tspan>
    </text>
</svg>

(also available at https://gist.github.com/michaelkirk/fa3fbca0c87ec8e78eee6abf2cc96fac?short_path=efa0067)
When run through the draw_bbox example, we get something like:

To clarify what I'm looking for, compare to this output from chrome's developer tools, which bounds preserved whitespace:

So, Is it possible to get a text's computed bounding box with usvg, like that rendered in chrome?
In my spelunking, I believe this information is last available in: convert::text::text_to_paths text_to_paths, where we track last_x and the advance.
But then the calling method recomputes the bbox based on rendered glyphs in convert::text::convert, which does not account for trailing spaces.
For more context, my application is using usvg to render text widgets. Something like this:
// space pad, left aligned to 3 chars
let text_to_display = format("Foo: {:<3}", "A");

// note trailing spaces 👀
assert_eq!(text_to_display, "Foo: A  ");

// this method builds svg text, which it passes through `usvg::Tree::from_str`
let widget = render_my_application_widget(text_to_display);

// `bbox` is based on rendered size and location of glyphs,
// but there is no glyph for "space", so the widget bbox only reflects the right most rendered glyph
layout(widget, widget.bbox())

I'm using the xml:space="preserve", which seems to be sufficient to respect leading space (format("Foo: {:>3}", "A") works, format("Foo: {:<3}", "A") does not). I think this is because the spaces, though not rendering glyphs, offset the position of the subsequently rendered glyphs.
/cc @dabreegster

Interesting. Basically, the problem is that usvg::Node::calculate_bbox calculates a bbox of a path, which obviously doesn't account "spaces", because there are none.
So the only solution is to store the original bbox in usvg::Path.

Otherwise, the text handling code is correct. This is how it looks with debug bboxes:

Turned out that this is not only a useful feature, but actually a correct way to handle text. objectBoundingBox actually requires text bbox, not path bbox.


laravel8安装失败

执行 composer require overtrue/laravel-easy-sms 提示
Problem 1
- overtrue/laravel-easy-sms[2.0.0, ..., 2.0.1] require laravel/framework ^9.0 -> found laravel/framework[v9.0.0-beta.1, ..., 9.x-dev] but it conflicts with your root composer.json require (^8.75).
- Root composer.json requires overtrue/laravel-easy-sms ^2.0 -> satisfiable by overtrue/laravel-easy-sms[2.0.0, 2.0.1].

laravel 8 请使用 1.x 版本， 2.x 要求 laravel 9


GPU acceleration of programs under Hyprland not working

Hyprland Version
Hyprland, built from branch  at commit 739598717bbff79aad8599dcd0ca7a423682482a  (). Tag: ?  flags: (if any) debug
Bug or Regression?
Bug
Description
Mesa was bumped to 23.1.4 in nixpkgs but Hyprland's flake pins a version of nixpkgs with Mesa 23.1.3. Mesa has apparently added a strict version check recently, so when the compositor (Hyprland) is built against a different version of Mesa than everything else (i.e. from nixpkgs) including the active Mesa version being used as the GPU driver, Mesa falls back to llvmpipe and you don't get any hardware acceleration.
See also https://gitlab.freedesktop.org/mesa/mesa/-/issues/9527.
How to reproduce
You can reproduce this by compiling Hyprland against a version of Mesa that is different from the current version of Mesa being used as the GPU driver.
Crash reports, logs, images, videos
No response

You can fix this by setting inputs.hyprland.inputs.nixpkgs.follows = "nixpkgs"; in your flake.nix, for example. Note that this will require recompiling Hyprland; since that makes its inputs different from what's provided by Hyprland's binary cache, the cache can't be used anymore.


Consider redesigning docs in a single column

I find the akdubya.github.io/dustjs page difficult to read/scan because of the unconventional layout. Side-by-side columns that are taller than the viewport are confusing and disorienting; that's why no one uses them. It's harder to jump between sections when there are two dimensions to think about. The page is seriously inaccessible for people who have difficulty with reading comprehension. Please consider redesigning it as a normal, linear document. I've tried to check out dust on several occasions (I keep hearing good things about it) but I always give up after a few minutes because I just can't follow the docs.

You might try LinkedIn's new docs at http://dustjs.com

Thanks!


Race condition possible when deleting HealthCheck

Describe the bug
Rarely, an error will occur if a HealthCheck resource is deleted while it has a corresponding child workflow currently running.
To Reproduce

create a healthcheck (kubectl create -f examples/inlineHello.yaml)
while a corresponding workflow is running, delete the healthcheck (kubectl delete healthcheck inline-hello-abc01)
infrequently, but eventually, this will cause an error seen by the error condition log message indicated below.

Expected behavior
Regardless of WHEN a healthcheck is deleted, it and any corresponding workflow resources should be successfully cleaned up. The repeating workflow executions should also stop at this time.
Error Condition
2019-08-09T15:55:18.479-0700    ERROR   controllers.HealthCheck Error updating healthcheck resource     {"HealthCheck": "health/url-hello-dkkxt", "error": "Operation cannot be fulfilled on healthcheck.activemonitor.orkaproj.io \"url-hello-dkkxt\": StorageError: invalid object, Code: 4, Key: /registry/activemonitor.orkaproj.io/healthcheck/health/url-hello-dkkxt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a7c1b9b6-3e8f-4200-b41a-d81f53447ba2, UID in object meta: "}
github.com/go-logr/zapr.(*zapLogger).Error
        /Users/dmasselink/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
github.com/orkaproj/active-monitor/controllers.(*HealthCheckReconciler).watchWorkflowReschedule
        /Users/dmasselink/go/src/github.com/orkaproj/active-monitor/controllers/healthcheck_controller.go:229
github.com/orkaproj/active-monitor/controllers.(*HealthCheckReconciler).createSubmitWorkflowHelper.func1
        /Users/dmasselink/go/src/github.com/orkaproj/active-monitor/controllers/healthcheck_controller.go:147

Version
0.1.0

closed by #13