Race condition-free backups

I'm looking for ways to maximize reliability of LN node and I don't see any way to do backups reliably without race conditions. It'd be useful to have some API which calls into external service/command/... and blocks opening the channel until backup is completed. If the call fails for whatever reason channel is rejected. This way LND/power crashing at any moment couldn't cause loss of sats. As far as I can see, currently on may at best watch for FS changes and backup immediately hoping nothing crashes between opening a channel and performing backup.
Note that such situation isn't crazy-improbable because reasonable backup solutions relies on a remote machine and the connection may be unreliable.
There are two ways to do this: GRPC or having something like backupcommand option in LND that runs the specified command and waits until it exits. Pros and cons:

Shell looks easier to implement - for client and maybe also for LND; note that GRPC probably requires two methods (but maybe I'm misunderstandig GRPC).
GRPC can more easily pass structured metadata (channel point, remote peer, ...), shell would have to parse it and could be vulnerable to classic shell bugs (someone sets node alias to ; rm -rf / and if mishandled the system is nuked)
Both can run backup tool under a different user but scripting languages can't use SUID which may be annoying.
Not having unneeded idle process in RAM is nice but systemd can solve it using socket activation with socket saving. The question is whether GRPC implementations support this exotic feature (my guess is not).

I think I myself would prefer running a command but can adapt and respect the choice of using GRPC.

I think the main problem is the order of events here. lnd will only craft the channel backup once it sent the FundingSigned message. And after you've sent that message to your peer, there is no way to "reject" the channel anymore (only force-close).
But with some refactoring I think it would be possible to create a new BackupInterceptor RPC that is invoked before sending the FundingSigned message. If a client is registered to that interceptor then it needs to acknowledge the channel as being backed up before the final signature is sent.
gRPC is definitely the preferred way to implement something like this!

Interesting didn't expect the actions would be in this order although doesn't sound difficult to change. Anyway shouldn't the backup be created even before signing to account for external signers?

didn't expect the actions would be in this order

I think that's just a consequence of the channel update notification mechanism being re-used for triggering the backup creation. And that notification only triggers after the channel is fully signed. But a new backup interceptor mechanism could hook in earlier and ensure atomicity. And yes, I'd expect the mandatory interceptor mechanism of the RPC middleware interceptor to be re-used for a new interceptor like this.


GH-310 Add quoted argument resolver. Add related tests.

Resolver GH-310

https://cdn.discordapp.com/attachments/896936981861392446/1178851334577991721/20231128-0012-39.3250561.mp4

https://cdn.discordapp.com/attachments/896936981861392446/1180271920114061393/20231201-2217-46.4150947.mp4

It seems that the modified functions have been basically completed, and maybe they can be merged.


:pencil: Typo in README wisley --> wisely

I saw you post a PR in my fork, I'll close this.

I saw you post a PR in my fork, I'll close this.


Uncaught RuntimeError: abort(OOM)

Describe the bug
A clear and concise description of what the bug is.
pthread sent an error! blob:http://localhost:8000/73a709bf-5aad-42ad-9d99-c65f6a3bfc09:1: Uncaught RuntimeError: abort(OOM). Build with -s ASSERTIONS=1 for more info.

To Reproduce
Steps to reproduce the behavior:
As soon as I added the -ignore_loop 0 option, it just errors out.
-ignore_loop 0 -i Grape.gif -i juice.gif -filter_complex [1:v]overlay=format=auto,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse preview.gif

Expected behavior
A clear and concise description of what you expected to happen.
Not to run out of memory.
Desktop (please complete the following information):

OS: [e.g. iOS] MacOS
Browser [e.g. chrome, safari] Chrome
Version [e.g. 22] 99.0.4844.84

Related to #183 #78


Generic Type Keys

Hi @MichaCo,
I have started using your library and it is working great for me.
Right now for Dictionary, Memory Handle and all other handles, only string is supported as keys.
If you can support Generic Type Keys, it would be great.
Usage
I have an object which identifies unique request which is made to database server. I want to cache the response of the database query, and for that I want the key to be set as that object.
Sample Code
I am looking for something like the below signature:
void Put<T1>(T1 key, TCacheValue value);

Hi @mansoor945
thanks for the feedback!
There are currently no plans to add complex objects as keys.
That was by design, primarily because distributed caches use strings as keys, obviously.
To use complex objects, you can/have to implement the transformation to a string yourself. Either override ToString or serialize your key, whatever.
It is not really hard to do, that's why I never saw the point of adding and better keep it simple...


(#326) Add _statement suffix to all statement translating functions

Close #326

@kolumb looks goode to me! :+1: Thank you for your contribution!


Exclude certain directories while performing search

Hello,
I just went through,
https://blog.ambar.cloud/mastering-ambar-search-queries/
but couldn't find a way to exclude certain file type or a directory from result.
Is there any way to do so yet?

Nope, it can't be done via query. But you can exclude files during crawling. Check the latest installation instruction


Individual commands should be plugins

Currently all commands have a resource file(form layout) somewhere in https://github.com/bitshares/gwallet/tree/master/resources, a header file in https://github.com/bitshares/gwallet/tree/master/include/panels and implementation in https://github.com/bitshares/gwallet/tree/master/panels
Plus the cpp haves to be included to the cmakelist  and some other changes need to be done to https://github.com/bitshares/gwallet/blob/master/panels/wallet.cpp for the command to show up in the list.
I think each command should be a plugin so the transfer plugin will be something like:
plugins/transfer/
  includes/
     transfer.hpp
  resources/
    transfer.xrc
  transfer.cpp
  CMakeLists.txt

So we will have a core and the commands isolated from it where new commands could be implemented easier than currently for newcomers among other benefit the modular solution will bring.

work for this started at https://github.com/bitshares/gwallet/commit/4ba72c41fd9156c55911ff9f3ba417927318affc

In addition to https://github.com/bitshares/gwallet/commit/4ba72c41fd9156c55911ff9f3ba417927318affc the following commits are related:
https://github.com/bitshares/gwallet/commit/36ac0738de89ea0e3a4d67750a169f4440f97bd4
https://github.com/bitshares/gwallet/commit/5bdfed4bc9220e7cc4abc41d554ba732a15b53d9
https://github.com/bitshares/gwallet/commit/5cf9639c919753b23654bdafdd84775e17b42d23
1 plugin for each commands was too much so i decided to make it by category. We have 5 plugins now with code of the related commands inside:

assets: create, issue, transfer, sell, borrow, cancel order, update feed producers.
accounts:  register, create with brain key, upgrade, set proxy.
htlc: create, redeem, extend
getters: about, info, committee member, witness, account, account history, asset, order book, active witnesses, active committee, htlc
governance: vote for witness

There are some improvements and cleanup to be made however they will be done in new issues.


EKSPlugin

The AWS X-Ray SDK for Java has an EKS Plugin available. This adds the container ID, cluster name, pod ID, and the CloudWatch Logs Group.
This feature would be greatly beneficial for anyone running python workloads in EKS

Hi @james-hercock-vgw, thanks for creating this feature request
I also wanted mention that this feature is supported through AWS resource detectors (specifically the EKS resource detector) in opentelemetry python. This documentation provides all the steps on how to set up opentelemetry with X-Ray and configure resource detectors as well


TBS Discovery J120 Power

The TBS Discovery build shows the J120 hooked up directly to the power coming from the battery. I thought the power fluctuations coming from the battery had the potential to damage the equipment. Have you guys had any issues with this?

While we have not had any issues with such connection, it may indeed cause some problems.  First of all, as long as you are using LiPo batteries 2S-4S you should be fine - J120 power voltage range is 7-17V. Using anything above 4S will damage the board. We tried 2S, 3S and 4S batteries on different platforms that used J120 (drones and rovers) and they all worked fine.
However, even when a proper battery is used, there still might be problems due to voltage spikes/drops. As long as you are monitoring the battery level and avoid flying when the battery level is low then most likely you should not experience any problems. In case Jetson is unexpectedly powered down midflight, PX4 safety mechanism will kick in and the vehicle will switch from Offboard mode to the last mode (e.g. Position Hold). PX4 in Offboard mode requires companion computer to publish updates at the minimum rate of 2Hz.
Now, the reason we had to make such connection is TBS Core power module that we planned to use originally to power Jetson, does not provide enough power for the Jetson: its 12V output is only 600mA, while Jetson requires at least 1A to power on reliably at that voltage.
Ideally, there should be a completely separate battery (e.g. small 2S battery) to power Jetson alone, but TBS Disco platform is too small for that.

Just sharing my experience.
I had similar setup J120 + TX1 connected directly to 4S battery. We use 4in1 ESC that have very minimal filtering capacitors. As a results there were ~5V spikes when motor started/stoped. My J120 did not survived that :( it released the blue smoke... along with ZED camera, the TX1 survived somehow...
That was expensive smoke... Upon contacting Auvidea they advised to setup 12V DC/DC converter in between battery and the carrier board. Fully charged 4S is 16.4V which is very close to the J120 limits.

That sucks, the J120 board costs almost as much as the TX2! If you don't mind a little extra weight then using transmitter lipos is the best solution I've found. It has undervoltage protection (9V) and overcurrent protection (3A). I use these for my LIDAR aswell.
https://hobbyking.com/en_us/turnigy-9xr-safety-protected-11-1v-3s-2200mah-1-5c-transmitter-pack.html

ZED camera cost about the same :) blue smoke is expensive.
After this disaster I had extra battery but it was bothersome. I am using the DC/DC converter for 1.5 year now and all is ok.

Sorry to hear that. As I mentioned, ideal solution would be a separate battery but if it's not possible then appropriate voltage regulator should help.


Refactor/frontend config to backend

What (if anything) did you refactor?

Refactored to consume config backend data

@mexi-cano Could you fix the merge conflicts?

@mexi-cano Could you fix the merge conflicts?

Do you have any suggestions to make that easier? The original branch was closed so when I pushed an update, a new one was created that's disconnected from the dev branch.

@mexi-cano Could you fix the merge conflicts?

Do you have any suggestions to make that easier? The original branch was closed so when I pushed an update, a new one was created that's disconnected from the dev branch.

The problem might be that the pr was merged, but then reverted. You could try to revert 836407c1, which should add back the changes from the previous branch.

I've merged in the dev branch but there are still merge conflicts where it tries to add files that were deleted or revert changes. I'm hesitant to try to remerge a second time and get stuck in this loop. Any suggestions?  @CalebPena

I've merged in the dev branch but there are still merge conflicts where it tries to add files that were deleted or revert changes. I'm hesitant to try to remerge a second time and get stuck in this loop. Any suggestions? @CalebPena

Did you merge the latest version of of dev?

I've merged in the dev branch but there are still merge conflicts where it tries to add files that were deleted or revert changes. I'm hesitant to try to remerge a second time and get stuck in this loop. Any suggestions? @CalebPena

Did you merge the latest version of of dev?

Yes. The last commit was an update to referralOptions which is now on the backend. The previous commit changes can be found in this branch now.

Closing in favor of config v2


Remove stream info cache.json

remove stream_info_cache.json

delete this too https://github.com/vv181/lbry/blob/6574e2c0fc095afb4fabd0a42540e7280422e9c4/lbrynet/daemon/Daemon.py#L659

Rebase the three commits into a single commit please.
Otherwise LGTM.

due to my limited git knowledge, I'm messed this up,
here is the new pull request https://github.com/lbryio/lbry/pull/780


No such module 'KituraTemplateEngine'

Getting this error in Router.swift on version 0.13.1. This error only appears in Xcode. When using swift build from the command line, the build is successful.

got the same problem,
and got this error when I remove the KituraTemplateEngine:
ld: library not found for -lCHttpParser for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

This appears to have popped up in the 5/3 snapshot. It didn't seem to occur in 4/25. Will continue investigating.

I discovered a workaround for now that compiles for me. In Xcode, rename the Kitura-TemplateEngine target to KituraTemplateEngine.

@kweinmeister Hmm. That's interesting. So currently, I will need to do that each time I run swift build --update right? Hopefully we can get this fixed permanently, but I guess that helps for now.

I believe you also have to point KituraNet and Kitura targets' Library Search Paths to $(SRCROOT)/.build/debug for a successful Xcode compiling.

I believe I have fixed the issue with building in Xcode.
Using the latest Kitura (0.13.4) and Swift 3.0 05/03, you can now build a Kitura based application in Xcode without manually fixing anything.
Note: The issues around running the application from Xcode still remain.

I tested this and the issue has been resolved.

This issue was reported to the Package Manager team and they have resolved it on their side today.
https://bugs.swift.org/browse/SR-1444

Thank you for reporting this and following up @dsperling!


adds AWS S3 storage system

This pull request adds a storage backend based on Amazon AWS S3 storage system. See #15
The AWS S3 storage system allows us to treat:

Each paste as an Amazon S3 object
Each past_id as an object key (or key name)
Each metadata as an object metadata (which is accessible in a limited way, and which cannot be merely modified)

You find more information regarding Amazon S3 objects and their data, key, and metadata here.
Due to the lack of options, this PR does not implement any preventive approach for each communication with the Amazon S3 storage (e.g. setting a timeout).
Pair-programmed with @c-w

Thanks for another great backend service!

Hi @DaKnOb, @c-w and I finally fixed the merging conflicts for this PR and we also reduced redundancy of code by moving repeated decorator functions into utils.py .
This is ready to be merged!


Rollout options

"Enable a Staged Rollout of a specific feature on your tenant" section:
Password Hash Sync (on its own) should be a supported option and should be added to the list (for example, as the top option!).
The third bullet (which says "Not supported - Password hash sync + Pass-through authentication + Seamless SSO") should appear AFTER the list of supported items (i.e. either in a separate list or in the next paragraph) - it doesn't belong in the list of options which CAN be rolled out!

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 3f2a7330-82fa-9c0d-0f34-b2a99ef13c64
Version Independent ID: d941f435-3443-02a1-57c0-7f4173173249
Content: Azure AD Connect: Cloud authentication via Staged Rollout - Microsoft Entra
Content Source: articles/active-directory/hybrid/how-to-connect-staged-rollout.md
Service: active-directory
Sub-service: hybrid
GitHub Login: @billmath
Microsoft Alias: billmath

@si-chan
Thanks for your feedback! We will investigate and update as appropriate.

@si-chan
Thanks for bringing this to our attention.
I'm going to assign this to the document author so they can take a look at it accordingly

Thank you for submitting this issue.  Unfortunately, due to high volumes I was unable to respond to this in a timely manner.  At this point, I am going to close this due to it's age and the possibility that it may have been addressed.  If this is still a problem, please consider submitting a new issue.  Thank you.
#please-close


Testing the plugin with Redmine 4.1.0

Hello
I cannot wait to test this amazing plugin.
Unfortunately, I have some issues when I try to use it on my Redmine instance (using Redmine 4.1.0, without any other plugin).
I would like to help, but the errors I get are in the JavaScript part, and I am not an expert with VueJS.
Do you know if the plugin is compatible with Redmine 4.1.0?
Do you currently use it in production?
Here are the errors that show up in my browser console:

Thank you for this impressive work!

Sorry I did not tested with Redmine 4.1 but I just created a fix in the feature/dockerize branch, can you try and get back to me ?

For information, the element requested by the plugin does not exist anymore in recent version of redmine.

Just tested the feature/dockerize branch. It works well with Redmine 4.1.0.
Thank you, Victor!
About the missing element... I don't see any error in my logs. Do you have some details about this issue?

Nice I'll merge then.
Nevermind the error , it won't happen ;)

I've just seen your changes regarding the projects dropdown.
Great job! Thanks.

Thx :) probably need a bit of refactoring... in the next days !


Regexps for maps

feature / merge request : make regexp pattern checking available for maps with integer keys ...
modification in core.py
    # old
    # is_present = any([re.search(required_regex, v) for v in value])

    # new
    #
    is_present = any([re.search(required_regex, str(v)) for v in value])

Hi! I was just about to submit the same pull request, as I'm hitting the same issue as you. I think this would be an improvement. But, I realized that there is a larger issue here: kwalify seems like it can handle only string map keys, while YAML allows for any map keys. This is particularly problematic because YAML will automatically guess type, so if you ever accidentally type a key that looks like an int, then pykwalify will crash with this cryptic error:
  File "/usr/local/lib/python2.7/dist-packages/pykwalify/core.py", line 549, in _validate_mapping
    is_present = any([re.search(required_regex, v) for v in value])
  File "/usr/lib/python2.7/re.py", line 146, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer

It made me wonder: Would it be possible extend the pykwalify syntax to allow map keys be first-class citizen nodes, rather than strings? In that case, they could be any type, and they could also have additional constraints put on them, such as range. In the absence of that, this patch is a good workaround.
@Grokzen, what do you think?

OK, looks like you can do this via a custom python extension. If you do func: validate_map_keys and define def validate_map_keys() as an extension, then you can check everything to your heart's content. That said, native support for this in the syntax without writing code would still be super handy, and the ability to apply regexes would also be handy.

@brownberry @martingkelly I have to think about this because casting values to str to enable regex checks for other types feels weird in the kwalify context. To regex check a value feels like you are doing it wrong in some way.
On the idea to reworking the map keys to first-class citizens feels like a major project to work on and it would uproot a lot of stuff in the code-base. In theory sure, it could be a good idea to make working with keys easier, but on the other hand it would go away so far from the kwalify standard i am not sure i want the validation framework to go in that direction right now.
I will think about it but i will not act on it right now.


[Proposal] router switch conditionals

Router Switch Conditionals
What is
Sometimes we may wish to have a better way of controlling if a route is going to be rendered or not, without a redirect (as the wiki proposes). With this we can pass a conditional to a route as a prop so the switch may see if the route can render.
To this day work-around
While we don't have this resource, what I usually do, is pass a function to the path parameter of the route (inside a switch), an example as follows:
var user = null;

function isGuest(route) {
   return user === null? route:"impossible route";
}

export default function App (props) {
   return (
       <Switch>
              <Route path={isGuest("actual/route")} component={Component} />
       </Switch>
   );
}

What I'm doing is rendering an impossible route if the condition inside the function is not met, with this I can generate tons of functions to describe those conditions, such as: isGuest, isLogged, isAdmin. Of course, that this method is not ideal.
Sample
How the router switch conditionals would work:
var user = null;

export default function App (props) {
   return (
       <Switch>
              <Route path="actual/route" condition={user === null} component={Component} />
       </Switch>
   );
}

Returning true would allow the switch to consider this route when choosing which one to use.
Changes
The changes in the source code would be near to minimal
Switch class
class Switch extends React.Component {
  render() {
    return (
      <RouterContext.Consumer>
        {context => {
          invariant(context, "You should not use <Switch> outside a <Router>");

          const location = this.props.location || context.location;

          let element, match;

          // We use React.Children.forEach instead of React.Children.toArray().find()
          // here because toArray adds keys to all child elements and we do not want
          // to trigger an unmount/remount for two <Route>s that render the same
          // component at different URLs.
          React.Children.forEach(this.props.children, child => {
            if (match == null && React.isValidElement(child)) {
              element = child;

              const path = child.props.path || child.props.from;
              const cond = child.props.condition || true;

              match = (path || cond)
                ? matchPath(location.pathname, { ...child.props, path })
                : context.match;
            }
          });

          return match
            ? React.cloneElement(element, { location, computedMatch: match })
            : null;
        }}
      </RouterContext.Consumer>
    );
  }
}

The only thing that we are doing is adding a const that reads the condition and passing it for the match test altogether with the route comparison.

This reminds me of the following two issues: #6745 and #6683. But there is more about this:

if there should be such a feature we should name it when instead of condition to be consistent with <Prompt />
it also could be solved via prop when we support function for prop. But this seems a little awkward to me.
This could also be solved via <Route children={} /> (or <Match /> in the future).

Anyway. I think this could be a good extension for React Router.

You should do this as a condition of rendering the Route at all.
<Switch>
  {user === null && <Route path="actual/route" component={Component} />}
</Switch>

That would skip the render entirely, which is better for performance.

So your argument would apply to prompt's when too.
But anyway, I think this is horrible for the code's visual.

I think it is a fundamental question about which design line we prefer for React Router. Either we go with when for <React /> and <Prompt /> or we drop it for both. (Yes, I know that <Prompt /> will be dropped in some upcoming versions.)

Personally, I dislike inline-conditionals in my JSX and avoid them whenever possible (by moving the decision to the component in question). It makes for a lot of visual clutter.
A when prop on Redirect and Prompt is preferable to that the clutter. And since this such a minor impact on RR code, I vote for adoption.

There is one thing I dislike about this idea: when there is a when condition I don't expect that there is also a path prop. Either or, but not both. Maybe this is also a case for your new <Match /> component?
<Switch>
  <Match when={(location) => location === "actual/route" && isGuest()}>

  </Match>
</Switch>```

<When> actually would work as a safeguard for the route, it's a optional with default value of true. Maybe Match would do the trick, but this would be used as a content filter depending on what user is or if it is logged.

I get your point but I dislike it after all. Looks redundant to me.

I don't think it's redundant. Even less so with the Match component:
<Match path="/projects/{id}" when={isProjectMember()}>
   <ProjectMember/>
</Match>

It's basically a sentence. Keeping code readable and intuitive to understand is IMHO more important than concerns about finesse.
The function has more flexibility, but I don't think the added complexity justifies that small gain. And it might spawn some really wonky hacks that we shouldn't encourage.

I really don't agree with this. It makes sense in the case of <Prompt>, where you're mapping an imperative action to React's declarative form. But you shouldn't be rendering a <Route> at all if you intend it not to display due to a conditional.
It's the same reason we don't have an <If> element in JSX. You have to bail out to regular JS to handle this stuff. Maybe JSX 2 will eventually address this, but for now, you should be wrapping the entire element's existence in a conditional. That's just idiomatic JSX and React.

The "don't render if it doesn't do anything" applies just as much to Prompt.
I accept an argument for performance, but with RR related components, that's pretty negligible in almost all applications.
To me, this is more an issue of preference. I'll accept the majority opinion, but I don't agree that this is a straightforward case of "that's just how JSX works". It's a flaw in JSX we can provide a nice and easy workaround for. And it only costs giving up the purism.
(That's all I have to say about this. I'll submit a PR if consensus can be reached)

In my sincere thought, this is a long due addiction. But as "newbie" in programming, I always abstained from any discussion. Is it necessary? Of course not, nor react, you could build everything from the ground up, nasty as it looks, it will work. But this is something that makes react-router modular in a way that could help both new comers and experienced professionals, it also looks good and is readable. From my look,  would disarm this feature, but one or another should be end up into the API.

@timdorr If we move the part where we check the condition in when to the <Switch /> component then there would be no unnecessary render of any <Route /> component. Actually, this is exactly what we are doing right now for the path and from prop.
const path = child.props.path || child.props.from;

(Source: https://github.com/ReactTraining/react-router/blob/master/packages/react-router/modules/Switch.js#L31)

it does feel redundant as it's only "if" part ot the sentence. What about "else" part? Sooner or later somebody will start asking for it.
We are talking about a router here, not just some random component toggle. Route "unavailability" should be always handled and just falling through to 404 is just bad design / user experience as you are not providing real feedback for the situation. To some extent it would make sense to have a static routing in place. But to extend the original switching issue - please allow <React.Fragment> wrapping support for conditional routing grouping.

@allankikkas, else is already included because of the switch

<Switch>
    <Route path="hi" when={false} />
    <Route path="hi" when={false} /> //This works as else if
    <Route path="hi" /> //This works as else
</Switch>

While I'm not privy to the exact details and the API may change, this post shows that the next version of RR will have a validate prop on Routes, which you could hijack to accomplish this.
In the meantime, there are a number of ways to accomplish this behavior without adding API.

Tim's suggestion of {condition && <Route ... /> for one off conditions
Custom Switch/Route components. Neither component is particularly complex (the Route has a bunch of boilerplate/prop checks but nothing too crazy), so copying the source and modifying to suit your needs isn't too onerous.
Rendering different route sets based on a condition. This works well if you split a number of roots on one condition.

user === null
  ? <NonAuthRoutes />
  : <AuthRoutes />

I'd actually recommend #3 in general anyways. It's much less likely that you make a small mistake and then accidently expose private routes to public users.

@RafaelCorreaQuantik yes but with this syntax you are polling your code with duplicate paths. path can be complex and have multiple conditions and this syntax without inline else statement defeats its purpose of being simpler. If this statement can be somehow mapped to component switcher there then it would make sense (component={() => {https://i.imgur.com/Rggzlxh.mp4} is not the way as compnent will get re-mounted every time path changes which is not the desired outcome}

If using an inline conditional is better for performance then you can still use a conditional even if you implement this feature. The user can then decide which way to go with (readability vs performance).


gtk4: fix FileChooserDialog with language bindings that uses dlopen

Description of changes
https://discourse.gnome.org/t/strange-filechooserdialog-behaviour-in-gtk-4-10/14435
Fixes https://github.com/NixOS/nixpkgs/pull/226138#issuecomment-1509615369
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

:+1:  on backporting this.

Looks like this is already part of 4.10.2.

nautilus 44.1 now requires gtk 4.10.3, can we just bump gtk4 in master since it has 509 rebuild on x86_64-linux?

Sounds good.


Update incorrect data processing in DataCollatorForChatML

What does this PR do?
Fix the extra BOS token and the absence of an EOS token in the returned input_ids, and potentially the absence of a target string in the returned labels.

Fixes #2169
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[x] Did you read the contributor guideline,
Pull Request section?
[x] Was this discussed/approved via a GitHub issue? Please add a link
to it if that's the case. (https://github.com/huggingface/trl/issues/2169)
[x] Did you make sure to update the documentation with your changes? Here are the
documentation guidelines.
[x] Did you write any new necessary tests?

Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

awesome @ruijunfeng can we also have a test for this?

awesome @ruijunfeng can we also have a test for this?

Sure thing, I have tested it on Llama2 series and gemma1 series with my own dataset, and it seems to work well. Let me know if you need me to provide anything.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

sorry for the misunderstanding, I meant something like:
class TestDataCollatorForChatML(unittest.TestCase):
    def setUp(self):
        self.tokenizer = AutoTokenizer.from_pretrained("gpt2")
        self.tokenizer.pad_token = self.tokenizer.eos_token
        self.collator = DataCollatorForChatML(tokenizer=self.tokenizer, max_length=20)

    def test_data_collator(self):
        examples = [
            {
                "messages": [
                    {"role": "user", "content": "Hello!"},
                    {"role": "assistant", "content": "Hi there! How can I help you today?"},
                    {"role": "user", "content": "What's the weather like?"},
                    {"role": "assistant", "content": "I'm sorry, but I don't have access to real-time weather information."},
                ]
            },
            {
                "messages": [
                    {"role": "user", "content": "Tell me a joke."},
                    {"role": "assistant", "content": "Why don't scientists trust atoms? Because they make up everything!"},
                ]
            }
        ]

        batch = self.collator(examples)

        self.assertIn("input_ids", batch)
        self.assertIn("attention_mask", batch)
        self.assertIn("labels", batch)
        self.assertIn("prompts", batch)
        self.assertIn("prompt_attention_mask", batch)

        self.assertEqual(batch["input_ids"].shape[0], 2)
        self.assertEqual(batch["attention_mask"].shape[0], 2)
        self.assertEqual(batch["labels"].shape[0], 2)
        self.assertEqual(batch["prompts"].shape[0], 2)
        self.assertEqual(batch["prompt_attention_mask"].shape[0], 2)

        # Check if the shapes are consistent
        self.assertEqual(batch["input_ids"].shape, batch["attention_mask"].shape)
        self.assertEqual(batch["input_ids"].shape, batch["labels"].shape)
        self.assertEqual(batch["prompts"].shape, batch["prompt_attention_mask"].shape)

        # Check if the prompts are shorter than or equal to the full input
        self.assertTrue((batch["prompts"].shape[1] <= batch["input_ids"].shape[1]).all())

so we can explicitly check for the incorrect data processing and the fix you so kindly provided

Hi there, I have run your test code, and I think your test code has a small mistake. You are using the tokenizer for GPT-2:
self.tokenizer = AutoTokenizer.from_pretrained("gpt2")

However, GPT-2 does not have a default chat_template, so it will cause an error in this line of DataCollartorForChatML
self.tokenizer.apply_chat_template(messages, tokenize=False).

I believe the correct way to test this is by manually setting the chat_template for the tokenizer, like this in your setup function:
tokenizer.chat_template = "{{ bos_token }}{% for message in messages %}{{ message['role'] }}: {{ message['content'] }}{% endfor %}{{ eos_token }}".

Alternatively, you could use a model that has been fine-tuned on instructions, such as Llama-Instructed, whose tokenizer has a default chat_template.

sorry again for the misunderstanding, what i wanted to say is that you can use the above as a template to write the tests in your PR

@lewtun i added a test that fails on main and passes here and @ruijunfeng I pushed it into your PR

@ruijunfeng kindly run the pre-commit in the root dir of TRL to fix formatting

@kashif and @lewtun, thank you both for adding the tests and comments. I’ve double-checked the tests and made updates to the assert statements and comments to improve consistency and clarity. Additionally, I noticed that the current check for the EOS token in input_ids only verifies its presence. I have modified it to ensure that the last token of input_ids is the EOS token for a more thorough check.

yes i didnt take that into account

@qgallouedec fixed the test taking padding into account

@qgallouedec I have:
input_ids[15:]  # 15 first tokens are padding
[1, 518, 25580, 29962, 1724, 338, 2253, 1135, 22769, 29973, 518, 29914, 25580, 29962, 25685, 29889, 29871, 2]
(Pdb) self.tokenizer(self.tokenizer.apply_chat_template(self.examples[0]["messages"], tokenize=False), add_special_tokens=False)
{'input_ids': [1, 518, 25580, 29962, 1724, 338, 2253, 1135, 22769, 29973, 518, 29914, 25580, 29962, 25685, 29889, 29871, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

@ruijunfeng can you kindly check with the current refactoring of the datacollator, I have simplified it

@kashif Hi there, I still found a small bug in the refactor code. I used the dataset in your unit test and print out the results like this:
>>> tokenizer.decode(data["prompts"][0])
'<s><s><s><s><s><s><s>user: What is better than ugly?</s>'
>>> tokenizer.decode(data["input_ids"][0])
'<s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s>user: What is better than ugly?assistant: Beautiful.</s>'
>>> data["labels"][0, -6:]
tensor([ -100, 22137, 29901, 25685, 29889,     2])
>>> tokenizer.decode(data["labels"][0, -5:])
'istant: Beautiful.</s>'

Seems like the labels mistakenly wrap part of the "assistant: " and the prompts has missed the "assistant: ".

just trying to reproduce this on my end, I have as output from the data collator:
self.tokenizer.decode(input_ids[0])
'</s></s></s></s></s></s></s></s></s></s></s></s></s></s></s><s> [INST] What is better than ugly?  [/INST]'

and the labels are only set for the completion:
labels[0]
tensor([ -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,
         -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,
         -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100, 25685,
        29889, 29871,     2])

self.tokenizer.decode(labels[0][-4:])
'Beautiful. </s>'

at which point are you printing the data from?

@kashif Sorry I used a wrong version of the code. I have tried it again and the refactored code is all good.


Oxford Common File Layout (OCFL): Application-independent file management within digital repositories

The Oxford Common File Layout (OCFL, https://ocfl.io/ ) initiative is developing a specification for an application-independent approach to the storage of digital information in a structured, transparent, and predictable manner. The focus is on the  promotion of long-term object management best practices within digital repositories that might be built with Samvera, with other technologies, or as part of a mixed ecosystem.
The session will be split into two parts. The first will be a description of the OCFL initiative through the lenses of motivating use cases and past experience. The second portion will describe how we imagine OCFL fitting into our different local institutional archival storage ecosystems and will summarize the current status of the specification.
Simeon Warner<EMAIL_ADDRESS>Rosalyn Metz<EMAIL_ADDRESS>

Simeon confirmede (also for Rosalyn) and recording OK 8/8

Simeon registered 8/14 - system info


Correcting broken snippet for javascript console

Correcting two broken snippets, cv and cod weren't working with ${0}, but they do with ${1}

Thanks!


MSFT_xExchInstall: Fix PSSA Issues

PowerShell Script Analyzer is bringing up a few issues for the MSFT_xExchInstall resource.
No errors, but there are a few warning/informational issues that should be either fixed or excluded.
This is the result of running Invoke-ScriptAnalyzer:

(Please disregard issues brought up by the PSDSCDscExamplesPresent and PSDSCDscTestsPresent rules for now.)

Hi @kwirkykat, how are we supposed to handle the very first error in this output? We do in fact want to require a reboot before proceeding, and all examples I've seen to require a reboot use $global:DSCMachineStatus = 1.

We are almost done compiling a list of which PSSA rules we are going to consider as 'errors' (cannot be suppressed), 'warnings' (can be suppressed with approval), and 'ignore' (we won't check them). The list is almost finalized. We will post it ASAP!
But for now I can tell you that the 'PSAvoidGlobalVars' rule is in the 'warning' camp so this rule can be suppressed and since setting $global:DSCMachineStatus is the correct way to trigger a machine restart this suppression is definitely an approved case.
As the list currently stands for the rules mentioned here (still subject to debate and change):
PSAvoidGlobalVars - Warning
PSUsePSCredentialType - Warning
PSPossibleIncorrectComparisonWithNull - Error
PSUseShouldProcessForStateChangingFucntions - Ignore
PSAvoidUsingInvokeExpression - Error
PSDSCUseVerboseMessageInDscResource - Warning

So what's the correct way to suppress this particular warning then?

For now, add the rule suppression above the param block of any function that uses this global var:
[Diagnostics.CodeAnalysis.SuppressMessageAttribute("PSAvoidGlobalVars", "")]
param()

This is an approved suppression.

Consolidating into https://github.com/PowerShell/xExchange/issues/149


upgrade all dependencies and compatible with PHP 8.1

Description
Hi,
In this PR, I upgraded all dependencies and now it's compatible with PHP 8.1.
kalnoy/nestedset version 5 requires some illuminate packages below 8.43 and those illuminate have deprecated warning for PHP 8.1 (return type) so needs to upgrade laravel/framework and nestedset.
How To Test This?
run composer install.
Bagisto should work with PHP 7.4 to 8.1

Hi, @ghermans and @jitendra-webkul,
After this, we need to drop all the support for PHP 7 and 8.0 because there are many more new syntaxes that come and we will not be able to write new codes if we continue giving the support.
Even Laravel also drops the support.

Laravel 8.75 alerts are popped up I think we should start with PHP 8.1 and Laravel 8.7 and up.

Hi @AmooAti,
Please resolve conflicts.

Hi @devansh-webkul
Conflicts are resolved.


ms-rest-js v1.11.0 breaks ms-rest-azure-js

node_modules/@azure/ms-rest-js/es/lib/axiosHttpClient.d.ts(6,25): error TS7016: Could not find a declaration file for module 'tunnel'. '/home/vsts/work/1/.tmp/ms-rest-azure-js/node_modules/tunnel/index.js' implicitly has an 'any' type.
  Try `npm install @types/tunnel` if it exists or add a new declaration (.d.ts) file containing `declare module 'tunnel';`

due to the move of @types/tunnel from dependencies to devDependencies in #422

Fixed by #434


Not able to sign using above steps.

Getting below error when trying to login after following above mentioned steps. Screenshot of Azure AD is also of earlier version.
"Sign in
Sorry, but we’re having trouble signing you in.
AADSTS650056: Misconfigured application. This could be due to one of the following: The client has not listed any permissions for 'AAD Graph' in the requested permissions in the client's application registration. Or, The admin has not consented in the tenant. Or, Check the application identifier in the request to ensure it matches the configured client application identifier. Please contact your admin to fix the configuration or consent on behalf of the tenant"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9d8e29ab-1b2f-a412-977c-c536623e7ab2
Version Independent ID: 4fddf50c-0ffe-765d-723b-38e0e473c2d1
Content: Tutorial: Azure Active Directory integration with SAP Business ByDesign
Content Source: articles/active-directory/saas-apps/sapbusinessbydesign-tutorial.md
Service: active-directory
Sub-service: saas-app-tutorial
GitHub Login: @jeevansd
Microsoft Alias: jeedes

@sandeepshivhare This error means that you have wrongly configured the application in Azure AD. Please make sure that the Identifier value for the app matches to the SAML Issuer value which you are getting in the SAML Request. After that the error message should go away.
Please let us know how this goes after making this change.
Thanks.

@sandeepshivhare Is this issue resolved per my suggestion above? Can you please confirm?

@chetansriv FYI

Issue is not resolved yet.
On Tue 2 Apr, 2019, 3:47 AM Jeevan Desarda<EMAIL_ADDRESS>wrote:

@sandeepshivhare https://github.com/sandeepshivhare Is this issue
resolved per my suggestion above? Can you please confirm?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/28356#issuecomment-478767798,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AuzNCeZr5j8e2XYXSaTMLtdr7ZknFgENks5vcoWQgaJpZM4cSzug
.

@sandeepshivhare could you please confirm if this issue still persists?

Yes issue got resolved.
On Tue, 27 Aug, 2019, 4:23 AM Jeevan Desarda<EMAIL_ADDRESS>wrote:

@chetansriv https://github.com/chetansriv and @v-nagta
https://github.com/v-nagta for visibility.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/28356?email_source=notifications&email_token=ALWM2CMO6KYJXUNOICXZDF3QGRNE5A5CNFSM4HCLHOQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5F5MRI#issuecomment-525063749,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALWM2CMAYRBXLBMNOPEEX6LQGRNE5ANCNFSM4HCLHOQA
.


Refactor end-of-life logic into module format

Scenarios still being developed, will be picked up in future issues


Replace … with …

I think it is clearer if we don't use the HTML entity.

Codecov Report

Merging #2973 (af50f23) into develop (bd0b3c0) will increase coverage by 0.05%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #2973      +/-   ##
===========================================
+ Coverage    22.88%   22.93%   +0.05%
===========================================
  Files           51       51
  Lines        10895    10895
===========================================
+ Hits          2493     2499       +6
+ Misses        8402     8396       -6

Impacted Files
Coverage Δ

modules/default/calendar/calendar.js
33.29% <100.00%> (ø)

modules/default/updatenotification/node_helper.js
74.28% <0.00%> (+8.57%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Maybe we can remove the safe markers for nunjucks now? see https://github.com/MichMich/MagicMirror/pull/2427

Maybe we can remove the safe markers for nunjucks now? see https://github.com/MichMich/MagicMirror/pull/2427

so should this be done with this PR oder in another one?

I'd prefere a second PR if one of you could do it. Dont think it should go into the next release since it might need some feedback if it breaks something.


Implemented screenshot API

enhancement
PR checklist:

[x] Addresses an existing issue: fixes #31
[x] Includes tests: graphic test
[x] Documentation update: js docs for new method

Overview of change:
Added a method takeScreenshot on IChartApi to get image of the whole chart as a single image

I can't sign in to github
On Sun, Jun 23, 2019, 4:23 AM Evgeniy Timokhov<EMAIL_ADDRESS>wrote:

@timocov commented on this pull request.
In src/gui/chart-widget.ts
https://github.com/tradingview/lightweight-charts/pull/48#discussion_r296470405
:

	targetHeight += paneSeparator.getHeight();

}

if (this._options.timeScale.visible) {

	targetHeight += this._timeAxisWidget.height();

}

const targetCanvas = document.createElement('canvas');

resizeCanvas(targetCanvas, new Size(targetWidth, targetHeight));

const ctx = ensureNotNull(getContext2d(targetCanvas));

let targetX = 0;

let targetY = 0;

const drawPriceAxises = () => {

	for (const paneWidget of this._paneWidgets) {

		const image = ensureNotNull(paneWidget.priceAxisWidget()).getImage();

		ctx.drawImage(image, targetX, targetY);

		targetY += paneWidget.getSize().h;

Where? It looks like there is no adding SEPARATOR_HEIGHT to the targetY
variable, or I miss something?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tradingview/lightweight-charts/pull/48?email_source=notifications&email_token=AMMZ2VOJHAIBKVK6B73HMLTP34XGTA5CNFSM4HW4P3F2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB4LFGUI#discussion_r296470405,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMMZ2VLYYPS4GO2J5EBUFLTP34XGTANCNFSM4HW4P3FQ
.

Hello how can we use this method on the chart?

See https://github.com/tradingview/lightweight-charts/blob/dd7fb362a646bc8021b050b32f2dca233baf5e12/src/api/ichart-api.ts#L150-L154 for API reference or https://github.com/tradingview/lightweight-charts/blob/ad8326c28b2d38c0de445eab172a6a63688ea7ff/tests/e2e/graphics/test-cases/take-screenshot.js#L43-L48 for usage in our test cases.
We need to add docs for this method...


Update executable builds to Python 3.12

Python 3.12 is now stable, so should update the build exe version and verify all is well.

Fixed by c0866978d6c4e18dc39e316b8e90f9e3698c5402, dff56edc38f8c4f8527f0a6f90fe1932dcb811f2


[11.0.x] ISPN-12502 GC logging provided by default with GC_LOG is valid for Java 8 only

https://issues.redhat.com/browse/ISPN-12502
https://issues.redhat.com/browse/JDG-4177

Merged


A more accessible download link

Dear Ziang, first of all thanks for releasing the code for this promising pproach and making it publicly available. Is it possible to upload the pretrained model file to a more accessible platform, such as google drive or dropbox? That would be great, as it is not that straigthforward to download files from Baidu outside China, as far as I know. At least, my attempts were not successful so far. Thanks in advance!

Thanks for your attention. We have added the link to google drive


Export to where?

Hi,
The export bottom is not working for me (Chrome, win8). The copy to clipboard doesn't work very well, when I paste the data in excel the result is not the same of tabula's preview. If I click the export, doesn't matter the format, it loads and nothing else happens. Is it exporting to some place that I don't know?
thanks

Are you using Tabula v1.0.0 or v1.0.1? You should try the newest one, v1.0.1.
The new version, v1.0.1, (released last week) fixes the problem you're having where the download button is broken in Chrome.
(Or, you can try selecting the TSV option in the dropdown, then click Copy to Clipboard, which may work better for copy-pasting into Excel.)


Add Tokyo campus

Describe the pull request
Added the Tokyo campus information following the tutorial on https://github.com/42atomys/stud42/wiki/Contributions
Checklist
I didn't manage to open the application in my Browser to see the changes (localhost:3000 keeps loading without displaying anything)
Also, I'm sorry, I don't know how to run the linter...
link to the issue #347

Hello @masakiva,
Thanks for your first contribution 🎉
your follow is great, for your problem on run the project, the installation is explain step by step on : https://github.com/42Atomys/stud42/blob/main/CONTRIBUTING.md ?
FYI: Formating the interface can be run under vscode tasks : >Run Tasks >Format interface or with yarn make-pretty. But actually your work pass the linter and test so no formatting is needed, congratulations 🎉

The last step I need to handle before merge is to valid with a staff member of 42 Tokyo (this is for data protection and data process law in France).

Thank you for your kind reply, and for the explanations.
I asked the staff, I will come back when I will get a response!

The staff is OK for merging, I gave them your email adress (contact@atomys.fr), you should receive an email soon!

@42atomys Thank you for the small fix, and for the merge! I look forward to using the new version with my campusmates soon!!


Fix HeroTabBarAnimation would black view crash if switch tabs quickly

Fix HeroTabBarAnimation would black view crash if switch tabs quickly
https://github.com/lkzhao/Hero/issues/166

Thanks for contributing!


Rename data sources to data connections

Description
Renames all references from datasources to data connections to disambiguate from the other datasources in OSD
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Are any integration tests checking for the old strings?

Fixed up a few things here: ac3f9e4, thanks! Maybe we should remove the tests since they are causing more harm than good with so many things changing at the moment.


onBrokenAnchors reports named anchor tags as broken links

Have you read the Contributing Guidelines on issues?

[X] I have read the Contributing Guidelines on issues.

Prerequisites

[X] I'm using the latest version of Docusaurus.
[X] I have tried the npm run clear or yarn clear command.
[ ] I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
[ ] I have tried creating a repro with https://new.docusaurus.io.
[ ] I have read the console error message carefully (if applicable).

Description
When you use named anchors (E.g., Some Text or self-closing  ) in  a doc page, onBrokenAnchors reports it as a broken link.
Reproducible demo
No response
Steps to reproduce

In docusaurus.config.js, set onBrokenAnchors to Warn (if not using the default setting).
Edit a doc page.
Add a named anchor anywhere on the page. E.g., Some Text
Add a link to the named anchor, e.g., [Link Text](#anchor)
Save the page.
Build the site (yarn build).

Expected behavior
No console feedback. onBrokenAnchors ignores the arbitrary named anchor because it's only supposed to work for anchors declared with the Heading component.
Actual behavior
onBrokenAnchors reports the named anchor as a broken anchor. Example:
[WARNING] Docusaurus found broken anchors!

Please check the pages of your site in the list below, and make sure you don't reference any anchor that does not exist.
Note: it's possible to ignore broken anchors with the 'onBrokenAnchors' Docusaurus configuration, and let the build pass.

Exhaustive list of all broken anchors found:
- Broken anchor on source page path = /user-types-and-permissions:
   -> linking to #anchor (resolved as: /user-types-and-permissions#anchor)

Note that on the rendered page, the anchor works.
Your environment

Public source code:
Public site URL: (issue encountered locally during build)
Docusaurus version used: 3.1.1
Environment name and version (e.g. Chrome 89, Node.js 16.4): Version 120.0.6099.129 (Official Build) (arm64)
Operating system and version (e.g. Ubuntu 20.04.2 LTS): macOS Ventura 13.4.1 (22F82)

Self-service

[ ] I'd be willing to fix this bug myself.

We'll try to improve this, but this is currently the intended behavior, because all anchors must be "collected" for the broken link checker to see them.
CF-related issue: https://github.com/facebook/docusaurus/issues/9763#issuecomment-1905657358
The current suggested workaround is:
import Link from '@docusaurus/Link';

<Link id="my-anchor"/>

Note that  is not valid anymore as per HTML5 spec, that's why I choose to not support it: https://stackoverflow.com/questions/484719/should-i-make-html-anchors-with-name-or-id

I'm keeping this open as a request request
But please share more details about your use-case. For now we don't even know if you create that JSX inside a React file or an MDX file, and that matters to me to design an improved solution, for which the technical solution could be radically different.

Yes exactly, my immediate use is to be able to have arbitrary (non-heading) anchors in MDX files. I haven't needed to use them in React pages.
I would love for onBrokenAnchors to be able to validate arbitrary anchors, along with heading anchors, in MDX pages. That's the case with the workaround you provided (Thanks!).
If onBrokenAnchors can't validate arbitrary anchors, I'd expect it to ignore them instead of mistakenly reporting them as broken.

I would love for onBrokenAnchors to be able to validate arbitrary anchors, along with heading anchors, in MDX pages. That's the case with the workaround you provided (Thanks!).

We'll try to add support for
,  etc in Markdown files.
However it's not possible to add support for <MyCustomComponent id="anchor"/> and you'll have to keep using the Docusaurus API in that custom component.

If onBrokenAnchors can't validate arbitrary anchors, I'd expect it to ignore them instead of mistakenly reporting them as broken.

We validate each link anchor against all the anchors we successfully extracted from the target page, so it's important to collect all of them.
For a given page, we either validate or not, but there's no middle ground. What you ask for basically means disabling the broken anchor check

We'll try to add support for ,  etc in Markdown files.

That would be great!

For a given page, we either validate or not, but there's no middle ground. What you ask for basically means disabling the broken anchor check

I understand. That's not what I want. I would much rather the arbitrary anchors get checked along with the heading anchors. It's just that right now,  anchors return false errors because they're not supported, which makes the broken anchors check noisy.
I really appreciate your taking the time to consider this, BTW.

Is there some way to suppress this warning, this is currently blocking us from upgrading

@nijikokun a warning does not prevent your site from building, and yes this is configurable as explained in the docs and release notes:
https://docusaurus.io/blog/releases/3.1#broken-anchors-checker
onBrokenAnchors: "ignore"

I am having the same issue but I am using MD and MDX files.  I guess I just don't understand why this would be an issue when using markdown files because they get transpiled by docusaurus.  Wouldn't docusaurus properly convert a markdown link to the required HTML so it "is not a broken anchor"?
For example, I have the markdown link [Width](./Velaptor.SizeU.md#width) with other variations that I have tried, but I always get the warning when building.
Here are the variations I have tried with no good results:

Width
Width
Width
Width
Width
Width
Width

Of course, this could be maybe understanding what a "broken anchor" means.  Even though I get the warning, the anchor works locally with pnpm start, pnpm run serve, and in production.
So I guess I do need an explanation of what "broken anchor" means?

I am having the same issue but I am using MD and MDX files. I guess I just don't understand why this would be an issue when using markdown files because they get transpiled by docusaurus. Wouldn't docusaurus properly convert a markdown link to the required HTML so it "is not a broken anchor"?
For example, I have the markdown link [Width](./Velaptor.SizeU.md#width) with other variations that I have tried, but I always get the warning when building.
Here are the variations I have tried with no good results:

Width
Width
Width
Width
Width
Width
Width

Of course, this could be maybe understanding what a "broken anchor" means. Even though I get the warning, the anchor works locally with pnpm start, pnpm run serve, and in production.
So I guess I do need an explanation of what "broken anchor" means?

It's quite difficult to see what problem your are facing with the current information, perhaps we could help you more if you try to recreate your error with https://docusaurus.io/docs/playground
The feature basically collects each link in a page and then try to resolve the link with or without anchor and if nothing is found its considered as a broken link / anchor
I don't know how is your Velaptor.SizeU.md page but #widthshould refer to a markdown heading
// hello.md
# heading

a link to my [title](#heading)

// world.md
# heading

a link to my [hello heading](./hello#heading) // also should work depending on the path
a link to my [hello heading](hello#heading) // should work

I am having the same issue but I am using MD and MDX files. I guess I just don't understand why this would be an issue when using markdown files because they get transpiled by docusaurus. Wouldn't docusaurus properly convert a markdown link to the required HTML so it "is not a broken anchor"?
For example, I have the markdown link [Width](./Velaptor.SizeU.md#width) with other variations that I have tried, but I always get the warning when building.
Here are the variations I have tried with no good results:

Width
Width
Width
Width
Width
Width
Width

Of course, this could be maybe understanding what a "broken anchor" means. Even though I get the warning, the anchor works locally with pnpm start, pnpm run serve, and in production.
So I guess I do need an explanation of what "broken anchor" means?

It's quite difficult to see what problem your are facing with the current information, perhaps we could help you more if you try to recreate your error with https://docusaurus.io/docs/playground
The feature basically collects each link in a page and then try to resolve the link with or without anchor and if nothing is found its considered as a broken link / anchor
I don't know how is your Velaptor.SizeU.md page but #widthshould refer to a markdown heading
// hello.md
# heading

a link to my [title](#heading)

// world.md
# heading

a link to my [hello heading](./hello#heading) // also should work depending on the path
a link to my [hello heading](hello#heading) // should work

Sorry about that.  I forgot to escape the markdown examples. I fixed my comment.  I will also try and recreate the error in the playground.

@OzakIOne
I used the codebox.io playground to set up the broken link example.
codebox fork
This is the warning that I got in the terminal when running npm run build
[WARNING] Docusaurus found broken anchors!

Please check the pages of your site in the list below, and make sure you don't reference any anchor that does not exist.
Note: it's possible to ignore broken anchors with the 'onBrokenAnchors' Docusaurus configuration, and let the build pass.

Exhaustive list of all broken anchors found:
- Broken anchor on source page path = /docs/intro:
   -> linking to /docs/tutorial-basics/create-a-page#MyHeading

@CalvinWilkinson what you report here is not related to the original issue. I'm going to fold all these comments as "off topic", but feel free to open a separate issue/discussion if you want to. I'd encourage you to join our Discord server in case you need community support on this.

Your link is [My Heading](./tutorial-basics/create-a-page.md#MyHeading)
And the target page has # MyHeading
First, we don't create anchors for docs titles / top-level headings. You can just link to the page directly because that title should rather be at the top.
Basically, you can see it does not have a # next to it when you hover it. If you used ## MyHeading instead you would get an anchor.

If you want to change this behavior and have top-level headings create anchors, you can swizzle our @theme/Heading component.
Second, the anchors we generate are "sluggified" using the same logic GitHub uses. And anchors are case-sensitive. If you use ## My Heading then the anchor is #my-heading, not #My Heading

Please note that  is not valid anymore as per HTML5 spec, that's why I choose to not support it in our Link component.

I came across this issue because we include a bunch of generated content in our docs which includes a few anchors like this:
-  Item 1

The tool that generates the content has a PR open to align to HTML5 spec, but in the meantime I made a quick little rehype plugin that will swap all name attributes to id instead. This is definitely a preference but it helped remove all the errors and still works as intended.
Here is the code if anyone also has this issue:
rehypeNameToId.ts:
import { Plugin } from 'unified';
import { hasProperty } from 'hast-util-has-property';
import { Nodes } from 'hast';
import { visit } from 'unist-util-visit';

const rehypeNameToId: Plugin<never> = () => {
  return (tree) => {
    visit(tree, 'element', (node: Nodes) => {
      if (hasProperty(node, 'name')) {
        node.properties.id = node.properties.name;
        node.properties.name = '';
      }
    });
  };
};

export default rehypeNameToId;

docusaurus.config.ts:
import rehypeNameToId from './rehypeNameToId.ts'

export default {
  // repeat in preset, docs-plugins, blog-plugins, etc.
  presets: [
    [
      'classic',
      {
        blog: {
          rehypePlugins: [rehypeNameToId],
        }
      },
    ]
  ]
}

First, we don't create anchors for docs titles / top-level headings. You can just link to the page directly because that title should rather be at the top.

So, all h1s are not anchorable? That seems counterintuitive.
I understand not allowing anchors for the title and for the first h1, but all h1s? h1s aren't for titling documents, they are for formatting and navigation.
Markdown's headings are based off of headings in word documents. All top-level h1 headings in Microsoft Word are "anchored" and can be navigated to with links in the word document.
I do agree that anchoring the first h1 is pointless, but it is also harmless.

@anonhostpi

So, all h1s are not anchorable? That seems counterintuitive.
I understand not allowing anchors for the title and possibly the first h1, but all h1s?

It's usually a best practice to only have one h1 per page

h1s aren't for titling documents, they are for formatting and navigation.

Oh really? Can you back this up with reference material from an SEO and accessibility authority? For example from Google?

More specifically, markdown's headings are based off of headings in word documents. All top-level h1 headings in Microsoft Word are "anchored" and can be navigated to with hyperlinks in the word document. There's no reason that h1s should be treated different here.

We are not a text editor, and don't particularly plan to inspire from them.

I do agree that anchoring the first h1 is pointless, _but it is also harmless.

If it prevents you from introducing an antipattern to your own site, we can even consider this restriction as a feature.

The good news is that Docusaurus is flexible enough to let you plug your own opinions if you don't agree with our defaults.

swizzle @theme/Heading to alter all our theme headings
swizzle @theme/MDXComponents/Heading to alter all # Markdown Title of your site, or provide a custom h1 in the MDXComponents map.
use JSX
 or <MyCustomH1Heading/> and implement whatever you want

And remember h1 are not anchorable by default on the web.

OOC, are other headings anchorable by default? Don't they all require ids?
Also, "should multiple h1s be allowed" has been a long flame war on MDN, but the opinion of the writers is also that there's no legitimate use case of multiple h1s.

We are not a text editor, and don't particularly plan to inspire from them.
And remember h1 are not anchorable by default on the web. If you don't like that, then maybe propose to browser vendors to make all headings anchorable by default?

Even if you are not a text editor, blocking anchoring doesn't follow the outline behavior specified by the HTML specification:

https://html.spec.whatwg.org/multipage/sections.html#headings-and-outlines:~:text=These elements represent headings.

This is why I said headings are for navigation.

If it prevents you from introducing an antipattern to your own site, we can even consider this restriction as a feature.

I would argue that allowing multiple h1s while simultaneously not allowing anchors on secondary h1s is an antipattern itself.

The HTML specification can be poor at defining semantics and often relies on the English language for that. So, to further drive my point that headings are for formatting, here is 2 definitions from the Cambridge dictionary stating that headings are for titling sections of text (not documents)
https://dictionary.cambridge.org/us/dictionary/english/heading

This is for general typesetting. It has nothing to do with web page structure. You can find numerous resources:

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/Heading_Elements#avoid_using_multiple_h1_elements_on_one_page
https://webaim.org/techniques/semanticstructure/#headings
https://www.a11yproject.com/posts/how-to-accessible-heading-structure/#one-h1

 should be functionally equivalent to the page's <title>, only displayed to the user. Everything else comes as
.

Anyway none of this really pertain to Docusaurus. This issue is about  which is explicitly invalid HTML. We are only keeping it open to improve the DX by recommending alternatives.
s don't get IDs, and we are unlikely to differentiate between "first h1" and "later h1s" because that shouldn't matter.

It does matter. h1-h6s are not only for formatting, but also for navigation. Not allowing anchoring goes outside the intended outlining behavior defined by the HTML spec. If a user creates a link to a secondary h1, that link should work. Otherwise, it's non-compliant:

The outline should be used for generating document outlines, for example when generating tables of contents. When creating an interactive table of contents, entries should jump the user to the relevant heading. - 4.3.11 Headings and outlines, HTML Specification

This also applies to links within the document.

I'd suggest opening a different issue/discussion for the case of h1 headings in particular.
Let's keep this issue clean related to onBrokenAnchors reporting #anchor as a broken link for .
I'll hide all the h1-related discussions above.


Revert "entg: add Resolver annotation"

Reverts facebookincubator/ent-contrib#22
Going to revert it until I’ll fix the multi-annotations as it requires from us to add this new annotation to all edges explicitly.
After this PR, it will generate resolvers to all edges by default, and those resolvers can be overridden on the actual resolver if necessary.

@giautm, I'm OK with having a NoResolver annonation which will allow us to explicitly disable resolver generation.


Spacing

+1

Coverage remained the same at 78.819% when pulling 350dcbc61d354493efbc503eb6da9f5b6cb56810 on rmcintosh:patch-4 into 111d81ab41e6247b5ac7edd13612e26d2caee374 on linode:develop.

Coverage remained the same at 78.819% when pulling 350dcbc61d354493efbc503eb6da9f5b6cb56810 on rmcintosh:patch-4 into 111d81ab41e6247b5ac7edd13612e26d2caee374 on linode:develop.

Coverage remained the same at 78.819% when pulling 350dcbc61d354493efbc503eb6da9f5b6cb56810 on rmcintosh:patch-4 into 111d81ab41e6247b5ac7edd13612e26d2caee374 on linode:develop.


Tab size defaults to 1 when no indent_size is set and editor.tabSize is set to auto

I'm not sure if this is intended behaviour with vscode-editorconfig, but as seen on https://github.com/Microsoft/vscode/commit/5da0947edd3598894064ef4738b5941c32c99698#commitcomment-14856269, in some cases, if no indent_size value is set and editor.tabSize is set to "auto" in the VSCode configs, the tab size will default to 1 instead of the default VSCode settings which is 4.

@resir014 use the forked plugin https://github.com/editorconfig/editorconfig-vscode. This is fixed in v0.2.2.


Fixed some Chinese characters anaylize error.

Check Contour edge.

Thank you for this interesting correction. Please tell me on what font and symbol the effect of this correction can be seen.


Fibonacci example fix

Current version of Fibonacci example computes not 1000th, but 1002nd Fibonacci number.
This PR changes this example to computation of 1001st Fibonacci number by putting only [1] to the input. This approach allows to remain 1000 as cycles number, changing only stack and a comment.

Looks like main README was the only place left to fix. Fibonacci also mentioned in the Benchmarking CLI but without specifying what number we are counting, so there is no need to edit anything there.


input for DREME

Hello,
Is it possible to adapt your tool for other staph species? If so, what would be the appropriate input to DREME?
Best wishes,

Hello,
Yes that's on my list for future things to do! The input for DREME would be agr operons of known agr groups.
For example:
dreme -k 31 -e 0.0001 -m 15 -dna -oc ./dreme_gp1 -p gp1_operon.fna -n gp1_negative_operon.fna
This would output fifteen 31mers that are enriched in only the group 1 operons ( gp1_operon.fna ) compared to "NOT" group 1 operons ( gp1_negative_operon.fna ) - which is a collection of every other agr operon EXCEPT group 1. This would need to be repeated for every agr group for that given species. And this entire process would need to be repeated for each species.
However DREME has been deprecated since the development of AgrVATE so I will be looking at alternatives.  Please feel free to let me know if you have any suggestions!
Vishnu

Thank you so much for your quick reply. And is there any way to recover the operon sequence for other staph species from their genome sequence? In other words, how did you recover the operon sequence for the different groups. And again very interesting paper on biorxiv/tool!


Automatically write formatted policy back into file

Category
CLI features/changes
Describe the feature you'd like to request
It would be awesome if the cedar format command wrote the formatted policy back into the file if the policy was pulled from a file (when policies_file != None)
To prevent accidental overwrites, or bugs in the formatter (such as #787), I suggest we put it under a -w/--write flag.
Describe alternatives you've considered
Manually copying the stdout into the file, but it's a little tedious, and it seems like a reasonable thing to support.
Additional context
Currently I copy from stdout and write back into the file manually.
Is this something that you'd be interested in working on?

[X] 👋 I may be able to implement this feature request
[ ] ⚠ This feature might incur a breaking change

Sounds reasonable to me

Cool, I'll take a stab at implementing it :)


Fix a CI failure on h5py 3

Closes #387

This fixes the failing build, it seems. Now we're just left with the various pickle errors in jit-free mode, which I need to fix in another PR at some point


add kafka spout for heron

Updated the storm kafka spout code to make it work with Heron. Updates are:

remove Kafka bolt and trident related code.

change Scheme related interface to accept the byte[] instead of ByteBuffer

change the pom.xml file for compiling

Contrib code is put in Heron repo temporarily,  it should be moved to a separate heron-contrib repo finally. Setting up such a repo takes time so currently the code is put here to unblock others.

@nlu90 - can we write a build file so that we can build using bazel under the config, if needed.

@nlu90 - the reason why I suggested BUILD file, we can occasionally use it for release purposes and also ensure that the code build manually.

I was wondering do they have unit tests as well

should we change src/jvm to src/java directory to be consistent with others?

@kramasamy

For release purpose, you can use the pom.xml file and maven to do a binary release. And also since the code will finally be moved into a separate repo, we should keep it decoupled with other part of the heron repo. Adding a build file also requires changes in the WORKSPACE file to include the extra binary dependencies, changes in other build scripts to integrate with the release scripts. Those changes need to be reverted once the code is moved. I think it's not a good practice.

This PR also includes the unit tests

I'm fine with either. I can change it to src/java

@nlu90 - can you make changes as @mycFelix mentioned?

@mycFelix, thanks for mentioning it, I've updated it.
@kramasamy the BUILD file for bazel is also added.

Ship it!


revert options.formatOut back to 'input' ?

What are you trying to achieve?
I want to switch back to options.formatOut == 'input' after working with raw data (call to raw(), which sets options.formatOut to 'raw') without triggering the Error: Unsupported output format but also using a neat method as part of the object, rather than modifying the value of options.formatOut to 'input'
Have you searched for similar feature requests?
Yes, and I see some related issues, but I don't see any that mention
What would you expect the API to look like?
/**
 * Use the default options for output image.
 *
 * @example
 * // Output initial data from JPEG input
 * const { data, info } = await sharp('input.jpg')
 *   .input()
 *   .toFile('./output.jpg');
 *
 * @returns {Sharp}
 */
function input () {
  return this._updateFormatOut('input');
}

What alternatives have you considered?
sharp_image.options.formatOut = 'input';
Is there a sample image that helps explain?
No.

Closing due to inactivity but please feel free to reopen with more details if further help is required.


release 1.0

composer req kevinpapst/tabler-bundle

still pulls 0.22, can you tag 1.0 as a release?

one thing after another... i want to wait a bit and give people the chance to test master first

Closed as 1.0 is released


creating input.yml

Hi,
I've just trying to figure out what kind of files your package can generate. I understand that I can not create a cwl file with a Workflow class, but can I create an input.yml file for a cwl with a CommandLineTool class?

Hi,
The package supports the generation of CommandLineTool and also Workflow thanks to @kellrott.
Unfortunately, the creation of input.yml file is not possible yet but could be considered for future development.

FYI: cwltool can create a blank input job document; this code may be of interest: https://github.com/common-workflow-language/cwltool/blob/840bb3b3ac889c9de33638ae8af707b5fd3fb326/cwltool/main.py#L61

Hey @djarecka, it appears your question was answered:

For existing CWL files, this project won't help you generate an inputs file - but cwltool can.
For workflows you build with these classes, you could add something like this to the Workflow class:

def get_inputs_dict(self):
    return {i.id: None for i in workflow.inputs}

I'm going to close this issue in the meantime, but feel free to open it if you have more questions.


White list a whole domain or domain regex

Hi,
I have a use case where sub domains are created dynamically so I'm not able to whitelist them beforehand.  Would it be possible to whitelist an entire domain or provide a regex to whitelist?
e.g something like
whiteListedDomains: ['.example.com']
or
whiteListDomainRegex: '(\w+\.)?example\.com$'

That's definitely something we can look at doing. I think the easiest approach would be to whitelist a domain so that any subdomain applies.

I would need this feature too. On my projects, for each PR made by the team, a review app is deployed on Heroku. It's domain looks like my-app-pr-123.herokuapp.com. Even if the risk is pretty small, I'd like to not white list herokuapp.com but rather myapp(-pr-\d+)?\.herokuapp\.com$

This may be helpful but may not be doable for many cases.  I was originally looking to use a regex match for something similar to *.herokuapp.com, but decided on a different strategy.
During bootstrap, I am making an http request to my backend to get some configuration variables.  I'll set that as a const and then inject it into my jwt config.  Inside that const is the API URL that I will be using for this client.  I happen to know this URL beforehand, some might not.
function jwtInterceptorConfig ($httpProvider, jwtOptionsProvider, APP) {
    // Set whiteListedDomains based on APP.apiURL
    var whiteListedDomains = []
    var parser = document.createElement('a')
    parser.href = APP.apiURL
    if (parser.hostname) {
      whiteListedDomains.push(parser.hostname)
    }
    jwtOptionsProvider.config({
      whiteListedDomains: whiteListedDomains,
      ...

In one of my use cases, having a function providing the "white listed" hosts to angular-jwt would be the only viable option as that "list" is dynamic.

Implemented in 0.1.9

@dusty Mind sharing how you're doing an http call in your bootstrap?  I keep running into the problem of making sure the http call happens before the jwtOptionsProvider is configured.

Sure.  This is how I bootstrap angular.
function bootstrap() {
    var $injector = angular.injector(['ng'])
    var $http = $injector.get('$http')
    $http
      .get('/app/info')
      .then(function(response) {
        var app = JSON.parse(response.data)
        angular
          .module(moduleName)
          .constant('APP', app)
          .config(locationConfig)
          .config(routeConfig)
          .config(httpSyncConfig)
          .config(deleteHttpConfig)
          .config(xmlHttpInterceptorConfig)
          .config(authInterceptorConfig)
          .config(jwtInterceptorConfig)
        angular.element(document.getElementById(moduleId)).ready(function() {
          angular.bootstrap(document, [moduleName])
        })
      })
      .catch(function(error) {
        console.log('error', error)
      })
  }

  bootstrap()


mobile support

Please check for duplicates or similar issues before creating this issue.
nope none
What is your suggestion, and why should it be implemented?
fr MOOBILE support becuz android players :p, just asking. If you add this or well  do another repository, or well.. in the next release make an apk, it does.

alright

fr closed


[Release] Bumped postgres version to 2.2.3

Motivation
Fix bug

Codecov Report

Merging #2402 into master will increase coverage by 0.34%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #2402      +/-   ##
==========================================
+ Coverage   78.56%   78.91%   +0.34%
==========================================
  Files         205        5     -200
  Lines       13796      498   -13298
  Branches     1460       88    -1372
==========================================
- Hits        10839      393   -10446
+ Misses       2568       73    -2495
+ Partials      389       32     -357


.map() doesn't garbage collect correctly when used in concurrency mode.

What version of bluebird is the issue happening on?
3.4.6

What platform and version? (For example Node.js 0.12 or Google Chrome 32)
v6.8.0

Did this issue happen with earlier version of bluebird?
don't know.

Issue:
getting heap overflow error when trying to call a large number of sequelize calls.
It appears that .map isn't correctly garbage-collecting when being run in concurrency mode.
This code snippet fails:
let trims = [...]; // length is about 50,000
return models.sequelize.transaction((t) => {
  return Promise.map(trims, (trim) => {
    // sequelize
    return models.Trim.upsert(trim, {transaction: t});
  }, {concurrency: 100});
});

This code snippet is successful:
let trims = [...]; // length is about 50,000
let batches = [];

while(trims.length) {
  batches.push(trims.splice(0, 2500));
}

return models.sequelize.transaction((t) => {
  return Promise.each(batches, (batch) => {
    return Promise.map(batch, (trim) => {
      return models.Trim.upsert(trim, {transaction: t}));
    });
  });
});

What is it about .map that prevents the first snippet from correctly garbage collecting?
Is there a way to fix that code snippet so that it will work correctly without overflowing the heap?

The same problem, I can not use the map function with more than 60 http requests
let promises = [];

for (let index = 0; index < 100; index++) {
	promises.push(request('https://...'));
}

let [ response ] = await Bluebird.map(promises, promise => promise , { concurrency: 1 });

I know this is old, but if anyone runs into this, the pattern @monolithed shows generates too much queued work.
This is because all request instances 'start' immediately.
Try creating an array of URLs, wrap in Promise.resolve([urls]). Then chain, like so:
let urls = ['https://...']
let results = Promise
  .resolve(urls)
  .map(url => request(url), { concurrency: 1 });
results.then(r => console.log('Request Responses: ', r))

This will scale to almost as much RAM as your urls array can use.

@charlesdeuter I don't think this is a bug at all.
Sequelize's .upsert() method returns a Sequelize model instance which is a pretty heavy object which takes a fair bit of memory.
Bluebird's Promise.map() behaves like Javascript's Array.prototype.map() - it runs the handler function on each item in the input array, and stores all the results returned from the handler in a new array.
So, regardless of what concurrency setting you use, you're still slowly building up an array of 50,000 of these heavy Sequelize objects. This leads to escalating memory usage.
But it's not a memory leak in Bluebird - you're just using Bluebird to do something which uses up a huge amount of memory!
Your second example does not cause excess memory usage as Promise.each() does not return the results from the handler - so the results of each batch are discarded before the next batch commences.
If you don't want to keep the results, just don't return them from the handler.
e.g.:
let trims = [...]; // length is about 50,000
return models.sequelize.transaction((t) => {
  return Promise.map(trims, (trim) => {
    // sequelize
    return models.Trim.upsert(trim, {transaction: t})
      .then( () => null ); // discards result of `.upsert()` and returns null instead
  }, {concurrency: 100});
});

It would be useful if Promise.each() had a concurrency option too, but that's not the case at present.
@petkaantonov I think this issue can be closed.


Always restart kubelet

We observed multiple instances of the kubelet just exiting without stating any reason, exit code 0.
It makes sense to restart it on all occasions, not only when it failed.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks for this PR. Do you have any other information about the state of the kubelet exiting? What version of the AMI are you using? Any custom kubelet configuration?

What we observed was just that the kubelet was stopped (inactive in systemctl status kubelet, exit code 0), and journalctl -u kubelet ended with normal log messages, nothing indicating a crash or any other reason. At the same time, we had another pod on the instance go OOM, so that might or might not be related. Anyways, is there a valid reason for not restarting the kubelet, independent of exit code?

@devkid Could you provide the output of systemctl status kubelet during a time when kubelet is stopped? We fixed an issue in #137 that makes systemd restarts kubelet in case the process is terminated with signal=PIPE and I want to check if this also fixes your use case.

I will record the exact output if this happens again. I assume you might be right with SIGPIPE. Is it possible that journald was OOM killed in this low-memory situation?

Closing with no response. Feel free to reopen if you find anything


3D images

Is it good for 3D convolutional Neural Network? I mean the input shape is like this (nb_ex, nx, ny, nz, nb_channel). Many thanks.

I also want a 3D convolutional Neural Network version . Many thanks.

I am the same to you,do you have it ?thanks!
@pantheon5100 @xuzhang5788 @jacobgil


Document additional behavior change in 5.0 when a reference navigation property is set eagerly by the entity instance

When setting a related entity to a default value via the constructor. I.e.:
public Blog() {
	this.Author = new Author();
}

the property does not get replaced correctly with the entity that was retrieved from the database. Thus all changes to it are lost on SaveChanges.
This worked correctly in EF core 3.1.
Steps to reproduce
Run this project
EFCoreCtorBug.zip
Output: "Author Name:"
Switch to EF core 3.1
Output: "Author Name: NonMVPAuthor"
Notice how the Author property is not replaced with the object retrieved from the database on line 29 but instead is the default object set in the ctor.
Further technical details
EF Core version: 5.0 RC1
Database provider: seems to be universal to all providers.
Target framework: .NET Core 3.1

@ajcvickers if this is going to be documented as a breaking change from EFCore 3.1 to EFCore 5.0 what is the recommended way to set a default value for a relation in EFCore 5.0? The current EFCore 3.1 behavior is not only more usefull but also consist with the behavior from other popular ORMs such as Hibernate or Doctrine.

@daniel3303 Yes, this will be documented. EF Core doesn't support creating a default instance of an entity types since this implies a default key value. The discussion in https://github.com/dotnet/efcore/issues/18007 covers this.

Also, there may be a feasible to determine in a performant way that an existing object instance was used instead of the one returned from the query. We should investigate this a bit more for a future release.

@ajcvickers Is there an open issue that covers this investigation?

@ChristophHornung We're not actively investigating this currently. I will create an issue.


Can the text-encoder / vision-encoder be optional rather than mandatory?

There are times that, we only need half of CLIP, such as when a Vision Model is connected to an LLM after conversion, or when a Text Model provides guidance for Diffusion.

Yes, this is also a common use case for indexing with the image encoder and searching with the text encoder. The new GGUF format makes working with multiple formats easy so this will be handled when upgrading to the GGUF format.

Released in #75


Also, it doesn't seem practical to have complete commit history for installation purposes.

Hi @danielshahaf
I do agree that many people would know about the  --depth option. However, it's not that uncommon to use it. For example, here is a similar change in golang's repository.
With time, the repository history size will only increase and it can make for a really frustrating 5 minutes in areas with patchy network.

I'm sorry, but the answer's still no.  This repository is not large (it's about half the size of a release tarball of zsh itself), and on balance I don't think people on 56kbps modems who don't already know about the --depth option are a large enough group to merit cluttering the documentation for everyone else.
% units -t '2.1 MiB / 5 minutes' 'bits per second'
58720.256

It's entirely standard for installation instructions to have to be customized for the local setup.
Also, people on narrow connections have the option of using z-sy-h release tarballs.  (Now that I mention it, I'm not even sure that git clone --depth is designed to minimise network traffic.)

The tragedy is that those people are on so called "4G" connections. 😅
I don't believe that the --depth option was specifically designed to minimize network traffic but the way it works uses less data.
I respect your decision to have less cluttered installation instructions. You may close this.

No other opinions, so closing. Thanks for discussion!


Updated package.json after moving the repo to xolvio github organization

:tada: This PR is included in version 2.1.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Player name string locks session if name is too long

If a player enters a long string, the game session locks forever

I can't reproduce this.


Could not able to connect to Google Tag Assistant in GTM and it shows no tags found

Added partytown to my gtm script i could not able to debug/tests the events are firing and tracking in GTM. After lots of research so i came to know this component is helping to debugg
After installing the @superside-oss/partytown-gtm and followed the steps as per documentation. Still i could not able to see the tags in Google tag Assistant. please help on this @slawekkolodziej
React code in header component
<---Head Tag --> <Partytown         forward={[           'dataLayer.push',           GTM_TAG_ASSISTANT_FORWARDER         ]}         mainWindowAccessors={[           GTM_TAG_ASSISTANT_ACCESSOR         ]}         resolveUrl={partytownResolveUrl}       />       <GTMScript gtmId={'GTM-MKPX3TV4' as string} />    <--- Body Tag --->    <Partytown debug={true} forward={['dataLayer.push','gtag']}>              </Partytown>

You have two <Partytown> components in your code snippet. This isn't right. Do you see any errors in network tab?
I highly doubt it's a bug in the library. Try checking out this repo and trying with the provided NextJS sample app.

yes, for head tag and body tag
so i have added partytown component to both script tags

I'm using partytown-gtm-plugin to debug the Google tag manager using partytown. Here is my implemented code. Still  i could not able to debug/tests the events are firing and tracking in GTM. @slawekkolodziej please help me on this.
"@superside-oss/partytown-gtm": "^1.0.0-alpha.12"
<EMAIL_ADDRESS>"^0.9.2"
<PageContext.Provider value={{ pathPrefixes, websiteLocale }}>       <Helmet         htmlAttributes={{           lang: websiteLocale?.languageCode,           lang_font:             websiteLocale.attributes.locale === 'he-IL'               ? 'he-IL'               : websiteLocale.attributes.locale === 'lo'               ? 'lo'               : 'default_language',           dir:${
get(websiteLocale, 'attributes.hasRightToLeftLanguage', websiteLocale.hasRightToLeftLanguage ?? false)
? 'rtl'
: 'ltr'
}`,
}}
>
{website.isTemplate ? (

) : null}
    <Partytown
      forward={['dataLayer.push', GTM_TAG_ASSISTANT_FORWARDER]}
      mainWindowAccessors={[GTM_TAG_ASSISTANT_ACCESSOR]}
      resolveUrl={partytownResolveUrl}
    />
    <GTMScript gtmId={'GTM-MKPX3TV4' as string} />

    <link rel="preconnect" href="https://mktmediadev.wallstreetenglish.com" />
    <link rel="dns-prefetch" href="https://mktmediadev.wallstreetenglish.com" />

    {/* <link rel="preconnect" href="https://fonts.gstatic.com" />
    <link rel="dns-prefetch" href="https://fonts.gstatic.com" /> */}
  </Helmet>

{children}

</PageContext.Provider>

);
};`

It's not how you're supposed to use Partytown. Go ahead and review a sample NextJS app here: https://github.com/superside-oss/partytown-gtm-plugin/tree/alpha/packages/integration/nextjs
Also you should set up server-side proxy for assets that misses CORS headers. GTM won't work without it.
I am closing this issue as it does not look like a bug in the library.

I does not receive any CORS error in network tab.
I will followup the docs which you shared. thank you for your support

do you have any example to setup server side proxy for Gatsby

Nope, I did not work with Gatsby for years.


Allow injection of final for use in switches

It's very common to need final switch. Allowing it to be injected in D2 builds would be really useful for stage 2 projects.

Holding this a bit until @mathias-lang-sociomantic attempts his D1 compiler tweak to allow and ignore final in such contexts as that would allow for a cleaner transition.

Oh right, allmost forgot about this:
https://github.com/dlang/dmd/pull/6833

Closed in favor of dmd1 solution


Add -A and -B to gopass pwgen

This commit adds two new flags to the internal pwgen implementation.
RELEASE_NOTES=[ENHANCEMENT] Add -A and -B to pwgen
Signed-off-by: Dominik Schulz<EMAIL_ADDRESS>

Codecov Report

Merging #1716 (55b9aae) into master (774eb1d) will increase coverage by 0.02%.
The diff coverage is 71.88%.

@@            Coverage Diff             @@
##           master    #1716      +/-   ##
==========================================
+ Coverage   58.54%   58.56%   +0.02%
==========================================
  Files         190      190
  Lines        9120     9142      +22
==========================================
+ Hits         5339     5354      +15
- Misses       3168     3175       +7
  Partials      613      613

Impacted Files
Coverage Δ

internal/action/pwgen/pwgen.go
0.00% <0.00%> (ø)

pkg/pwgen/memorable.go
87.50% <50.00%> (ø)

internal/action/pwgen/commands.go
100.00% <100.00%> (ø)

pkg/pwgen/cryptic.go
86.59% <100.00%> (ø)

pkg/pwgen/pwgen.go
95.24% <100.00%> (+1.49%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 774eb1d...55b9aae. Read the comment docs.

Codecov Report

Merging #1716 (55b9aae) into master (774eb1d) will increase coverage by 0.02%.
The diff coverage is 71.88%.

@@            Coverage Diff             @@
##           master    #1716      +/-   ##
==========================================
+ Coverage   58.54%   58.56%   +0.02%
==========================================
  Files         190      190
  Lines        9120     9142      +22
==========================================
+ Hits         5339     5354      +15
- Misses       3168     3175       +7
  Partials      613      613

Impacted Files
Coverage Δ

internal/action/pwgen/pwgen.go
0.00% <0.00%> (ø)

pkg/pwgen/memorable.go
87.50% <50.00%> (ø)

internal/action/pwgen/commands.go
100.00% <100.00%> (ø)

pkg/pwgen/cryptic.go
86.59% <100.00%> (ø)

pkg/pwgen/pwgen.go
95.24% <100.00%> (+1.49%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 774eb1d...55b9aae. Read the comment docs.


add "contract status" to table show contract "Active | Close "

Status lable added.
I think when the contract is closed save button and other input box should hide and disable.