Project detection not working on WSL windows

[x] I'm sure this issue is not a duplicate.

VS Code instances running on WSL do not seem to auto detect project or workspace settings, this means that opening a project on Windows will set a preset and later on, any project that i open from WSL will be using that preset, the only way to change it is setting up another preset on a Windows project. Icon presets seem to work really well on a single window, but with multiple vscode windows, they just mix and act weirdly, i made a comment in here  about presets mixing between to vscode windows tho that is under another discussion, i made searches about WSL and auto detection and couldn't find anything so i hope this is not an isolated issue, but it would be nice to at least be working on a single WSL project at a time (due to the preset mix) and get auto detection.
In this example i'm opening two workspaces that switch between angular and nestjs preset, both projects are brand new created with @angular/cli and @nestjs/cli
Also on that demo i didn't try to se this "vsicons.projectDetection.autoReload": true on the WSL instances but later on i did and the result was the same, nest project stayed with angular preset even after deleting settings.json(workspace) reloading windows, creating a new settings.json(workspace) and setting "vsicons.presets.nestjs": true in it.

This requires investigation something that I can't do atm. If I had to guess I would say that this behavior is occurring because Windows and WSL share the same vscode installation and thus the same icon manifest file. How about two separate vscode installations, one user and one system or even portable mode?

Come to think of it user and system installations share the same icon manifest file too.

It may have something to do with #2199

So I found some time to look into this. Things are exactly as I suspected.
Both vscode instances use the same vscode-icons extension simply because we have told vscode that it should run vscode-icons from the local file system and not the remote one (see #2404). If you open the Extensions explorer you will notice that vscode-icons is only installed locally and not on WSL and that vscode does not provide an option to install it in WSL.
Now if you go to your settings (settings.json) and add the following:
"remote.extensionKind": {
    "vscode-icons-team.vscode-icons": ["workspace"]
}

by reloading vscode you will now notice an option to install vscode-icons on WSL. By doing so you will see vscode-icons showing up in the WSL part of the Extensions explorer.
From there on, experiment further.
Hope this helps.

That workaround works perfectly! Is this something that you could eventually work into the extension itself to avoid having to do this on each machine?

We are still contemplating on it.

@JimiC I have made this and I could make the installation successful in WSL, but somehow, the extension is appending C:\ to my project path. This is what the logs show.
[2021-12-10 09:44:12.579] [exthost] [info] ExtensionService#loadCommonJSModule file:///c:/Users/Omar/.vscode-insiders/extensions/vscode-icons-team.vscode-icons-11.8.0/dist/src/vscode-icons.bundle.js
[2021-12-10 09:44:13.359] [exthost] [error] Activating extension vscode-icons-team.vscode-icons failed due to an error:
[2021-12-10 09:44:13.359] [exthost] [error] Error: ENOENT: no such file or directory, open 'C:\home\omar\CODE\Lullo\Client\package.json'
[2021-12-10 09:49:45.396] [exthost] [info] extension host terminating: renderer closed the socket
[2021-12-10 09:49:46.402] [exthost] [info] exiting with code 0

I'm suspecting this is why my custom icons aren't loading


make update_operating_status_interval configurable

make update_operating_status_interval configurable

cleaned some commit msg history and submitted a new one. just closing this one.


RFC: in TC resolution, ← instantiateMVars (← inferType mvar) is computed 2 or 3 times over for every new goal

Proposal
These three lines do this same computation:
https://github.com/leanprover/lean4/blob/f05a82799a01569edeb5e2594cd7d56282320f9e/src/Lean/Meta/SynthInstance.lean#L286
https://github.com/leanprover/lean4/blob/f05a82799a01569edeb5e2594cd7d56282320f9e/src/Lean/Meta/SynthInstance.lean#L243
https://github.com/leanprover/lean4/blob/f05a82799a01569edeb5e2594cd7d56282320f9e/src/Lean/Meta/SynthInstance.lean#L474
I propose to do it once, and pass this result as an argument to all these functions.

Is there evidence that this change by itself would have a performance impact?

I hadn't tested it separately yet, but here's a PR for it now: #4379
Fixing only this also involved moving the withMCtx around a bit, because the instantiateMVars needs to be in the relevant metavariable context. I have an issue for this here #4286

The result of the test on Mathlib is that this change is responsible for all of the performance improvement in #4277.


Improve INFOPLIST_FILE handling to only omit used Info.plist's from Copy Bundle Resources Build Phase

Closes #1045
Background
Prior to #945, a lot of XcodeGen was build assuming that a file named Info.plist was the value used in INFOPLIST_FILE build setting. This file is different to other plists because it shouldn't be included in the copy bundle resources build phase.
If INFOPLIST_FILE was named anything other than Info.plist, by default, XcodeGen would include the file in the build phase which would result in a warning.

warning: The Copy Bundle Resources build phase contains this target's Info.plist file 'HardwareSupport-Info.plist'. (in target 'HardwareSupport' from project 'LegacySupport')

#945 took a stab at fixing this by relaxing the checks so that any file that had been suffixed with Info.plist would then be treated as an Info.plist to support the old(?) pattern of using a file named {TargetName}-Info.plist. This however caused a noticeable regression for projects that include the GoogleService-Info.plist file which should be treated as a resource (this file is nothing to do with an bundles info property list and instead is a resource used by popular Google SDKs.
So I looked back at @yonaskolb's initial comment:

but a proper fix might be to check the build settings of the target to find all the target's info plists and exclude them.

Description
In this change, I am taking a stab at improving things for both scenarios. To begin with in 5a856b8, I checked out 2.18.0 and added a GoogleService-Info.plist file to the resources and regenerated the fixtures as expected. I then went back to 2.19.0 and also generated the fixtures with a target which defines the Info.plist as App-Info.plist in
61e2c3b.
Now that we have updated fixtures that reproduce both use cases, it should now be a matter of changing things until the tests pass 😬
In PBXProjGeneractor, the final INFOPLIST_FILE build setting value for a given target isn't resolved until wayyyy after the array of SourceFile objects (which define the desired buildPhase) are created. I figured that the first task would be to resolve these values upfront so that we can use the actual INFOPLIST_FILE values to influence the SourceFile's that we resolve (instead of guessing based on filename). So far, I've refactored the build setting resolution logic in 56ef412e6e87431a33ab71470a6ca130a1ac27c4 so that I now have a [Config: String?] dictionary that contains the INFOPLIST_FILE values for each build config of the target.
What's Next
The next step is to somehow pass this information into SourceGenerator.getAllSourceFiles(targetType:sources:) so that it'll return a SourceFile with it's buildPhase value set to BuildPhaseSpec.none for the real Info.plist files (to make the tests pass). This is where I'm a little stuck.
After trying to poke around, I felt like it was quite difficult to pass the set of files deep into the SourceGenerator and before I continue, I want to confirm that I'm on the right tracks.
Part of me did wonder if I could maybe instead just mutate the target.sources array before passing it into the SourceGenerator since I could just pretend that it was just set in the spec file, but I wondered if it was wrong to override the value that was potentially set in the project spec in doing this. Ideally we should probably prioritise the explicitly set value before and just use .none as a fallback before the result of getDefaultBuildPhase(for:targetType:).
I've run out of time this evening on this one so before I look at it with a fresh pair of eyes later this weekend, I thought I'd push the changes up and maybe somebody else could give some input (👀 @yonaskolb 😬)

Thanks so much for looking at this @liamnichols, I like the way you're approaching it so far

@yonaskolb: I hit a bit of a bump here while trying to think things through throughly...
In theory, the value of the INFOPLIST_FILE build setting could be something like ${SRCROOT}/SupportingFiles/${CONFIG_NAME}-Info.plist and if this is the case, we'd have no way to be able to figure that one out how that maps to a SourceFile to be able to influence the buildPhase that is assigned.
While I think it's probably uncommon to change the filename based on random build settings, I don't think it's unrealistic for people to have carried over SRCROOT or concepts like that as a way to use a path relative to the project root.
These are the options that I came up with:

Just assume that if the user defined INFOPLIST_FILE in their spec, that the value is relative to the project root and does not include any variables that require expansion and accept that behaviour will be inconsistent.
Attempt to expand any variables - But I think that this is impossible because we don't have the compile-time build settings?
Only automatically alter the build phase in the following circumstances:
i. If the user defined the info.path property in the spec
ii. If the user neither defined info.path OR INFOPLIST_FILE and we instead XcodeGen automatically found the file called exactly Info.plist in the targets sources.

Option 3 would basically mean that if you defined INFOPLIST_FILE in your project spec and your info property list was called something other than Info.plist then you would have to explicitly stop that .plist from being treated as a bundle resource. I think that this is the most sensible option, but I'm interested in feedback.
Either way, we would probably have to document this a bit better somewhere.

@liamnichols. I think we want to optimise for a working project. At the end of the day, if the Info.plist file is bundled as a resource it's not a major deal, and there will always be a way to remove this by explicitly setting all those files to resource none. So we should use our best efforts to exclude it, but within reason. Option 2 is something I don't want to handle. I think option 1 is fine. We'll check against the path, and if that includes build variables it just won't match and won't be excluded.

Before I started trying to pass the phases in, I found a couple of method arguments that seemed to be redundant in SourceGenerator and did a little cleanup in 7dd55fa5b91fb0166276034073e843b0cc7c146d
getSourceFiles(targetType:targetSource:path:)
This method accepted a path value but it was always project.basePath + targetSource.path so seemed redundant injecting it given that we could compute the value inside the method.
Doing this also avoided a bit of confusion where parts of the method used path and other parts declared a local  folderPath argument which was also the same - let folderPath = project.basePath + Path(targetSource.path).
I might have missed the initial intentions of injecting the path argument, but since its redundant today, it seems ok to remove it.
generateSourceFile(targetType:targetSource:path:buildPhase:fileReference:)
This method was my main target during refactoring. It accepted a buildPhase property (optional, nil) that if set, was used as a priority over the value specified as targetSource.buildPhase.
It turns out that this argument was only ever used in one of the calls to this method, and instead of using this value as an override, the intention was that actually this value was to be used as a fallback in the event hat targetSource.buildPhase had not been set with had resulted in some duplicated code.
To work around this, I removed the argument and actually just incorporated the use internally within the method by refactoring some common logic into resolvedTargetSourceType(for:at:) in order to override the buildPhase to be .resources when the SourceType is .folder. The reasoning behind this should hopefully be clear in the diff of this commit specifically

Looks good to me @liamnichols. Thanks for cleaning this up. This file and the PBXGenerator have had all sorts of hands in it changing different things, so they need a bit of love 😄

@liamnichols whenever you have time to update this, it would be good to release this, as it's a fix in some respects

Ah thanks for reminding me. I’ll have some time over this weekend to get it ready 👍

Things left to do:

[ ] Rebase from master
[ ] Update CHANGELOG.md
[ ] Move the call to Path.absolute() into getInfoPlist(for:)
[ ] Add some extra unit tests to SourceGeneratorTests.swift
[ ] Clarify behaviour in docs if needed

[  ] Clarify behaviour in docs if needed

Not sure if i need to do anything here actually. Let me know what you think @yonaskolb.
This is ready now 👍


Prevent publishing over another users changes

When another user has modified a page published Obsidian then don't allow publish over the top.
Idea from https://github.com/obsidian-confluence/obsidian-confluence/issues/34

Resolved in https://github.com/obsidian-confluence/obsidian-confluence/commit/5c42d7787cf4c53098759ac221a81369e033df3d

@andymac4182  - this should be optional or overridable. I want to overwrite changes made by other users.

Will look shortly at adding a setting for this.


platform: refactor stellar payment observer

PR Checklist
PR Structure

[x] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
[x] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
[x] This PR's title starts with name of package that is most changed in the PR, ex.
paymentservice.stellar, or all or doc if the changes are broad or impact many
packages.

Thoroughness

[ ] This PR adds tests for the most critical parts of the new functionality or fixes.

What
Refactor Stellar payment observer to address the feedback from PR #179. The changes are:

Use limit=200 in the streamer for improved efficiency.
Explicitly order streamer with order=asc
Update the payment observer to accept both payments and path payments
Add parameter validation in stellarPaymentObserverService.

Make sure only successful operations will be taken into account.
Validate the payment amount

Rename token to cursor for improved clarity. This includes renaming a few classes to keep the consistency.
Use Log.warn.
Smaller polishes

Reference Server Preview is available here:https://anchor-ref-pr188.previews.kube001.services.stellar-ops.com/SEP Server Preview is available here:https://anchor-sep-pr188.previews.kube001.services.stellar-ops.com/


Enhancement: Github Conditional Requests

There is a 5000/hour rate limit for Github API requests. This limit may be extended upon request, but only if the application is already using the Conditional Requests feature (https://developer.github.com/v3/#conditional-requests). Suggest using the Redis system to cache contents calls to Github, globally.

Conditional requests now supported (and lightly tested). Server should now handle moderate traffic without exhausting the Github API limits.


support vuejs

Settings
Example:
        case 'scss':
            languageId = 'css';
        case 'css':
            beatiFunc = jsbeautify.css;
            break;
        case 'json':
            languageId = 'javascript';
        case 'javascript':
            beatiFunc = jsbeautify.js;
            break;
        case 'html':
            beatiFunc = jsbeautify.html;
        case 'vue':
            beatiFunc = jsbeautify.html;
            break;
        default:
            showMesage('Sorry, this language is not supported. Only support Javascript, CSS and HTML.');
            break;

It can just fall through
case 'html':
case 'vue':
    beatiFunc = jsbeautify.html;

@lizhongyi @jimmywarting
You're writing the code you want? How about about PR?

@lizhongyi  Why closed?

@bitwiseman I think they may not have enough free time, please forgive us.   :(

Any progress on this? Not many tools out there as good as or as widely used as jsbeautify and not supporting Vue is a huge headache.

@lonerdottie - It looks like this would a straightforward PR.  Make the change suggested above, add a few tests.  I doubt it would be full support, but it would be something.

@lonerdottie - Actually, it is unclear to me what part of the code it referred to above.  Are you using http://jsbeautifier.org or a specific editor/plugin?  What happens to your vuejs code when you put it in http://jsbeautifier.org ?

@bitwiseman take a look at this code for example and run it through.  The obvious issue when running it in jsbeautifier.org is that it ignores everything within the 'template' tag.
https://gist.github.com/lonerdottie/5f47b66dccd8c3ca6a2528adc06fcba4

@lonerdottie  Yeah, we treat template as "unformatted".  (see https://github.com/beautify-web/js-beautify/blob/master/js/src/html/beautifier.js#L99) .  You could try overriding that setting.
What other issues do you see?


Chart rendered in Grafana does not match that inside Azure

for some reason the legend doesn't transfer and the X-axis is a generic count rather than by the order I wish to use.
Query:
W3CIISLog
| where TimeGenerated >= ago(7d)
| where $__timeFilter(TimeGenerated)
| summarize
safari = countif(csUserAgent contains "Safari"),
ios = countif(csUserAgent contains "Mobile Safari"),
samsung = countif(csUserAgent contains "Samsung"),
chrome = countif(csUserAgent contains "Chrome"),
edge = countif(csUserAgent contains "Edge"),
firefox = countif(csUserAgent contains "Firefox") by bin(TimeGenerated, 6h)
| order by TimeGenerated asc
| render barchart
Azure Log Analytics Output:

Grafana Output:

Not sure what I'm missing.
Thanks,
James

I added a line to the query
W3CIISLog
| where TimeGenerated >= ago(7d)
| summarize TotalConnections = count(csUserAgent),
safari = countif(csUserAgent contains "Safari"),
ios = countif(csUserAgent contains "Mobile Safari"),
samsung = countif(csUserAgent contains "Samsung"),
chrome = countif(csUserAgent contains "Chrome"),
edge = countif(csUserAgent contains "Edge"),
firefox = countif(csUserAgent contains "Firefox") by bin(TimeGenerated, 6h)
| order by TimeGenerated asc
| render barchart
Ok, so I changed X-axis to time. And I see a bit different items, but still only getting a total collection output. I'm looking for a 7d aggraegate, not live connections for any single day. Putting 168h in Time range still just gives me a more accurate X-axis, but does not break out the values as I need them.

The default Units for Y aren't what I'm attempting to measure,

If I could figure out how to individual variables against the TotalConnections, rather than it just ignoring my vars, it would be good.
Thanks,
James

I'm guessing something happened with the way Grafana can access Azure Monitor as a data source? I don't need to write a query and everything is a dropdown selection.  Yay!
Thanks for everyone's assistance.


Enhancement : add reconciled amounts total

Hi,
On the dashboard we can consult the amount of each account. It includes amount of all operations. Could it be possible to add the amount of reconciled operations ?
Or maybe the delta between reconciled ones and the total amount.
It could be shown as this : 1375,26 € (1325,26 €)
Or like this : 1375,26 € (-50 €)
It could be useful to check with the online bank account if everything has been well reconciled.
Thanks ! :)
(Sorry for my English...)

Hello 🤗
I've check in Firefly, maybe I'm wrong, but I've seen it nowhere...
Thanks for your interest for this feature 🤗

I understand.
I thought, since you added reconcile status, you could calculate the sum of unreconciled amounts. Maybe it could be a way...
But it's ok, you done great jobs 👍


Context menu (sometimes) doesn't work on Firefox + MacOS

When using Firefox in MacOS, the right-click sometimes hide the context menu
Reported by @lucascpessoa

After some hours of frustration, I believe I've found the cause: the browser (sometimes) fires the 'click' event right after the 'contextmenu'. This triggers Menu#destroy (like any other click after the menu is shown).
The solution is very hacky (suggestions welcomed): after 'contextmenu', wait for 100ms before listening to 'click'


[$2000] Android-Chat-Unable to select all text in compose box

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Launch New Expensify app
Go to any chat
Enter several lines of text until a button to expand the compose box is visible
Select all text

Expected Result:
The text is selected
Actual Result:
Only part of the text is selected, all text cannot be selected
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.33-3
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/78819774/a045a9dc-2c7e-41cc-a199-759350fd612a
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01c89c63988d1a2bbc
Upwork Job ID:<PHONE_NUMBER>559113216
Last Price Increase: 2023-07-03

I dug into this a bit. It seems like it might be a bug with React Native's TextInput. I was able to repro here: https://snack.expo.dev/dD1gqbbV4?platform=android .
I haven't found a workaround yet. Just wanted to leave this update here for anyone investigating.

reviewing this today!

reviewing today and posting next steps!

Proposal
Please re-state the problem that we are trying to solve in this issue.
Android-Chat-Unable to select all text in compose box
What is the root cause of that problem?
Default behavior's of the TextInput selection for android selects only visible text in text area. As this is not a functionality fail but a difference in the way the feature works versus the expected user functionality that is required.
https://reactnative.dev/docs/textinput
What changes do you think we should make in order to solve the problem?
We can wrap the android TextInput with a TouchableOpacity component and then use listen to the mimic the double tap event by comparing the time difference between each tap on the onPress prop.
https://github.com/Expensify/App/blob/2ed613d27c45bfc4d9451f0073883bb00927e05a/src/components/Composer/index.android.js#L96-L101
<TouchableOpacity onPress= {this.handleDoubleTap}>         <RNTextInput                 autoComplete="off"                 placeholderTextColor={themeColors.placeholderText}                 ref={(el) => (this.textInput = el)}
This accepts a function that can be used to select and highlight all text in the input box
`
class Composer extends React.Component {
constructor(props) {
super(props);
    this.state = {
        propStyles: StyleSheet.flatten(this.props.style),
        lastPress: 0,
    };
}
handleDoubleTap = () => {
    // check the time difference on screen taps
    let delta = new Date().getTime() - this.state.lastPress;

    if (delta < 200) {
        // focus on text
        this.textInput.current.focus();
        // select all text
        this.textInput.current.selectAll();
     }
}
this.setState({ lastPress: new Date().getTime() })

`
https://github.com/Expensify/App/blob/b7004b9e9ff2bb86d6357e62ce15659322032c65/src/components/Composer/index.android.js#L67-L72
What alternative solutions did you explore? (Optional)

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~01d4badf0db1b2223d

@eVoloshchak let's get this proposal reviewed please! 🙇♂

I'm not sure why can't select all text. As you can see, if user touches the current cursor position, select all menu appears.
https://github.com/Expensify/App/assets/126839717/36ae364f-b92a-438e-86dc-87a8cbb9eea0

I agree with @kmwamasali's assessment here

Default behavior's of the TextInput selection for android selects only visible text in text area. As this is not a functionality fail but a difference in the way the feature works versus the expected user functionality that is required.

I can see the identical behavior in other apps on Android, I don't think this is a bug

@eVoloshchak since this default behavior exists in other files what is your advisory on proceeding

@eVoloshchak let's get an update today?

since this default behavior exists in other files what is your advisory on proceeding

@kmwamasali, I think "Do nothing" would be the best approach here. It's a system-wide behavior, we shouldn't change it

Agreed

@eVoloshchak are you saying we should leave this behavior as is? And not adjust it to mirror other platforms?

Reviewing today!


[FEAT]: registry-cli, docker-compose should have all the 2.x changes reflected

What is the feature request for?
Registry Core
What problem/inconvenience (if any) will this feature solve?

[ ] registry-cli, docker-compose should have all the 2.x changes reflected

Describe the feature clearly.
Add a new feature that does ....

https://github.com/Sunbird-RC/registry-cli/pull/3


"copy generated kitti_dbinfos_train.pkl into the kitti data path"

Hi,
Congrats on your paper's acceptance to CVPR'23! According to the provided instructions "copy generated kitti_dbinfos_train.pkl and kitti_infos_train.pkl into the kitti data path", there doesn't seem to be any newly generated kitti_dbinfos_train.pkl file? The previously generated one has already been moved into folder origin_label. I wonder what's to be done here.
Thx.

Hi. Thanks for your interest.
We're sorry for the error in our previous expression. All you need to do is copy the generated "kitti_infos_train.pkl" file, and we'll update the Readme to correct the mistake.


[INLONG-4144][K8s] HTTP error appears on the login dashboard when deploying InLong using helm

[Bug][K8s] HTTP error appears on the login dashboard when deploying InLong using helm #4144
Title Name: [INLONG-XYZ][component] Title of the pull request
where XYZ should be replaced by the actual issue number.
Fixes #
Motivation
Explain here the context, and why you're making that change. What is the problem you're trying to solve?
Modifications
Describe the modifications you've done.
Verifying this change
(Please pick either of the following options)

[ ] This change is a trivial rework/code cleanup without any test coverage.

[ ] This change is already covered by existing tests, such as:
(please describe tests)

[ ] This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation

fix #4144 [Bug][K8s] HTTP error appears on the login dashboard when deploying InLong using helm](https://github.com/apache/incubator-inlong/issues/4144)


Remove autoload code, it's not thread safe

Hi guys,
We've been running into problems with the autoloads in grape. When running grape in a multithreaded app server it would have multiple concurrent requires which can lead to all sorts of nasty conditions.
I couldn't discover a way to put in thread safe mode so I just removed all the autoloads.
Do you have any other concerns with running Grape in a multithreaded app server?

Woops, pardon me this doesn't work yet, I still have to reorder the dependencies and run the suite. Just submitting the PR as a discussion point for now :)

Alright, specs pass this way :)

Hmm, this will likely break people's workflow. How do other projects deal with this? Can we make Grape thread-safe(r) instead or turn auto-load off in production?
Either way it needs a CHANGELOG entry.

Mr Tenderlove has a blog article about it here:
http://tenderlovemaking.com/2012/06/18/removing-config-threadsafe.html
In ActiveSupport is a module called Autoload, it has a helper method that allows you to call eager_load on modules. What Rails does is including that module and then when config.thread_safe! is set, call it amongst other things.
The module is really tiny, you could opt for that as well, it would disrupt peoples workflows a bit less I suppose.

Grape already requires ActiveSupport, so maybe base it on that?

Definitely, I'll push -f a version that uses that when I get the time, in the mean time I've ran into a thread safety bug in activerecord as well, so enabling multithreading is fun times all around for us ;)

AFAIK, autoload is thread-safe in CRuby 2.0+. Am I wrong?

It is yes, but only in the sense that Ruby won't load the same file twice in the same thread. Race conditions and deadlocks might still occur. In any case, we are still locked on 1.9 for at least a couple of months. If you wanna go for just 2+ support then that's ok as well, we'll just keep using this fork in the meanwhile.

Thanks @dm1try. I definitely don't want to merge this "as is", I would however merge something that plays nice with ActiveSupport autoload options. Will leave this open.

PR #980 fixes this I believe


All functions directly present instead of single button

Hi,
I was trying your git plugin and saw that instead of single git icon, shown in your demo image,

directly all the options are displayed which really clutters the logseq ui
.

I have similar issues. Seems like the author use the mac layout which is incompatible with windows layout.

Sorry that Windows has not been tested since the release, version 1.0.2 should fix this issue.

@haydenull The new version does fix it! Thanks!

Currently, it can only be closed by clicking a button in popup, such as show log.

Cool! Noted.

Multiple buttons insted of single icon issue is resolved. But still facing same issue as faced by @CharlesChiuGit that the pop-out pane won't collapse no matter where I click. Other plugins icons have this feature. As you informed we can click on any button and it does that work and collapses the pop-out pane.
Windows 10, Logseq v0.7.0, logseq-git-plugin v1.1.0

In case somebody else has this issue: I was having the same issue as @CharlesChiuGit with the newest version of your plugin, but with the older Logseq v0.6.6.
Updating logseq to v.0.6.9 fixed the issue and now it works great.
Thanks a lot @haydenull !

Yes, logseq has modified the html structure of header in the new version.


Description formatting improvements

Story
As a user
I want nicely formatted proposal descriptions
So that I can read them
Impacts areas below

Go to governance homepage, it needs formatting
Look at the main proposals screen, it needs formatting
Click through to a proposal details screen, it needs formatting

Acceptance Criteria
Proposal Details View

[ ] Description should be removed from where it currently is since the text can be long
[ ] Instead display a type specific text string, in a few cases this is currently shown below, move it up instead

Network Param: "Change" network_param to new_value
New Asset:  Symbol: symbol. ERC20 Contract Address contract_address (This is basically already like this)
Update Asset: Asset_Name (asset_id)
New Market: This is already fine, it shows something like "Code: AAVEDAI.MF21. tDAI settled future."
Update Market: Market_Name. Market ID: market_id (small addition)
Freeform... dont know or care!

[ ] Description should appear as its own section under the heading "Description" above the block with the IDs etc
[ ] It should follow markdown formatting and we will put together a template for this

Proposals View & Homepage View

[ ]Apply short summary logic as above
[ ] Remove the description

Tasks

[ ] Agree
[ ] Build

Sketch
Additional details / background info

@JonRay15 no ability to get the asset name for the update asset data from the proposals query at the minute. I'll raise a ticket with core as I don't want to get this data in a separate query.

Understood. Anything else useful we can get from the proposal?

@JonRay15 no, we get everything we currently can. Core ticket here: https://github.com/vegaprotocol/vega/issues/8275

@JonRay15 Same for update market. Added to the core ticket


Ability to cache only some gems

I have a Rails app which depends on a couple of private gems.  These gems are not on GitHub; they are on my file system.
To deploy my app I need to get the private gems onto the server.  I could either push them to the server and install them beforehand, like a private version of bundle exec rake release, or include them in my app's repo so they are deployed with my app.  As there are only two private gems I don't feel it's worth running a private gem server.
The first option would mean cluttering up the private gems with knowledge of deployment servers, which doesn't appeal.
The second option requires copying the private .gem files into my app's vendor/cache directory and updating the Gemfile:
# 1. Gemfile - before
gem 'my-private-gem', '~> 1.0.0', path: '~/code/src/my-private-gem'

# 2. Either:
$ bundle package --all
$ rm vendor/cache/{all,other,gems}.gem

# or:
$ mkdir vendor/cache
$ cp ~/code/src/my-private-gem/pkg/my-private-gem-1.0.0.gem vendor/cache

# 3. Gemfile - remove the path option
gem 'my-private-gem', '~> 1.0.0'

# 4. Install without adding the others gems to vendor/cache
$ bundle install --no-cache

I don't want to add all the gems to vendor/cache because it would bloat my app's repo unnecessarily, and it would feel like the tail wagging the dog.
I think this would be neater with the following process, suggested by @indirect:

After thinking about this more, I think the best UI for this looks something more like this:

Gems already in vendor/cache are updated automatically during install
Gems not in vendor/cache are not added automatically during install
Adding all gems to vendor/cache is done by running bundle cache
Adding a specific gem to vendor/cache is done by running bundle cache foo

Hope that makes sense!

I have also found that having some but not all gems in vendor/cache prevents bundle install --deployment from working: it fails with Some gems seem to be missing from your vendor/cache directory..
My workaround is to use this instead:
bundle install --no-cache --path vendor/bundle --without development:test

Is there a particular reason the path option in the Gemfile is not suitable to your needs?

You mean copy the private .gem files up to a known location on the server beforehand, and point to that location with the path option in the :production group in the Gemfile?  I.e. the first option I suggested (and dismissed ;) )?

i mean, just have the gem source as part of the app and use path to specify it's location. such as:
gem 'my_private_gem`, path: 'ext/my_private_gem`

Oh, because they are independent chunks of functionality that are used by other apps too.

I have found that it's a bit clunky using the --no-cache flag on every bundle install so instead my Gemfile now installs from private repos on GitHub using a personal access token.


Move es2k related guide to guides/es2k and add doc to build P4 artifacts

Move es2k_guide.md to docs/guides/es2k/
Add documentation to build P4 artifacts

Review in progress. I'm committing my initial comments now and will follow up with more later.

Sure, es2k-guide.md will be heavily modified. I changed the location of guide and that's why it shows as a new file. Please don't review the contents of this file yet.

@nupuruttarwar @5abeel
Suggestion 1

es2k-guide.md
building-p4-programs.md
running-infrap4d.md
installing-tls-certificates.md (note name change)
security-guide.md

This is consistent with the principle of keeping PRs small and focused, it allows them to evolve independently, and it prevents documents that are essentially ready from being blocked from ones that are still being thought out.
Suggestion 2

Establish a minimum threshold (or perhaps a series of stages) for each document.
Allow an initial merge once a document has met the minimum threshold, with subsequent merges as it achieves readiness milestones.

Dan has made it quite clear that things don't have to be perfect for us to merge them. The Google code review standard is that a CL "improves the quality of the codebase." For documentation, I think this would mean that we have conveyed a significant amount of correct information (gain in quality) and there is little or no misinformation (loss of quality).
For documents that are referenced by other documents, we need to ensure that links are updated in a timely fashion. Moving or renaming a document is a breaking change. Our chief concerns are documents referenced by the IPDK Build README and the Networking Recipe README.


Resize colonne

Hello,
It's possible to resize colonnne at the size of data?

Please solve this problem.

Has this been solved yet or does anyone have a workaround?


groupby sorting - don't specify, or forbid?

From the last call, the cuDF folks stated that a goal of theirs is to let users prototype with pandas on CPU, and then switch to GPU with cuDF and have their code "just work".
One issue they currently face is groupby: pandas sorts by default, so that's what their users expect
Even if pandas were to adopt the standard in its main namespace, this would not accomplish their goal. This is because whilst the standard doesn't specify whether groupby should sort, it doesn't forbid it either - thus, pandas could continue sorting in groupby by default whilst respecting the standard, but then cuDF would be no better off.
cuDF also said that if pandas were to implement the standard in a separate namespace, then users wouldn't necessarily want to look up the standard and would expect things to just work as they're used to (i.e. pandas main namespace).
The only way I can think of of accomplishing cuDF's goal, without changes in cuDF, would be:

the Standard forbids sorting in groupby
pandas adopts the standard in its main namespace

I don't mean to be a "Pessimistic Pete", but the second one seems unlikely to land. Some deviations from pandas in cuDF may be warranted.
My suggestion is:

for cuDF's end-user groupby issue, they could remove the default from groupby, thus forcing users to specify a value for sort. Users would be required to type an extra 2 words, but they'd probably be better off because of it
the Standard stays developer-oriented, rather end-user-oriented
to minimise surprise to users of the Standard, that the standard explicitly forbid sorting in groupby. It's been mentioned that developers might only test their code using pandas and then expect it to work with other DataFrame libraries, so this would reduce the chances of surprises. And so long as it's in a separate namespace, there's no risk of breaking millions of users' code

EDIT: on the last point - maybe the standard doesn't need to forbid sorting, but the pandas implementation of it shouldn't sort by default

cc @shwina @kkraus14 for visibility (I don't know if you receive notifications from this repo, so tagging you just in case)

Thanks, @MarcoGorelli -- I agree with everything you've said here.
I do think that a user-facing dataframe API has much more value than a purely developer-facing API. And while I recognize that Pandas will likely always provide a superset of that API, I also think that it should be at minimum an implementation of the standard, and as flexible an implementation of the standard as possible.
So I think this would be the best outcome here:

maybe the standard doesn't need to forbid sorting, but the pandas implementation of it shouldn't sort by default

Hey - when I wrote

the pandas implementation of it shouldn't sort by default

I meant if pandas were to implement the standard in the separate namespace - it would be too disruptive of a change for the pandas main namespace.
I think this is probably the opposite of what you want - in which case, would making sort a required argument in cudf.groupby be feasible?

Sure, but if the standard makes no promises about groupby sorting, then there's nothing stopping pandas from keeping its current groupby behaviour.
And then that doesn't solve your issue that people come to cuDF expecting to also sort in groupby, which you said you've like to change

My takeaway from last week is that "The Standard" is completely irrelevant and all anyone really cares about is the pandas behavior, where cudf wants an API change.  The place to ask for that is on the pandas tracker, and the way to convince people is to argue it will benefit pandas users.

And then that doesn't solve your issue that people come to cuDF expecting to also sort in groupby, which you said you've like to change

From my perspective, the challenge is that there's a parameter with a default value and documented behavior from Pandas as opposed to it being an implementation detail. I.E. the docs for the sort parameter say:

Sort group keys. Get better performance by turning this off. Note this does not influence the order of observations within each group. Groupby preserves the order of rows within each group.

This dictates a few different things:

Default behavior is to sort by the grouping keys with sort=True
Pandas guarantees that the input dataframe order is maintained within each group for "payload" values regardless of the sort value
The fact that the input dataframe order is maintained for grouping keys when sort=False isn't documented behavior

It's unclear to me if this is an implementation detail or if this is guaranteed behavior from Pandas

In an ideal world, the API we're trying to define wouldn't have a sort keyword and would explicitly make no promises regarding the ordering of grouping keys or "payload" values as many dataframe implementations don't / can't support these behaviors.

My takeaway from last week is that "The Standard" is completely irrelevant and all anyone really cares about is the pandas behavior, where cudf wants an API change. The place to ask for that is on the pandas tracker, and the way to convince people is to argue it will benefit pandas users.

The conversation from last week was a bit all over the place, but my main takeaway was that the API we're trying to build now is targeted at library / applications developers much moreso than end users and that the first dataframe implementation of the standard that library / applications developers will build against will likely be Pandas. This leads to a concern that anywhere that Pandas supports more than the standard, that developers may use those features while thinking they are targeting the standard. They may then be disappointed when they don't get the implementation portability they were looking for.
For targeting end users, we threw around a lot of frustrations, ideas, concerns, and more. A lot of that surrounded the existing inertia of Pandas and how we can give Pandas users a happy path towards alternative dataframe implementations that are mainly offering enhanced performance and scalability as compared to Pandas. I agree that if our conversations lead down to "we want to change Pandas" then those conversations should then move to the Pandas issue tracker. I don't think we're there yet and have just really started the "end user" conversations.

In an ideal world, the API we're trying to define wouldn't have a sort keyword and would explicitly make no promises regarding the ordering of grouping keys or "payload" values as many dataframe implementations don't / can't support these behaviors.

OK, sure. Let's say the standard doesn't have a sort keyword.
Pandas would then implement a superset of the API (like Ashwin said) and would keep its sort keyword with its current behaviour.
How does this help cuDF?

OK, sure. Let's say the standard doesn't have a sort keyword.
Pandas would then implement a superset of the API (like Ashwin said) and would keep its sort keyword with its current behaviour.
How does this help cuDF?

If it's all going to be in the main namespace then I think ultimately it causes problems and makes the work we're doing here less valuable. I think it would be an unreasonable expectation to have people double checking all of their work to see if they're using parameters that are in Pandas but aren't in the standard. People would likely just continue writing Pandas code as they always have and other dataframe implementations will continue to get pressure to match Pandas API and behaviors.
If it's a separate namespace and Pandas doesn't implement the sort keyword in that namespace then I think that there would be enough justification to get library / application developers interested in having dataframe portability to not be reliant on the implementation details of Pandas with regards to its ordering behaviors.
To be clear, this is beyond just cuDF. My personal experience is with cuDF, Dask (both as a producer and consumer of dataframe APIs), Spark, Arrow, and Ibis (again both as a producer and consumer), all of which can be multi-threaded and/or distributed. This often makes following existing behaviors of Pandas challenging and a significant detriment to performance in many situations.

Sorry, I'm confused, and I feel like I'm hearing contradictory statements.
On the call, Ashwin said

You're right in that maintaining two namespaces is not tenable for the long run. The real canary is pandas. What pandas does, everyone will generally follow. Ideally, pandas would exclusively support the standard API. My hope is that, even if not 1,2,3 years, that eventually it would happen.

And here, you're saying that you want a separate namespace in which groupby doesn't have a sort keyword.
Which is it? Do you want the main pandas namespace to change so that the transition to cuDF is smoother, or do you want a separate namespace in which everything can be compatible from the get-go?
(I'm not sure how my tone is coming across, so just to be clear: I'm perfectly calm, and don't mean anything I've written in an aggressive/sarcastic way. I'm genuinely confused 😄 , and trying my best to make sense of what we're trying to achieve here)

On the call, Ashwin said

You're right in that maintaining two namespaces is not tenable for the long run. The real canary is pandas. What pandas does, everyone will generally follow. Ideally, pandas would exclusively support the standard API. My hope is that, even if not 1,2,3 years, that eventually it would happen.

And here, you're saying that you want a separate namespace in which groupby doesn't have a sort keyword.
Which is it? Do you want the main pandas namespace to change so that the transition to cuDF is smoother, or do you want a separate namespace in which everything can be compatible from the get-go?

I can't speak for Ashwin.
If there isn't a separate namespace in Pandas, I could only imagine having some package or something to validate you're following the standard and not using functions / keywords that aren't part of the standard. Without that, it will put undue burden on library / application developers to build against the standard effectively. I completely understand and think it's completely reasonable if the Pandas community did not want to support a separate namespace.
In an ideal world purely from the cuDF and other libraries perspectives, Pandas only supports the standard API and all dataframe implementations only support the standard API so users get full portability. In the real world, that isn't in Pandas users' best interests and isn't a feasible thing to do.

(I'm not sure how my tone is coming across, so just to be clear: I'm perfectly calm, and don't mean anything I've written in an aggressive/sarcastic way. I'm genuinely confused 😄 , and trying my best to make sense of what we're trying to achieve here)

Your tone has been perfectly cordial 😄. I appreciate the pushback to make sure that we're actually going to drive impact as opposed to being a perfect example of the xkcd comic (https://xkcd.com/927/).

I applaud @MarcoGorelli for not being a pessimistic Pete but I have no such compunction. We are getting nowhere and designing an API no one will use.  This is a waste of time and we should cut our losses.  Put a bow on what we have now and declare this endeavor a rousing success.

@kkraus14 @jbrockmendel @MarcoGorelli
I want to reiterate the original goals when we started this WG:

Provide developer-facing APIs to standardize among many dataframe library developers
Keep it minimal and simple
Agree on the core parts and do not rathole on the disagreements - document them and provide reasonable expectations and exclusions.

So, IMHO, not the perfect standard is better than no standard. We should not pick on something small, like sort, as there are more significant issues to agree upon.
For Pandas - I think having a standard package -> Pandas namespace -> main namespace path is good enough, as long the transition from the first step to the second is not years. From my team side, I commit to helping build, support, and maintain the Standard API namespace in Pandas.
For other libraries, cuDF, Modin, Vaex, Koalas, etc., targeting performance sort is less critical, but trying to be compatible with Pandas can be a problem, so Pandas as a big bear can keep the current semantics, as there are tons of code already written, meanwhile stating that sort is not compliant, but you can use pandas.standard_df.sort for compliant one.
We can't produce an ideal standard that will satisfy everyone - C++ is a good example. It took implementers ~5 years to accept ARM, ~10 years to produce the first standard from ARM, and another ~10 to bring everyone to the same turf. And there are still incompatibilities. Luckily, we are wiser and less fragmented, so I suggest focusing on pushing the standard out.

FYI @aregm is my boss at Intel.  By complete coincidence, I 100% agree with him.  [An imperfect] standard is better than no standard.  So let's punt on sorting and be done.

Right, let's try to get back on-topic. First, I need to address

I think having a standard package -> Pandas namespace -> main namespace path is good enough, as long the transition from the first step to the second is not years

This isn't my decision, and it's not Brock's either - it's a decision for the pandas community. Unless a compelling case with strong use cases it made, this will get voted down. If I am to try to make such a compelling case, then first I need to be sold on the initiative myself, and that's why I'm asking all these questions.

We should not pick on something small, like sort

I chose the sort argument in groupby as this one was specifically brought up by cuDF as one of their top 3 desired changes in pandas.

Right, back to groupby-sort. Let's say the standard has groupby without a sort argument. As per one of the messages above, pandas implements a superset of the standard, in which groupby has a sort argument. I then have the following outstanding questions:

why would this require a separate package / namespace / option?
how would this help cuDF / modin / any other package?

Maximally explicit answers please, there's already too much confusion and contradictory statements. Thanks 🙏

why would this require a separate package / namespace / option?

A separate package / namespace is a desire but not an explicit requirement. The idea being to ease developer experience in building against the standard. Assuming Pandas is the first / main dataframe development target, without a separate namespace, folks can easily write code that deviates from the standard and loses the dataframe implementation portability they were aiming / looking for.
Alternatively, instead of a separate package / namespace for Pandas, we could imagine having some type of stubs or other validation type of package where someone can use it during their development cycle to ensure they're not deviating from the standard. This stub or validation package could then be a community effort built as part of defining the standard or something like that?
Ultimately, I don't think it's reasonable that developers check every function call and every parameter against documentation to ensure they aren't using Pandas specific features. Without some guard rails in place, I think it will be a very poor developer experience trying to build something that follows the standard without using implementation specific functionality.

how would this help cuDF / modin / any other package?

Assuming that there isn't a separate package / namespace / option / stub / validation, it still defines and documents a standard API and set of behaviors that developers could be pointed to as a way to more or less guarantee portability between Pandas and cuDF, and therefore, CPU and NVIDIA GPU. The common case that folks have talked about is wanting to be able to develop on something like an M1/M2 Macbook which can't have an NVIDIA GPU and then move to a workstation or server with NVIDIA GPU with some level of confidence.

I think having a standard package -> Pandas namespace -> main namespace path is good enough, as long the transition from the first step to the second is not years

This isn't my decision, and it's not Brock's either - it's a decision for the pandas community. Unless a compelling case with strong use cases it made, this will get voted down.

To add to this: if the desired end goal is to also reach this third step (inclusion in the main pandas namespace), I would recommend that we reconsider our current approach for the standard API (design new methods vs define a subset of pandas, this has come up before ..).
Unless the standard API is a subset of the current pandas namespace (plus some (behavioral) changes that we deem important and have a realistic chance of getting support from the pandas community), I think the chance of inclusion in the main namespace (ever) is quite small.
Now, I personally don't have a strong opinion on how important this desire of inclusion in the main namespace actually is, and not going the pandas-subset way gives more freedom to design a better developer-oriented API (eg https://github.com/data-apis/dataframe-api/issues/89).

Note on the actual sort discussion: even if pandas (or the standard api namespace in pandas) would no longer sort for groupby (sort=False), pandas still gives a deterministic order, i.e. the order of appearance in your dataframe. So there is still some form of ordered-ness that people might rely on and that can break when moving to cudf, etc.

I think having a standard package -> Pandas namespace -> main namespace path is good enough, as long the transition from the first step to the second is not years

This isn't my decision, and it's not Brock's either - it's a decision for the pandas community. Unless a compelling case with strong use cases it made, this will get voted down. If I am to try to make such a compelling case, then first I need to be sold on the initiative myself, and that's why I'm asking all these questions.

Some missing context: Areg wants an implementation of the standard to live within a separate namespace within pandas (short-term).  The pandas team thinks this might as well live in a separate package, which wouldn't be their (our) responsibility to maintain.  Areg says "well what if I promise to have Intel maintain it?".  I suggested a compromise that the pandas team might agree to would be a) implement it as a separate package that b) pandas will document as the suggested standard-api-with-pandas then c) demonstrate that it will be maintained and d) if it gets sufficient adoption then e) we will consider upstreaming it to a separate namespace within pandas.
I personally would be OK with this (I think (d) is incredibly unlikely, but if it comes true...) but the pandas team has not agreed to anything.
The "-> main namespace" part is pure fantasy on Areg's part.  Last time he described it to me the path was something like "the standard is so overwhelmingly popular that pandas is compelled adopt it in the main namespace," and I made a joke about that being as likely as me dating Jennifer Lawrence.

The validation package is the idea I like the most out of the ones I've heard so far. The way I'm imagining it, you would run it on your codebase and get some errors like:
t.py:1:4: found DataFrame.groupby without `sort` argument, which is not guaranteed to work the same way across
         DataFrame libraries. Pass `sort=False` to accept this behaviour, or `sort=True` if you need sorted data
t.py:24:154: found DataFrame.sort_index. Note that the Index is not guaranteed to be available in all DataFrame libraries -
        make a column with your current index and sort by that instead
t.py:185:1: found DataFrame.update, which is not guaranteed to be available in all DataFrame libraries

and so on. Once you satisfy all of its pedantic checks, you should get something which is portable. Is this the kind of thing you had in mind Keith?

I believe the wording I used was that if the standard doesn't guarantee and/or document some behavior then it will cause pressure on downstream libraries to just match Pandas semantics regardless of the maintenance burden or performance implications of doing so.
I believe that if we documented the lack of guarantee in the standard that would give something for libraries that can't support certain ordered-ness guarantees something to point to in order to explain and justify their behavior. Especially for distributed implementations, the cost of maintaining ordered-ness can be extremely high and most of the time people aren't dependent on that ordered-ness.

Ok, clear thanks - if all we need is documentation, then we don't need a validation package, right? All you need is to define what's part of the standard, and then document how each package differs from it

I think there's two separate things here: APIs and behaviors.
For APIs, I think a validation package would make a huge difference for developers trying to build against the standard. Otherwise, they'll have to manually go through their code extremely carefully to make sure they didn't deviate from the standard as far as the functions and parameters they use.
For behaviors, I don't think there's any easy ways for us to build tooling to guarantee developers that they're following the behavior guarantees of the standard, where documentation is the best answer that I see that at least gives folks something to guide them.

If the validation package would make a huge difference, then why not take it one step further and also have the validation package require them to be explicit about whether they expect groupby to sort or not?
Documentation could then inform them that sort=True is expensive and might not be what they really want or need

Because, from a purely logical results perspective, there's no difference between doing df.groupby(...).sort_values(...) and df.groupby(..., sort=True), except that every DataFrame that implements the standard will now need to support the sort keyword in their groupby implementation as well.
One argument that I could see is that exposing the sort keyword could allow some libraries to possibly take a more efficient codepath for their actual groupby implementation knowing that they want the output sorted, like using a sort based groupby instead of a hash table based groupby, but I'm not sure if we want things like that in scope for the standard.

It wouldn't require every DataFrame library to support sorting in groupby - the standard could just say that they all need to have sort, and anything other than an explicit sort=False would be flagged by the validation package. DataFrame libraries not wanting to implement sorting in groupby could just raise NotImplementedError for sort=True.
By not saying anything about sort, we risk negatively affecting portability - is it realistic to assume that all users of the standard will read the docs, as opposed to just prototyping with pandas and then declaring themselves to be done when the validation package passes the checks?

It wouldn't require every DataFrame library to support sorting in groupby - the standard could just say that they all need to have sort, and anything other than an explicit sort=False would be flagged by the validation package. DataFrame libraries not wanting to implement sorting in groupby could just raise NotImplementedError for sort=True.

I vaguely remember early discussions we had regarding wanting to avoid "optional" functions and parameters because many of the dataframe library maintainers on the calls felt that "optional" pieces would quickly become effectively required pieces based on other libraries supporting the "optional" pieces.

By not saying anything about sort, we risk negatively affecting portability - is it realistic to assume that all users of the standard will read the docs, as opposed to just prototyping with pandas and then declaring themselves to be done when the validation package passes the checks?

Even if we have the sort parameter, Pandas has behavior of when sort=False that it maintains the input ordering whereas other libraries would presumably not want that restriction. No matter what there's going to be inherent behaviors of Pandas and other libraries that someone could erroneously build against. The best we can do in these situations is to do our best to explicitly document the behavior guarantees and non-guarantees.

Just for my understanding, what's the difference between pandas implementing a superset of the standard and something being optional?

I think my main issue is that I haven't understood whether the objective is:

1:

a clearly documented standard, which would be a subset of pandas;
a validation package which tells developers of when they're using
something which is not part of the standard (e.g. .index) and thus
might not be portable.

or 2:

a clearly documented standard, which some incompatibilities with pandas;
implementations of the standard for each library, possibly in separate
packages / namespaces (in particular, the pandas one would be separate, at least initially).

I've added this to Thursday's agenda

I think the way forward here is to keep the API as it is and ask Keith to make a PR describing sortedness-expectations in the GroupBy class-level docstring https://github.com/data-apis/dataframe-api/blob/main/spec/API_specification/dataframe_api/groupby_object.py#L7

keep the API as it is

Plenty of the API's methods aren't a subset of pandas (e.g. get_column_by_name), so I think that makes it pretty clear that it's option 2 that we're going for.
That's fine, and many of the messages here make it clear that this would not be a way to influence changes in the pandas main namespace. As long as that expectation is clear, and that cuDF / modin / vaex / etc. would still derive value from the exercise, all is good
I'll change my agenda item to focus more on use-cases then - I'd really like it if we could find a strong use-case, try implementing the standard in a separate package (even if in a quick and hacky way for now) and checking that it would allow that use-case to support multiple DataFrame libraries. This would really help with motivation and with determining what to include/exclude

I think my main issue is that I haven't understood whether the objective is:

1:

a clearly documented standard, which would be a subset of pandas;
a validation package which tells developers of when they're using
something which is not part of the standard (e.g. .index) and thus
might not be portable.

or 2:

a clearly documented standard, which some incompatibilities with pandas;
implementations of the standard for each library, possibly in separate
packages / namespaces (in particular, the pandas one would be separate, at least initially).
(there'd be no need for a validation package here because each package, being separate, would implement
only the standard, and so everything should be portable)

I've added this to Thursday's agenda

Some of my thoughts:

I don't have strong opinions about whether the API is a subset of Pandas or some incompatibilities with Pandas as long as we're not locked into Pandas defaults or behaviors.
For option 2: I don't think each package would implement only the standard. I.E. you could imagine a new DataFrame library wanting to use the standard as its primary API and then introduce some domain specific functionality on top of it that doesn't make sense to standardize. In order to allow developers to productively build against the standard, we either need a minimal reference implementation that implements only the standard or a validation package. If we go the minimal reference implementation route, Pandas would probably make the most sense in a separate namespace, but it would obviously depend on what the Pandas community wants to do because it would be a non-trivial amount of work. If the Pandas community understandably doesn't want to take on the effort, then I think a validation package would still be needed.

@kkraus14 would you be up for making a PR documenting the no-sortedness-assumption in the GroupBy docstring?

If we go the minimal reference implementation route, Pandas would probably make the most sense in a separate namespace, but it would obviously depend on what the Pandas community wants to do because it would be a non-trivial amount of work. If the Pandas community understandably doesn't want to take on the effort, then I think a validation package would still be needed.

How about a minimal reference implementation using pandas, but which lives in a separate package and which is maintained by the community?

@kkraus14 would you be up for making a PR documenting the no-sortedness-assumption in the GroupBy docstring?

PR opened here: https://github.com/data-apis/dataframe-api/pull/104

I'll change my agenda item to focus more on use-cases then

I think it is still useful to explicitly confirm with more people that we indeed want to focus on option 2 from https://github.com/data-apis/dataframe-api/issues/102#issuecomment-1448913756, and not on option 1
(because as a starter, I think you can then be sure this will never live in pandas main namespace, which is something some people still see as an end goal)

I don't have strong opinions about whether the API is a subset of Pandas or some incompatibilities with Pandas as long as we're not locked into Pandas defaults or behaviors.

Also if it is a subset, I assume there can be documentation about (non) guarantees for certain behaviours, like the whole discussion in this issue? The current proposal for groupby() is (I think) basically a subset of the current pandas API, and at the same time we want to document that the sorting behaviour of pandas is not part of the guarantees the standard provides.
So I think both options should give that freedom to deviate from certain aspects of pandas (although option 1 of course limits that freedom a bit more).

In order to allow developers to productively build against the standard, we either need a minimal reference implementation that implements only the standard or a validation package. If we go the minimal reference implementation route, Pandas would probably make the most sense in a separate namespace, but it would obviously depend on what the Pandas community wants to do because it would be a non-trivial amount of work. If the Pandas community understandably doesn't want to take on the effort, then I think a validation package would still be needed.

+1. I'll note that, from the lessons we learned from the array API work, this strictly minimal reference implementation is necessary. It can live as a standalone package (we currently have it as numpy.array_api, but it could be split out - doesn't matter too much). As long as it's available and with a clear purpose of validating that you're only using standardized API and that therefore your code is very likely to be portable.

I think it is still useful to explicitly confirm with more people that we indeed want to focus on option 2 from #102 (comment), and not on option 1

That sounds like an important exercise indeed.
I'll note that, depending on the level of incompatibility, a content manager or some such thing may still make sense, to switch deviating APIs to "standard-compliant mode". That may be a usability thing - just like switching imports, it's a toggle to enable the standard. It may just be an easier one.

(because as a starter, I think you can then be sure this will never live in pandas main namespace, which is something some people still see as an end goal)

I will note that the willingness of the pandas team to break backwards compatibility seems to have increased over the past couple of years (probably directly correlated to having more developer bandwidth). Even if a standard will never live in pandas' main namespace, it would be a great outcome if they take away a few lessons from the standard and avoid/change semantics that are bad for parallel/GPU/distributed/JIT-compiled execution.

All, I apologize for not responding earlier to this issue (have been out sick for the last few days - still am).
I am happy that there is alignment on (2) as the end-goal for this standard. I feel that will be a valuable outcome for cuDF.

Awesome, thanks.
For posterity, I'll post one phrase from the last call, which I'd like to make clear the initial goals:

If we manage to develop some standard which allows for fully portable code, people would still consider this to be a success even if it doesn't also make it into the main pandas namespace


Hostname not updating

Hello, it appears when a computer get a hostname change, it does not update the record in MunkiReport. So in the client summary page, the hostname at the top is persisting with the old one, but under the network tab, the hostname is shown properly. See this picture for details.

The name at the top is the computername which may or may not match the hostname.
Sent from my iPhone

On 14 Dec 2017, at 21:38, Daniel Anner<EMAIL_ADDRESS>wrote:
Hello, it appears when a computer get a hostname change, it does not update the record in MunkiReport. So in the client summary page, the hostname at the top is persisting with the old one, but under the network tab, the hostname is shown properly. See this picture for details.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Get logo in navbar and favicon

I am not a Front End Person™, but it works for me.

That is a very large favicon.
Maybe generating a smaller one from the svg with something like http://realfavicongenerator.net/ ?

Does having a big favico cause a problem?  eg does it not scale properly?  I figured detailed icons for hidpi scenarios was the way of the future!

There are six sizes in that favicon.ico.  I tried several svg-to-ico translators, and this is the first that made one that worked at all in my browser (OSX+Chrome).  I couldn't find a simple guide to the favicon.ico format, so I guess it's right?

Merging, but interested in reports of where it goes wrong.  The easiest way to test this is to land it!

👍


PyLance reports possibly missing string when assignement can be statically inferred

Environment data

Language Server version: 2022.7.20 (pyright 7cadfb74)
OS and version: Ubuntu 22.04 LTS
Python version: 3.10.4

Code Snippet
src/test.py:
from typing import TypedDict

class PartialCustomDict(TypedDict, total=False):
    myProp: str

def main():
    pdc: PartialCustomDict = {}
    pdc["myProp"] = "test"
    _ = pdc["myProp"]

if __name__ == "__main__":
    main()

Repro Steps
.vscode/settings.json:
{
  "python.analysis.typeCheckingMode": "strict",
  "python.analysis.logLevel": "Trace",
  "python.linting.flake8Enabled": true,
  "python.linting.enabled": true,
  "python.linting.mypyEnabled": false,
  "python.linting.pylintEnabled": false
}

Expected behavior
No error should be reported attempting to access pdc["myProp"]
Actual behavior
PyLance complains that pyProp might not exist, even if it's clear that it must exist at that point.
Could not access item in TypedDict
  "myProp" is not a required key in "PartialCustomDict", so access may result in runtime exceptionPylance[reportTypedDictNotRequiredAccess](https://github.com/microsoft/pylance-release/blob/main/DIAGNOSTIC_SEVERITY_RULES.md#diagnostic-severity-rules)

Logs

There was an error creating your Issue: body is too long (maximum is 65536 characters).

logs.txt
t

Note that if I initialise the variable with the prop, then it works correctly:
def main():
    pdc: PartialCustomDict = {"myProp": "test2"}
    _ = pdc["myProp"]  # <- No error now

Transferring issue to pyright issue tracker since this is a core type checking issue.


Not working cert

2024-03-17T02:30:19Z    INFO    mutator    transformer "docker.io rewrite rule" skipping rewriting "bitnami/kubectl:1.28.5" to "harbor2.xxxt/dockerhub/bitnami/kubectl:1.28.5", could not fetch image manifest: Get "https://harbor2.xxxx/v2/": tls: failed to verify certificate: x509: certificate signed by unknown authority
enable to make it works, try in 0.7 then rollback to 0.6.3, always the same message.
impossible to have a correct cert. when put checkUpstream to false: crash all container.
my value:
values: |
certManager:
enabled: true
    rules:
    - name: 'docker.io rewrite rule'
      # image refs must match at least one of the rules, and not match any excludes
      matches:
      - '^docker.io'
      excludes:
      # for example, exclude my repo from harbor's proxy cache
      - '^docker.io/fizzbuzz2/*$'
      replace: 'harbor2.home.oursain.net/dockerhub'
      checkUpstream: true
      # - name: 'docker.io ubuntu rewrite rule'
      #   # image refs must match at least one of the rules, and not match any excludes
      #   matches:
      #   - '^docker.io/(library/)?ubuntu:.*$'
      #   replace: 'harbor.home.oursain.net/ubuntu-proxy'
      #   checkUpstream: true # tests if the manifest for the rewritten image exists
      #   platforms: # defaults to linux/amd64, only used if checkUpstream is set
      #   - linux/amd64
      #   - linux/arm64

mybad, is mistake on cert manager


Pick up appsody/liberty 0.5.1 operators from certified-operators

0.5.1 fixes a bug that was found in 0.5.0.  We should try to pick this up for Kabanero 0.9.0 release.

This was delivered in #664


infomation icon overflows out of entry card

Tested on Chromes phone emulator with model iPhone 5

side thing: e- doesn't have a "i" icon cuz test database doesn't have it


BUG: Namadillo - IBC token balance in overview does not update on Shielded Deposit success

This bug was reported on Discord.

on the MASP deposit screen, Namadillo does recognize that i now have 4.97 OSMO, so the
0.03 OSMO tx fee was deducted. the Overview screen still shows 5 OSMO

Screenshots

This is actually correct, we are subtracting the Transaction Fee from the Balance to define the Available amount to be transferred
In the example, if you have 5 OSMO, you can only Shield 4.97 because you will need to pay the fee of 0.03


Initialize Incentive Implementation of "Planting"

Make changes to the landing page, nav-bar, cumulative page.
incentives to keep user engages

Due to time constraints, it has been decided to put this feature on hold.


Provide a token based authentication scheme instead of username/password

The only way to run the heroku gem on the server is to provide your credentials in that environment, which I'd rather not do.
I'd much prefer an auth_token that I can use (and regenerate at any time from the heroku site).
Any reason why this hasn't been done yet?

This idea, or something like it, is definitely on our radar. While I can't provide a specific ETA, we agree, and will have a solution for this.


nixpkgs manual: add section on submitting security fixes

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @ckauhaus

[ ] How about version numbers prefixed to CVE? https://github.com/NixOS/nixpkgs/pull/72076
[ ] How about patches that partially resolve a CVE?

Another open question:

[ ] how to name a single patch fixing multiple CVEs ? [A] or [B] or [C]

About version number prefixing the patch name -> I think it should be avoided, as the CVE description (from NVD or other sources) will state which versions are vulnerable and patched, and it is usually a range.

I suppose the version number was typically added when there are multiple versions of a package, and each version needs a patch for the CVE but a different one.

Going ahead and will open a separate issue with these questions.

questions are now in https://github.com/NixOS/nixpkgs/issues/73342


Using match on all items in an array?

Hello,
I'm trying to filter this data:
https://raw.githubusercontent.com/aEnigmatic/ffbe/master/skills_ability.json
And at first i tried with this function:
=IMPORTJSONAPI("https://raw.githubusercontent.com/aEnigmatic/ffbe/master/skills_ability.json","$.*.[?(@property==='name' && @.match('Gained'))]^","name,effects[0],effects[1],effects[2],effects[3],effects[4],effects[5]")
Which works well, but how would i go about filter in the effects column? I want to find results containing "9999" in any of the effects columns? I tried
"$.*.[?(@property==='effects' && @.match('9999'))]^"
"$.*.[?(@property==='effects[0]' && @.match('9999'))]^"
"$.*.[?(@property==='effects[*]' && @.match('9999'))]^"
To no avail, and then i ran out of ideas... is it possible to do what i want?

Is this any good?
=IMPORTJSONAPI("https://raw.githubusercontent.com/aEnigmatic/ffbe/master/skills_ability.json","$.*.effects.*[?(@.match('9999'))]", "^^^name, @")

Thank you very much! That worked perfect!


Optimize unfold/unfoldLeft for Lists

At the moment the List::unfold and List::unfoldLeft are implemented as Iteartor.unfoldLeft(...).toList() which works but perhaps a better implementation would be
return Iterator.unfoldRight(seed, f.andThen(tupleOpt -> tupleOpt.map(Tuple2::swap)))
               .foldLeft(List.empty(), List::prepend);

This builds the result without any additional reverse() operations and Streams in a single for-loop.

Hi, improving the implementations is always welcome! Are you volunteering for a PR?


Need some help with the config.txt part of Distributed Approach

I have the config.txt file as
`
server_num

<IP_ADDRESS> 8080 /home/a/castle/image

<IP_ADDRESS> 8080 /home/b/castle/image

`

the <IP_ADDRESS> is the IPv4 address,  8080 is the default port run by the application, last part is the image path.
Questions:
What is server_num?
Does the image path have to contain images or it can be empty folder?
Does the image path and the output path of the worker have to be the same?
Does the server or master IPv4 address have to be included in the config.txt?

server_num is the number of your servers. Since you have two IPs, the server_num is simply 2.
The image path of worker does not need to contain images. It's used to store images that master transfered to server.
Absolutely.
Yes. Master should know the ip address of all workers, so it can distribute tasks to workers. Currently, I do not find a proper way to avoid setting the ip address.

I am wondering what you use for the IP address. what linux command did you use to get the IP address?
I think port numbers are default to 8080 in the code
.sh file:
`
/home/j/GraphSfM/build/src/exe/colmap distributed_mapper
$DATASET_PATH/$log_folder
--database_path=$DATASET_PATH/database.db
--transfer_images_to_server=1
--image_path=$DATASET_PATH/images
--output_path=$DATASET_PATH/$log_folder
--config_file_name=/home/j/GraphSfM/scripts/shell/config.txt
--num_workers=8
--distributed=1
--repartition=0
--assign_cluster_id=1
--write_binary=1
--retriangulate=0
--final_ba=1
--select_tracks_for_bundle_adjustment=1
--long_track_length_threshold=10
--num_images_ub=$num_images_ub
--completeness_ratio=0.7
--relax_ratio=1.3
--cluster_type=NCUT # SPECTRA
--image_overlap=$image_overlap \
`
config.txt file
2
<IP_ADDRESS> 8080 /home/jccscs/GraphSfM/build/src/exe/output
<IP_ADDRESS> 8080 /home/j/gerrad-hall

Command ran ./distributed-mapper.sh /home/j/gerrad-hall  100 log /home/j/GraphSfM/scripts/shell

I am wondering what you use for the IP address. what linux command did you use to get the IP address?
 Virtual box is on another device while computer is on Linux.

@jchen706 , hello, I met the same problem. How did you solve it? Thank you!


Create ChemCrow/Langchain plugin for Semantic Scholar

Semantic Scholar Homepage
Semantic Scholar API Documentation

ToxPipe is querying Semantic Scholar via Scholar2ResultsLLM tool and paperscrapper, thanks to @vestalisvirginis.


Release 2.15.1

Closes n/a
Code Changes

[x] Pull in #3669
[x] Pull in #3713

Steps to Confirm

[ ] list any manual steps for reviewers to confirm the changes

Pre-Merge Checklist

[ ] All CI Pipelines Succeeded
Documentation:

[ ] documentation complete, PR opened in fidesdocs
[ ] documentation issue created in fidesdocs

[ ] Issue Requirements are Met
[ ] Relevant Follow-Up Issues Created
[ ] Update CHANGELOG.md
[ ] For API changes, the Postman collection has been updated

Description Of Changes
Incorporate bug and security fixes

deleting branch and PR due to git issues


conditional_probability_alive in ModifiedBetaGeoFitter and BetaGeoFitter

It's strange but there are different code realization for conditional_probability_alive for ModifiedBetaGeoFitter and BetaGeoFitter with the same link on Computing P(alive) Using the BG/NBD Model:
class BetaGeoFitter(BaseFitter):

    def conditional_probability_alive(self, frequency, recency, T):
        """
        Compute the probability that a customer with history (frequency, recency, T) is currently
        alive. From http://www.brucehardie.com/notes/021/palive_for_BGNBD.pdf

        Parameters:
            frequency: a scalar: historical frequency of customer.
            recency: a scalar: historical recency of customer.
            T: a scalar: age of the customer.

        Returns: a scalar

        """
        r, alpha, a, b = self._unload_params('r', 'alpha', 'a', 'b')
        return 1. / (1 + (frequency > 0) * (a / (b + frequency - 1)) * ((alpha + T) / (alpha + recency)) ** (r + frequency))

class ModifiedBetaGeoFitter(BetaGeoFitter):

    def conditional_probability_alive(self, frequency, recency, T):
        """
        Compute the probability that a customer with history (frequency, recency, T) is currently
        alive. From http://www.brucehardie.com/notes/021/palive_for_BGNBD.pdf

        Parameters:
            frequency: a scalar: historical frequency of customer.
            recency: a scalar: historical recency of customer.
            T: a scalar: age of the customer.

        Returns: a scalar

        """
        r, alpha, a, b = self._unload_params('r', 'alpha', 'a', 'b')
        return 1. / (1 + (a / (b + frequency)) * ((alpha + T) / (alpha + recency)) ** (r + frequency))

conditional_probability_alive BetaGeoFitter
conditional_probability_alive ModifiedBetaGeoFitter

I think for BetaGeoFitter it's a right equation according to Computing P(alive) Using the BG/NBD Model.
However for MBG/NBD may be not as I understand from  original paper Empirical validation and comparison of models for customer base analysis.
Equation for probability alive/active is in Appendix A, number (5).

Hm, it looks different, but the gamma ratios will simplify to the correct values - am I missing something?

Yes, you're right. Probably just need to delete 'http://www.brucehardie.com/notes/021/palive_for_BGNBD.pdf'  from description for conditional_probability_alive for MBG/NBD


Digit matching behaving as intended?

Reposted from the Swift forums: https://forums.swift.org/t/bad-digit-matching-bugreport-regarding-se-0354-regex-literals/57262/1
Problem: Some digit character groups match number-like grapheme clusters.
// this matches:
try /[1-2]/.wholeMatch(in: "1⃣")
// still matches:
try /[1-2]/.asciiOnlyDigits().wholeMatch(in: "1⃣")
// does not match:
try /[12]/.wholeMatch(in: "1⃣")
Above described behavior seems inconsistent and difficult to predict. Shouldn't [1-2] and [12] be identical? Should they match anything outside of ascii?
Note: :one: is U+0031 (ascii digit 1) U+FE0F (VARIATION SELECTOR-16) U+20E3 (COMBINING ENCLOSING KEYCAP)
Same is true for 1⃣: U+0031 (ascii digit 1) U+FE0E (VARIATION SELECTOR-15) U+20E3 (COMBINING ENCLOSING KEYCAP)

If there are questions about my report: I'm following this issue.

Note that the emoji compares between "1" and "2" as a Character. @natecook1000

Note that the emoji compares between "1" and "2" as a Character. @natecook1000

This explains a lot of above behavior—at least technically. On the other hand it scares me away from using regular expressions when writing code that should sanitize arbitrary input.
Maybe an ascii-only mode would help? In my code I'm manually checking input now:
input.utf8.allSatisfy { $0 <= 127 }.

Thanks for the example, @NikolaiRuhe! It looks like custom character class ranges will need something a bit different than Character.<, at least by default.

Note that in scalar semantics you should get your desired behavior. It's a reasonable interpretation to extend A-Z to "A"..."Z", but probably undesirable and counter intuitive. The Unicode pitch was arguing for using NFD for ranges, but that wouldn't affect this example.


Change default keyboard type

Also remove autocorrection on the keyboard.
Although the developer can change the keyboard type, I believe this is a more appropriate keyboard for OTPs which do not have options for - and Space

Makes great sense, good job!


self_link is removed in favor of id.

Resolves #2
A side-note: after this change, I was able to spin up a Vault successfully, while ran into 2 hiccups:

The NS record change propagation took more than 18 hours to take effect, would be better to note it in the documentation.
The Vault UI worked as a charm after unsealing but the Vault CLI failed to connect to the Vault server with an error :

$ export VAULT_ADDR=https://vault.mydomain.com:8200
$ vault status
Error checking seal status: Get "https://vault.mydomain.com:8200/v1/sys/seal-status": x509: certificate signed by unknown authority

I'm still looking into it and suspecting if that's related to the CA of my certs. (wouldn't it be easier to update the Terraform to use GKE managed TLS in this case?)
Anyways, looks like this PR shall fix the aforementioned issue.

Sorry for the bump :P @gatsbysghost

Hey there! Sorry for letting this go for so long--I really let a lot of my responsibilities go while I was in the EU these last two weeks! I am testing this right now, will give this PR a Yea/Nay today :)

This change shall be merged!


Major Desync Issues When Changing Time, or Random Events

Desync-04.zip
Desync-03.zip
Desync-02.zip
Using the Arbiter including one other person, and it seems we either desync after a random event, or when speeding up game time (Might be completely unrelated).
Windows 10 (64-bit)
If need be, I can post mod list, and load order

Playing almost vanilla to try to debug what is causing desyncs and it seems like even with only color moods running it is the changing of speed that causes desync. Have been playing at 1x for 30 mins with no problems.


Add logged in user, logout link

Required returning a Promise from urlp.loadApp, since we're fetching the user ID asynchronously. If network overhead is ever an issue, we can consider rendering index.html as a template, or sending back the logged-in user in an HTTP header.
Also, while experimenting with new end-to-end test cases, I realized that calling driver.close() would actually end the Chrome session, so the second commit removes that call from afterEach.

Coverage increased (+0.6%) to 98.42% when pulling aa6d1ad3acea811d16bf1bed5fda2f0570b69103 on login-logout into e66ca601da53c37db76c92bf629db9ae3f1eb44e on master.

Coverage increased (+0.03%) to 97.823% when pulling 062a6d2e6e59ae241f731b1adc0caf77a434c47c on login-logout into e66ca601da53c37db76c92bf629db9ae3f1eb44e on master.

Coverage increased (+0.03%) to 97.823% when pulling 062a6d2e6e59ae241f731b1adc0caf77a434c47c on login-logout into e66ca601da53c37db76c92bf629db9ae3f1eb44e on master.

Not really sure why it's reporting less coverage. Merging.


Issue With TA-LIB indicators

When I enable the Talib indicators in my strategy they tell me to install them (Error:
The plugin tradingAdvisor expects the module talib to be installed. However it is not, install it by running:
    npm install<EMAIL_ADDRESS>
After I do so this is what i get
C:\Users\xxxx\Desktop\BOTS\gekko-stable>npm install<EMAIL_ADDRESS>npm ERR! Windows_NT 10.0.15063
npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "install"<EMAIL_ADDRESS>npm ERR! node v7.9.0
npm ERR! npm  v4.2.0
npm ERR! code EBADPLATFORM
So can't we use them with windows 10 ?
npm ERR! notsup Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin,linux","arch":"any"} (current: {"os":"win32","arch":"x64"})
npm ERR! notsup Valid OS:    darwin,linux
npm ERR! notsup Valid Arch:  any
npm ERR! notsup Actual OS:   win32
npm ERR! notsup Actual Arch: x64
npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\xxxx\AppData\Roaming\npm-cache_logs\2017-05-04T01_17_35_913Z-debug.log

It compiles fine for me on windows 10 using bash
also npm install talib is sufficient you do not need the version number

Okay i'm going to try this and give you results

Still nothing it keeps saying the same error

Unfortunately TAlib is not really supported under windows, see this issue from the library Gekko uses for TAlib integration: https://github.com/oransel/node-talib/issues/8
There is nothing we can do about it at this point. There is a discussion going about using another TA library here: https://github.com/askmike/gekko/issues/708 But that is early stage and will take a while.

I suggest running linux (in a virtualbox or something if you really need talib).

You can run gekko with TA-lib on Bash on Windows 10.
Step 1: Open up bash and install node.js: (taken from https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions)
sudo apt-get update
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo apt-get install -y build-essential

Step 2: Install Git
sudo apt-get install git
Step 3: Download and install gekko: (taken from https://gekko.wizb.it/docs/installation/installing_gekko.html):
git clone git://github.com/askmike/gekko.git
cd gekko
npm install --only=production

Step 4: Edit the handle.js file (taken from https://github.com/askmike/gekko/issues/708#issuecomment-296329300)
Navigate to gekko/plugins/sqlite:
cd plugins/sqlite
Change line 53 in handle.js from
db.run("PRAGMA journal_mode = WAL");
to
db.run("PRAGMA journal_mode = DEL"); :
(I use nano for this --> nano handle.js)
Step 5: Start gekko:
node gekko --ui
Everything should work then!

Great instructions Deke, I reckon this is worth including in the Gekko documentation.

@thegamecat @askmike I created a PR to update the docs with this info.


Add Kingdraw

Add Kingdraw
Microsoft Reviewers: Open in CodeFlow

https://www.microsoft.com/en-us/wdsi/submission/a3c88c62-f540-4d4a-b89f-e1f42c318754

I got this through the smartscreen errors, but I am not seeing any files get installed with this app.

The default installation destination is "D:\KingDraw". Do you have D:? Or can I use command line arguments to change the installation destination? The installer type is nullsoft.

@oxygen-dioxide this is an older PR :smile:
Has anything changed with this package?
Do you want to convert the manifest and trigger another build or close this?


dashboard/app: better health monitoring

We have lots of health indicators that can be evaluated only over a time period, for example:

recent supposed drop in reproducers (#5426)
rate of crashes we fail to save and lost
rate of dashboard app errors
rate of bisection/patch testing errors
rate of syz-manager crashes
top crasher fraction
drops in coverage
and more

All of these can't be diagnosed at the instant (a single dashboard error, or a single repro failure may be ignored), and currently we don't do any monitoring for any of these (besides random wandering around).
We should collect data for these and at least visualize (e.g. rate of successful/failed bug reproductions, dashboard errors per day), and ideally maybe alert on sudden changes. Some alerts may be based on threshold (easier, e.g. >100 dashboard errors/day).

GCP offers the alerting functionality: https://cloud.google.com/monitoring/alerts (also based on the logs https://cloud.google.com/logging/docs/alerting/log-based-alerts)
It would be nice to figure out how to keep these settings in the git repository and be able to (re-)deploy them without having to go through the Cloud web UI interface.

If we want to reply on logs grepping, I think for reliability of parsing we will need a new interface along the lines of:
package log
func Metricf(typ Metric, description string, args ...any)
type Metric string

Otherwise matching logs is unreliable.
And, yes, it would be good to persist rules somewhere.


xming

PS C:\> winget install xming
Found Xming [colinharrison.xming]
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://ayera.dl.sourceforge.net/project/xming/Xming/<IP_ADDRESS>/Xming-6-9-0-31-setup.exe
An unexpected error occurred while executing the command:
InternetOpenUrl() failed.
0x80072efd : unknown error
I don't know if this is a valid issue, please disregard if it is.

The xming package download link has been updated via #5793. This issue and its duplicate #5770 can be closed once the package installation has been verified to work with the new download URL (when the changes have been replicated to the winget package index).

You all rock

You all rock


Small fixes to file header

I'd like to use this in a project, but some of the parts are outdated, so I figured I'd suggest an update for you.

Thank you @rpavlik


builder pattern

Progress:

[x] tests/01-parse.rs
[x] tests/02-create-builder.rs
[x] tests/03-call-setters.rs
[x] tests/04-call-build.rs
[x] tests/05-method-chaining.rs
[x] tests/06-optional-field.rs
[ ] tests/07-repeated-field.rs
[ ] tests/08-unrecognized-attribute.rs
[ ] tests/09-redefined-prelude-types.rs

whoops! sorry about that was just trying to make a pull request to my fork


Allowempty permits non-map values

Observed:
Providing 'allowempty' with type: 'map' validates non-map values.
Expected:
Based on documentation:

If True, the map can have keys which are not present in the schema, and these can map to anything.
Any keys which are specified in the schema must have values which conform to their corresponding constraints, if
they are present.

I would expect 'allowempty' to only permit:

an empty map
map with keys not in the schema

Otherwise, I would expect a NotMappingError
Example:
In [1]: pykwalify.core.Core(
    source_data=['not', 'a', 'map'],
    schema_data={'type': 'map', 'allowempty': True}
).validate(raise_exception=True)
Out[1]: ['not', 'a', 'map']

In [2]: pykwalify.core.Core(
    source_data=['not', 'maps'],
    schema_data={
        'type': 'seq',
        'sequence': [{
            'type': 'map',
            'mapping': {'a': {'type': 'str'}},
            'allowempty': True
        }]
    }
).validate(raise_exception=True)
Out[2]: ['not', 'maps']

Not very familiar with the code at all, but at a brief glance,
I'm wondering if this happens because _validate_mapping returns too eagerly in the case of allowempty. (https://github.com/Grokzen/pykwalify/blob/b064c609290866310f6ecc8cae42f3e7b4b81a67/pykwalify/core.py#L436 - commit sha here is to make sure I have the right line - not the commit that introduced problems)
I'd expect some sort of check that the value is a mapping at all by the time it gets to validate_mapping.

@kesre This fix for this was commited in this commit 8fa18e3add993db1c821c76d01fa60a93e358e1c and it fixed the problem. The correct way to deal with this was to move the type checking earlier in the mapping validation so that even if you do not speciefy the mapping keyword and the value type is wrong you should get a error that the value type is bad.
Just to clearify that the keyword allowempty have no impact on this. You really need to have the correct value type before it will be applied or even considered. The correct solution was to give a type error first.
Please open a new issue if you find any other issues with the keyword allowempty


Wrapper generator

Add support for namespace files, same format as Libamtrack's NAMESPACE
Refactor wrapper generation in order to make creating different kind of wrappers easier

@ptyszko please fix failing test