rust-analyzer failed to load workspace .cargo\bin (os error 5)

I just tried installing the extension from the vs-code marketplace but got the error as a popup
rust-analyzer failed to load workspace: "...cargo\\bin" "--version" failed: Access Is Denied (os error 5)

At the same time the following error got printed to the console
ERROR [7.9.2021, 16:09:46]: Cannot rename existing server instance: EntryNotFound (FileSystemError) (FileSystemError): Error: ENOENT: no such file or directory, rename 'c:\Users\tobia\AppData\Roaming\Code\User\globalStorage\matklad.rust-analyzer\rust-analyzer-x86_64-pc-windows-msvc.exe' -> 'c:\Users\tobia\AppData\Roaming\Code\User\globalStorage\matklad.rust-analyzer\rust-analyzer-x86_64-pc-windows-msvc-stale-5d728e5b2a.exe'
INFO [7.9.2021, 16:09:46]: Using server binary at c:\Users\tobia\AppData\Roaming\Code\User\globalStorage\matklad.rust-analyzer\rust-analyzer-x86_64-pc-windows-msvc.exe

I tried deleting the folder matklad.rust-analyzer but it didnt work

Do you have any RA settings or environment variables? The "Cannot rename" error is not an issue, but the cargo one is.
We run rustc --version (and maybe cargo --version), though, and on your system it's trying to run a directory.

This code path might be wrong: https://github.com/rust-analyzer/rust-analyzer/blob/19348bc/editors/code/src/toolchain.ts#L158-L166

I dont think i have anything set globally that could block anything. Let me know if you want any additional information. Glad that you were so quick to respond. appreciate it

log.error("Failed to read the fs info", err); doesnt get printed to the terminal though. but i dont know how you handle errors. That might not mean anything

I think it goes into the Output window, in the Rust Analyzer Extension channel (there's a picker in the top-right).

This is the current output for an older version I just installed, the same problem still applies
INFO [7.9.2021, 16:31:33]: Extension version: 0.2.694
INFO [7.9.2021, 16:31:33]: Using configuration {
  cargoRunner: null,
  runnableEnv: null,
  inlayHints: {
    enable: true,
    smallerHints: true,
    chainingHints: true,
    maxLength: 25,
    parameterHints: true,
    typeHints: true
  },
  updates: { channel: 'stable', askBeforeDownload: false },
  server: { path: null, extraEnv: null },
  trace: { server: 'off', extension: false },
  debug: {
    engine: 'auto',
    sourceFileMap: {
      '/rustc/<id>': '${env:USERPROFILE}/.rustup/toolchains/<toolchain-id>/lib/rustlib/src/rust'
    },
    openDebugPane: false,
    engineSettings: {}
  },
  assist: {
    importGranularity: 'crate',
    importEnforceGranularity: false,
    importPrefix: 'plain',
    importGroup: true,
    allowMergingIntoGlobImports: true
  },
  callInfo: { full: true },
  cargo: {
    autoreload: true,
    allFeatures: false,
    unsetTest: [ 'core' ],
    features: [],
    runBuildScripts: true,
    useRustcWrapperForBuildScripts: true,
    noDefaultFeatures: false,
    target: null,
    noSysroot: false
  },
  checkOnSave: {
    enable: true,
    allFeatures: null,
    allTargets: true,
    command: 'check',
    noDefaultFeatures: null,
    target: null,
    extraArgs: [],
    features: null,
    overrideCommand: null
  },
  completion: {
    addCallArgumentSnippets: true,
    addCallParenthesis: true,
    postfix: { enable: true },
    autoimport: { enable: true },
    autoself: { enable: true }
  },
  diagnostics: {
    enable: true,
    enableExperimental: true,
    disabled: [],
    remapPrefix: {},
    warningsAsHint: [],
    warningsAsInfo: []
  },
  experimental: { procAttrMacros: false },
  files: { watcher: 'client', excludeDirs: [] },
  highlightRelated: {
    references: true,
    exitPoints: true,
    breakPoints: true,
    yieldPoints: true
  },
  highlighting: { strings: true },
  hover: { documentation: true, linksInHover: true },
  hoverActions: {
    debug: true,
    enable: true,
    gotoTypeDef: true,
    implementations: true,
    references: false,
    run: true
  },
  joinLines: {
    joinElseIf: true,
    removeTrailingComma: true,
    unwrapTrivialBlock: true
  },
  lens: {
    debug: true,
    enable: true,
    implementations: true,
    run: true,
    methodReferences: false,
    references: false,
    forceCustomCommands: true
  },
  linkedProjects: [],
  lruCapacity: null,
  notifications: { cargoTomlNotFound: true },
  procMacro: { enable: true, server: null },
  runnables: { overrideCargo: null, cargoExtraArgs: [] },
  rustcSource: null,
  rustfmt: {
    extraArgs: [],
    overrideCommand: null,
    enableRangeFormatting: false
  },
  workspace: { symbol: { search: { scope: 'workspace', kind: 'only_types' } } }
}
INFO [7.9.2021, 16:31:33]: PersistentState: {
  lastCheck: undefined,
  nightlyReleaseId: undefined,
  serverVersion: '0.2.727'
}
INFO [7.9.2021, 16:32:20]: Renamed old server binary c:\Users\tobia\AppData\Roaming\Code\User\globalStorage\matklad.rust-analyzer\rust-analyzer-x86_64-pc-windows-msvc.exe to c:\Users\tobia\AppData\Roaming\Code\User\globalStorage\matklad.rust-analyzer\rust-analyzer-x86_64-pc-windows-msvc-stale-cedd319f1d.exe
INFO [7.9.2021, 16:32:21]: Removed old server binary c:\Users\tobia\AppData\Roaming\Code\User\globalStorage\matklad.rust-analyzer\rust-analyzer-x86_64-pc-windows-msvc-stale-cedd319f1d.exe
INFO [7.9.2021, 16:32:21]: Using server binary at c:\Users\tobia\AppData\Roaming\Code\User\globalStorage\matklad.rust-analyzer\rust-analyzer-x86_64-pc-windows-msvc.exe

Oops, sorry, that's a server error message. The problem is not in the extension.
We run cargo --version here and use this to find the executable.

I dont know what you mean by environment variable. Adding it to the path? It is added to the path and by running cargo --version i get cargo 1.54.0 (5ae8d74b3 2021-06-22) which should be the latest, as I installed rust very recently.

What does echo %CARGO% (or echo $Env:CARGO if you use powershell) show? It should either be nothing or the path to cargo.exe.

shows C:\Users\tobia\.cargo\bin

Open https://www.architectryan.com/2018/08/31/how-to-change-environment-variables-on-windows-10/, look for a CARGO variable and remove it.

okay! awesome! that fixed it. I believe there must be some reason i have added it to the path, but everything seems to be running still, so I guess its fine. I appreciate your quick help!!

Yeah, I think some tutorials tell you to set CARGO_HOME (though it shouldn't be needed AFAIK), and maybe that morphed into CARGO at some point.


Replace use of list for cached_data and subject_ids in pytorch dataset with polars objects

This fixes the increased memory consumption issues when using multiple pytorch dataloaders (issue #73). It also dropped the starting memory usage in my test case from 30GB to 12GB.
Removing this line makes all the difference: https://github.com/mmcdermott/EventStreamGPT/blob/b10e7415af1e9ea9517dfb52c343ae8155c40674/EventStream/data/pytorch_dataset.py#L309
Editing the following line didn't make much of a difference, but I edited it for consistency:
https://github.com/mmcdermott/EventStreamGPT/blob/b10e7415af1e9ea9517dfb52c343ae8155c40674/EventStream/data/pytorch_dataset.py#L306

Hey @juancq -- how does this impact the iteration speed through the dataloader, though? The motivation to convert things to lists was that with raw polars objects, the base iteration speed was much slower.

@mmcdermott I saw no noticeable difference in the iteration speed.

@juancq I'm working on a different solution for this problem that also addresses some other issues. I'll tag you in that other PR. It's not 100% ready but it is close. It is a larger change, but I'll explain more there.


Disable Presto health checks on debug

Disable Presto health checks on debug

AC

Failed with: https://github.com/prestosql/presto/issues/5422 and https://github.com/prestosql/presto/issues/5589


inventory & multiple-pymethods

The inventory crate has been archived. This brings into question the future of the multiple-pymethods feature, which currently depends on inventory.
As far as I know, the crate was archived because it relies on platform-dependent linker behaviour, and also has unstable behaviour on newer Rust versions (https://github.com/dtolnay/inventory/issues/32).
Deprecating this feature is the simplest path, however it is also a decent ergonomic improvement. In my day job I maintain a PyO3 codebase which exports a large number of #[pyclass] structs. To avoid repetition we have macros to write standard magic methods, e.g.
#[pyclass]
struct Foo { }

// Methods common to many types
basic_magic_methods!(Foo);
getstate_setstate!(Foo);

// Methods unique to this class
#[pymethods]
impl Foo {
    // ...
}

These helper macros basically extend to additional #[pymethods] blocks. Successfully migrating this codebase away from multiple-pymethods would require coming up with a suitable alternative pattern.

As an alternative to deprecating the feature, I've been wondering whether to experiment with bringing a minimalist in-crate implementation of linkme - the #[distributed_slice] concept should be enough to build #[pymethods] here.
In the long-term, it would be interesting to see if we can garner support for such a primitive in the language itself. See https://internals.rust-lang.org/t/from-life-before-main-to-common-life-in-main/16006
If a #[distributed_slice] implementation was landed in nightly Rust, we could then switch the multiple-pymethods feature to be nightly-only, with the hope that it would one day be available on stable.

Does linkme not have similar problems to inventory? It is also archived.

If a #[distributed_slice] implementation was landed in nightly Rust, we could then switch the multiple-pymethods feature to be nightly-only, with the hope that it would one day be available on stable.

I think that would be the best long-term solution. Especially since the underlying issue not specific to PyO3.

Does linkme not have similar problems to inventory? It is also archived.

It does, however not fully understanding the exact issue that's causing the problem (and the fact that I've not seen any reports on PyO3 of multiple-pymethods not working), I am unsure whether the issues which plague the general solution affect the specific use case here.

To avoid repetition we have macros to write standard magic methods, e.g.

Not as well-organized as the original, but this is viable too:
// Methods unique to this class
#[pymethods]
impl Foo {
    // Methods common to many types
    basic_magic_methods!();
    getstate_setstate!();

    // ...
}

It does, however not fully understanding the exact issue that's causing the problem

See https://github.com/rust-lang/rust/issues/47384:

Statics that are marked no_mangle and/or used only get to the linker if they are in a reachable module of a reachable crate. The static itself does not need to be used in code, only some function in the same module.

So, probably that's why you have no errors (yet?) because your #[pymethods] are reachable?

To avoid repetition we have macros to write standard magic methods, e.g.

Not as well-organized as the original, but this is viable too:
// Methods unique to this class
#[pymethods]
impl Foo {
    // Methods common to many types
    basic_magic_methods!();
    getstate_setstate!();

    // ...
}

I don't think this works - the #[pymethods] macro will not see the result of expanding basic_magic_methods! etc. So the methods included in that macro will not be exposed to Python.

I don't think this works - the #[pymethods] macro will not see the result of expanding basic_magic_methods!

This makes me curious: what's the priority of procedural macros and macro_rules?

AFAIK it's always outermost macro gets expanded first, then inner ones.

Similar to @davidhewitt's case, we use macros to generate python methods. The macros mimic Rust traits, that are defined in a core crate and are implemented in more specific crates. In every specific crate Python classes are defined, that call the macro to automatically implement the functionalities given by the Rust traits. This structure relies heavily on the multiple-pymethods feature. Therefore, deprecating the feature would have quite a detrimental effect on our crates.

Interested to see how you solve this as I'm both a user of pyo3 and inventory ...

I haven't looked at how multiple-pymethods is implemented, but #[distributed_slice] would be enough to implement the inspection / pyi generation for multiple #[pymethods] block.

It's live again:
https://github.com/dtolnay/inventory/releases/tag/0.3.0

For Rust 1.62+, though. But it's good news nonetheless!

Yep. Since Debian 12 may freeze in 1/2023, we have a good chance of a new enough toolchain making it in there.

Given that there's a possible soundness concern, I'd argue that in PyO3 0.17 we should actually update inventory to 0.3 and so require that multiple-pymethods uses Rust 1.62.

Fair enough.


Master is not merged into the companion pr before building

The old script merged the latest master into the companion branch (but only locally in the pipeline) before running the tests. We still merge substrate master into the pr branch (also locally in the pipeline) and this can break the compilation.

Do I get it right: the problem is that substrate's PR merges master during the companion job and others don't?

Yes

Sorry, I'm still not following the issue here...Are we saying we no longer want to merge master into the companion branch entirely?
Is this just for substrate PR merges or all merges?

No, we currently don't merge master into the companion branch. This is a problem.
So, for example, we are checking out the polkadot branch and should polkadot master into this branch.


use pytest-virtualenv to install dvc into

had success with pytest-venv too, but didn't submit yet (instead just using venv CLI in run.sh), because it would be nice to think ahead about api tests and how they would be configurable.

I guess we'll just get back to this once we start messing with API. closing for now


The current slider doesn't show the values, this is confusing

In the cms a moderator can add a slider with a minimum value, maximum value and default value.
These boundaries are not shown in the layout, so the user doesn't know what value he/she is sliding

Opgelost.


have ability to ask follow-up questions that remember context of previous result

This would be a very useful feature to have.  I have noticed that with further testing (at least using gpt3.5), asking follow-up queries, it will not remember the context of the previous query, and instead give an "i dont know".
For example, I ask "what is the the ghg footprint of yellow pea protein?"
and it will give an answer like "15.5kg".
Then I ask a follow-up question, "what about green pea protein?"
In normal chatgpt, and other llms, it would normally remember the context of the previous answer it gave, and follow up with the ghg footprint of green pea protein.  But with this application, it will instead say "i don't know" to this follow-up question.
Is this something that can be easily added?  It may be something to do with langchain and how we query the llm.  Not really sure.
If it's something you can look into, that would be a very useful enhancement.
Thank you!

Thanks for the suggestion.
In the chat interaction, the limited context window of the model is filled with previous fragments of conversations, creating the illusion that the chatbot has a memory. In reality, you need to send the entire conversation back and forth.
RAG is different from regular chat. All the context window memory of the model is used to fill it with document extracts to ground the answer, rather than relying on previous conversations.
I am afraid it is not feasible with the current context window lengths, unless you are using expensive, extremely long context window models. While it will remember some of the previous requests, it will have less factual data to provide the answer.
It might become possible in the future as the context length of the models increases.

thank you for the response.  Not much we can do about this at this time then.  It's not even a big deal.  It's a nice to have, but not really something that can't be mitigated with prompting.
I'll close this for now.


Rule AA0137 (Unused variable) not working for global variables defined on the same line separated with comma

Describe the bug
Rule AA0137 "Variable * is unused in *" does not work for global variables in an object that are defined on the same line separated by a comma (i.e., GlobalUnusedVariable, GlobalAnotherUnusedVariable : Boolean;). For local variables this works fine.
To Reproduce
Example (see screenshot):
pageextension 50100 CustomerListExt extends "Customer List"
{
    trigger OnOpenPage();
    var
        UnusedVariable: Boolean;
        AnotherUnusedVariable, YetAnotherUnusedVariable : Boolean;
    begin
        Message('App published: Hello world');
    end;

    var
        GlobalUnusedVariable: Boolean;
        GlobalAnotherUnusedVariable, GlobalYetAnotherUnusedVariable : Boolean;
}

Expected behavior
Warnings for unused global variables that are defined like this.
Screenshots

5. Versions:

AL Language: 6.1.381323

Business Central: Version: W1 17.2 (Platform 17.0.19353.19730 + Application 17.2.19367.19735)

In the latest AL version 9.0.615906, it shows AA0205 instead ... ?

Thanks for reporting this issue. Sorry we haven’t completed it yet, but we’ve had to prioritize elsewhere. We’re planning to give the CodeCop engine and its rules an overhaul in a future major release. Thanks for your patience.
Going forward we will not track progress on CodeCop issues here on GitHub. Hence, we are closing this issue.

Still not fixed for RunTime 11.0.x (Just for Reference, if someone searches for this issue in the future)


Add comparison post between Pulumi and Terraform

@TechWatching just saw this pass by on Twitter. Nice article! Tnx.


Docs minor updates to letters, self-mailers

Fixes #<INSERT ISSUE NUMBER HERE>
Checklist

[X] Up to date with main
[X] All the tests are passing (All except

[ ] Delete all resources created in tests

[X] Prettier
[X] Spectral Lint
[X] npm run bundle outputs nothing suspect
[x] npm run postman outputs nothing suspect

Changes

Update sort_by shared parameter description to include generic items text instead of postcards since it is being used across different resources
Add 11x9_bifold enum to self_mailer_size.yml
~Add cards property to letters resource file - no_extra_service.yml~
[ ]  Figure out why cards property is not displaying for letters LIST endpoint in docs
Rebuild redoc index.html

Areas of Concern (optional)

Related to enums, there's a couple of other minor fields that we might make as enums. Like orientation & object in the cards resource

Talking to @maelingmurphy and we think we need to re-build the redoc docs/index.html so that letters has the new cards examples.
That's coming shortly.

Since this has a conflict, I'm closing it and creating a new PR with the desired changes rolled in.


:bug: Fix: should use klog.FromContext instead of log.FromContext

Summary
When I was debugging the registraiton issue, found this part of log is not shown. It turns out using the wrong FromContext function.

/assign @qiujian16

/approve
/lgtm


Have all AI chat calls use same code path and UX

Fixes #152.
We currently have 3 different places where we interact with OpenAI:

When you ask a question in the prompt form
When we retry an AI message with a different model inline
When a user requests that we summarize a chat in the share modal

These all worked a bit differently, so we had 3 different code paths. However, the one we use for the prompt form was the most mature, and included nice UX features like pause/resume and cancel.
I've amalgamated all three of this into a single one.  The low-level stuff now lives in src/lib/ai.ts and the UI components use it via src/hooks/use-chat-openai.ts.
Some callers need to be able to stream responses, some don't, so I've made something that you can pass event handlers to, but you can also use it via a promise if all you care about is the result.
I've also reworked the code to have both the prompt form and inline retires use the same UI components.  As part of this I fixed some visual bugs, and one downside was that I had to remove the click-to-pause feature.  I think now that we have an actual pause button, this is less important, but let me know if you disagree and I can find a way to re-enable it.
One thing I haven't done yet is fix the auto-scrolling for retry messages, which is #105, and I'll tackle that separately.
I've also fixed a stray import in src/components/Message/MessageBase.tsx we missed earlier today.
This needs a bunch of testing.  Try to look past UI bugs that aren't related to what I've done here (I know that's hard), but I just need to get this new OpenAI logic landed and then I can work on fixing things in common code.

haven't run into any bugs, appreciate the improvements. Time to finish testing in prod.


Request - bump pod version

Hi there, thanks for the nice pod and can't say enough thanks to you guys.
I've converted my app to swift 4.2 and compiler is complaining on CircleProgressView. It seems that the latest swift 4.2 was made to master yet pushed to pods. can you do a version bump and pod repo push?
Thanks

Running into this issue as well.

Me too

Same

Version 1.2.0 has been pushed and includes the changes for Swift 4.2.


Safari fix: re-use existing AudioContext

Safari puts a limit on the amount of AudioContexts that can be created. An application that instantiates the recorder multiple times must therefore be able to re-use an existing AudioContext instance.

@alexthewilde I can also do the changes if you want, there should be a "Allow edits from maintainers" checkbox somewhere in the right sidebar.

@eliasmeire yes please! You should be able to make changes (that box has been checked already).

:tada: This PR is included in version 3.4.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[rnv 0.31] Broken TypeScript support

Describe the bug
After update from 0.30 to 0.31 typescript files are not transpiled and it causes error on compiling time.
To Reproduce

Create new project (e.g. blank template)
Rename ./src/app/index.js to ./src/app/index.tsx

Expected behavior
Project should compile correctly and it should work
Screenshots or copy&paste

Desktop (please complete the following information):

OS: OSX
Node Version v14.13.1
RNV Version 0.31.1

@rroslaniec could you try 0.31.2 . should be fixed

@pavjacko works, thanks for fast fix 🚀


Support passing --netrc or --netrc-file to curl

Closes #28

@johnhampton can you check if this works for you?

If the desire is to pass a bunch of additional options to curl, perhaps we should recommend that people just put a wrapper for curl on the path that adds the options? We could in principle otherwise end up passing through lots of options.

Wrapping curl might be a bit onerous but why not using .curlrc? (Why didn’t I think of it before??).

@johnhampton can you check if this works for you?

This worked for me. Thank you!

Wrapping curl might be a bit onerous but why not using .curlrc? (Why didn’t I think of it before??).

I'm not a curl expert so I missed the .curlrc option. Allowing that to be specified  with curl --config would also work.

I prefer to document how to use .curlrc rather than adding specific support for it in foliage.

@andreabedini Do you have any interest in putting in an option to specify the curlrc file? It would be way more flexible than having to put one in the home directory.

@johnhampton I think curl come to help again:
              When curl is invoked, it (unless --disable is used) checks for a default config file and uses it if found, even when --config is used. The default con-
              fig file is checked for in the following places in this order:

              1) "$CURL_HOME/.curlrc"

              2) "$XDG_CONFIG_HOME/.curlrc" (Added in 7.73.0)

              3) "$HOME/.curlrc"

              ...

setting CURL=$PWD should be enough to have curl look into the current directory.


PowerShell version of run.sh

Removed first line, which PowerShell may interpret as a command before the parameters section.
Also changed the single quotes to double quotes.
This is a guess. The script runs without errors on the Windows Server VM, but apparently not on a Windows 10/11 desktop.


Fix starvation with async server

Make server async to handle high throughput from client (the number of requests a sync server can handle is limited by server side thread number, since each thread handles a request in blocking manner).

This CR cannot merge. It is driving contention (probably GIL) causing stalls.
https://xprof.corp.google.com/trace_viewer/rwitten-7572422995686954905?host_index=0

Can you please add a xprof with the new changes so we can compare against the GIL-contention xprof that Rafi provided and rule-out any contention issues?

Can you please add a xprof with the new changes so we can compare against the GIL-contention xprof that Rafi provided and rule-out any contention issues?

Added a xprof in https://docs.google.com/document/d/1PWje3lup0ZHjtgbcWQbK8r1nNwgKE2rHBdVLbWU2aas/edit?tab=t.0#heading=h.f39bxdhl45e2 (Since here is public, just shared in the internal doc).


Restore {url} tag use in browser arguments and add a few other useful tags...

This is a proposition to fix the regression on the use of the tag {url} to specify the position of the url in the argument to call the browser with.
This should also allow a simple fix for this issue by using the following line in the config file instead : chromeKiosk = "C:\Program Files\Google\Chrome\Application\chrome.exe" --app={url}
I added a few more tags too since it wasn't costing anything much and would allow some other interesting uses, you can see the details of them in the proposed modifications to the readme.md.
I plan to use these tags in my own config file by adding the following lines for example:
[browsers]
steam = "%ProgramFiles%\Steam\steam.exe" steam://openurl/{url}
noQueryFF = "%ProgramFiles%\Mozilla Firefox\firefox.exe" "{authority}{path}{fragment}"

[urls]
# open Steam related links in the Steam application
steamcommunity.com* = steam
steampowered.com* = steam
*.steampowered.com* = steam
# strip Amazon tracking queries in shared links
amazon.com* = noQueryFF
amazon.fr* = noQueryFF

Hey @Elihpmap , thanks for the contribution. I'll review this ASAP - Christmas family activities and travel are keeping me off the computer.

No problem! Take your time and have a great christmas!


[KYUUBI #2402] [Improvement] addTimeoutMonitor for trino engine when it run query async

Why are the changes needed?

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

Codecov Report

Merging #2404 (5e08e67) into master (26d52fa) will increase coverage by 0.01%.
The diff coverage is 50.00%.

@@             Coverage Diff              @@
##             master    #2404      +/-   ##
============================================
+ Coverage     62.80%   62.81%   +0.01%
  Complexity       69       69
============================================
  Files           360      360
  Lines         17187    17196       +9
  Branches       2314     2314
============================================
+ Hits          10794    10802       +8
- Misses         5382     5384       +2
+ Partials       1011     1010       -1

Impacted Files
Coverage Δ

...uubi/engine/trino/operation/ExecuteStatement.scala
71.73% <44.44%> (-6.64%)
:arrow_down:

...engine/trino/operation/TrinoOperationManager.scala
86.36% <100.00%> (ø)

...in/scala/org/apache/kyuubi/config/KyuubiConf.scala
96.51% <0.00%> (+0.12%)
:arrow_up:

...ache/kyuubi/operation/KyuubiOperationManager.scala
96.29% <0.00%> (+1.85%)
:arrow_up:

...rg/apache/kyuubi/engine/trino/TrinoStatement.scala
71.26% <0.00%> (+2.29%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26d52fa...5e08e67. Read the comment docs.

thanks, merging to master


Revert async

Please make sure that the tests pass before merging this. Also when performing the import, please make sure that the previously failing internal tests succeed.

It looks like the internal test failures were due to mocked servicers "helpfully" returning a dummy experimental_thread_pool, which was then being used to submit jobs. Manually fixing the test mocks requires cumbersome code like the following:
mock_servicer = MagicMock(some_grpc_servicer)
del mock_servicer.NeedToDoThisForEveryMethod.experimental_thread_pool

This kind of sours me on mocks in general, but AFAIK we don't provide users any guidance that they shouldn't use mocks, and it doesn't seem we want to legislate that now either way. I'd prefer to fix this forward by adding an instanceof check (yes, that's cumbersome too, but at least its only for our own experimental-and-internal API so seems like an ok bolt-on solution for now), which is done in https://github.com/grpc/grpc/pull/18172


Update Flask to work with Werkzeug 2.2.3 due to open CVE

Hello Flask Team:
I am aware that bug reports regarding sub-dependencies may not be welcome here, but I feel that this is too important (in my case it's most likely a complete show-stopper).
We have invested a significant amount of time building Rest APIs and webapps with Flask, and our CICD pipeline incorporates a well-known vulnerability scanner. Our build pipeline fails due to an open CVE in Werkzeug: https://nvd.nist.gov/vuln/detail/CVE-2023-23934
There are only two known versions of Werkzeug that do not have an associated CVE:  2.2.0 and the latest 2.2.3. However due to the issue ImportError: cannot import name 'parse_rule' from 'werkzeug.routing'  we cannot update Werkzeueg to 2.2.3.
The recommended solution is to downgrade to Werkzeug to 2.1.2 (I believe) and that does work. However that version and all previous versions have CVEs.
Is it possible to make Flask work with the newest Werkzeug 2.2.3? As of now we cannot push any new changes into production and are basically stuck until Flask works with the newest version of Werkzeug. I could be mistaken but I think this may be the case.
And as always, thank you for the support.
Environment:

Python version: 3.8
Flask version: 2.2.3 (newest Flask at the time of writing this issue)

Flask works fine with Werkzeug 2.2.3. It sounds like another library you're using is incompatible with recent releases. For example, in the question you linked, the error is coming from Flask-RestX, which I'm fairly sure was updated a while ago.

Note that the vulnerability you linked is "Low", and requires a confluence of other vulnerabilities to actually get to the point of mattering in Werkzeug itself.

Actually is this for flask-restx? (someone else's project) or is it for Flask as well? It's a bit hard to tell in the logs.
Unfortunately, even the "low" ones are flagged for us :( My apologies if this is actually related to restx.

It's for whatever is trying to do that import that no longer exists. You didn't show a traceback, so I can't say what it is, but it's not Flask itself.

Okay thank you @davidism will close and reopen with flask-restx. The error is actually from when importing their library, but I was mistaken thinking that it was actually coming from their usage of Flask.

It was already reported to Flask-RestX in https://github.com/python-restx/flask-restx/issues/460, fixed in https://github.com/python-restx/flask-restx/issues/463, which seems to have been released in Flask-RestX 1.0.0.


Review and update README

The doc is old. I want to review that it's still current and still addresses the right audience.

I've always thought that the figure at the top of the README (from dissolve.py) was a little too abstract. Particularly when the opening statement is quite technical. The principal use of shapely is for geographic data, although I'm sure it has applications outside this area. It might be nice if there was some context for the figure - what are the points, why are we buffering them? At the same time I understand the desire to stay away from the obviously recognisable "world map" type figure, as shapely doesn't natively deal with coordinate systems. A couple of examples that would be similar, but with some context: identifying "green corridors" created by tree canopies between parks in an urban space; or checking for line of sight for microwave transmissions.

I use shapely for lithography mask design in the context of photonic chip design but examples are probably to complex...

@snorfalorpagus the README could use a refresh for sure! I'm open to anything simple and compelling.

@snorfalorpagus I swapped in the dilation and erosion image from the manual and think it looks better: https://github.com/Toblerity/Shapely/tree/maint-1.6.


How to update the HTML of a document?

I have the following snipper of code which should update all the attributes in the document accordingly.
// Load the HTML document
doc, err := goquery.NewDocumentFromReader(resp.Body)
if err != nil {
	log.Fatal(err)
}
selections := doc.Find("*")
for _, node := range selections.Nodes {
	for _, attr := range node.Attr {
		attr.Val = prefix + attr.Val
	}
}

However, it is not reflected when I try to get the HTML document from the selection:
goquery.OuterHtml(selections)
doc.Text()
doc.Html()

Hello Kristian,
That's because node.Attr returns a copy of the Attribute, so the change doesn't stick. You can use goquery.SetAttr to update an attribute's value, however there's no way to list the attributes with goquery so you still need to loop over the node's attributes. Something like that would work:
var data = `

			Title

`

const prefix = "!!!"

func main() {
	doc, err := goquery.NewDocumentFromReader(strings.NewReader(data))
	if err != nil {
		log.Fatal(err)
	}
	doc.Find("*").Each(func(i int, s *goquery.Selection) {
		for _, attr := range s.Nodes[0].Attr {
			s.SetAttr(attr.Key, prefix+attr.Val)
		}
	})
	h, err := doc.Html()
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(h)
}

Hope this helps!
Martin

That works for iterating over the nodes. However I am hitting a performance wall.
What I am trying to change is have all src/href attributes point to my own domain.
So I am now doing
prefix := "localhost:8080/content/"
s.SetAttr(attr.Key, prefix+attr.Val)

which runs fine for the setAttr(), but when I do doc.Html() it halts forever (waited up to 1 minute).

Are you sure you're only timing the call to doc.Html() and not anything else? That would be very surprising that one prefix causes that slowdown but not the other, it sounds as though something is making network calls (presumably to that localhost address) when you hit a slowdown?
Calling doc.Html is basically calling Go's net/html parser to render the nodes to an HTML string. I'd be very surprised if an attribute's value could slow down that rendering like this, and if that rendering somehow involved network calls (AFAICS this package does not use net).

I was unable to recreate my bug in a minimal testing environment so it is probably due to some concurrency. Consider this closed :) Thanks for all the help!


PowerDistributor should look at battery temperature, too.

What's needed?
Right now the PowerDistributingActor tries to keep equal SoC level in all batteries. (in pool)
It could also try to keep equal temperature in the batteries.
However equal SoC level should be more important then equal temperature level.
What are the benefits:

maximize total available power (when battery is to hot, then its SoP decrease)
battery could live slightly longer - not tested.

Proposed solution
No response
Use cases
No response
Alternatives and workarounds
No response
Additional context
No response

Would it also report the temperature then? Right now it only provides soc, capacity and power bounds


Two Live Feed

Hi Bro,
If we have two live feed how to select which feed to be viewed in the m3u8 file?
Ex: Sky News by default Covid Live is picked, but the 2nd Live feed is the news coverage. How do we select the news coverage here?
https://www.youtube.com/user/skynews
TIA

There is a unique video id for every video. The link can be given as:
https://www.youtube.com/watch?v=X_uOlJPyd1s
https://www.youtube.com/watch?v=9Auq9mYxFEE
respectively.

Thanks. But it tried the link like..
https://www.youtube.com/watch?v=9Auq9mYxFEE
and
https://www.youtube.com/watch?v=9Auq9mYxFEE/live
But unfortunately the video (either the one) doesn't play using these links

https://www.youtube.com/watch?v=9Auq9mYxFEE : this link is there in my playlist, and it fetches the m3u as well.

the script fetches m3u8 from the page source.

Thanks but unfortunately that doesn’t play

Just checked the link again and found out that it doesn't play in vlc. But with other players including tivimate and kmplayer, there are no issues. I don't think people would use vlc in their TV to watch live channels.


Add documentation and story for DateInput component

In this PR

Add props documentation, component comment, and stories for DateInput
Update the prop type of formatOptions from any to DateTimeFormatOptions
Fix a bug where the text input did not have enough right-padding to accommodate a right icon when there is also a left icon

Note that I used the formula (verticalPadding * 2) + glyphWidth from semantic-ui, which is how they end up with the crazy number 2.67142857em
This CSS could also be used for any other inputs that need both right and left icons, something that Semantic UI doesn't support out of the box

Good catch @dleadbetter!


JSON Driver support and examples

Please add support for a JSON driver, since these days, JSON is extensively used as Input and output source.

Sorry, I have no time for open source contributions these days. I will happily accept a patch or pull request.


Update flux to 39.983

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

https://justgetflux.com/mac/Flux39.987.zip was the versioned URL in the appcast?

@commitay Yes. Feel free to revert. Have forgotten that flux appcast is out of date for this cask.
I would even recommend to completely remove the appcast and use :latest as it's out of date for quite long.

See https://github.com/caskroom/homebrew-cask/pull/46263.


Bump Danger (which now ships Chainsmoker!) and some housekeeping

See https://github.com/danger/danger-js/issues/431#issuecomment-480269809 and danger/danger-js#850

Messages

:book:
:sparkles: Thanks for your contribution to Shields, @paulmelnikow!

  Generated by :no_entry_sign: dangerJS against c6964000d25e0f85f9ce68cd1773956aa042fe2d


Convert "InternalTektonResult" from string to int enum

Every Step that Pipelines runs emits a JSON object with metadata about its execution: the content of task results, a StartedAt timestamp, step exit codes etc etc.
The JSON object is passed back to the Tekton Controller by way of the "Termination Message", a Kubernetes feature that allows a Container to write a short amount of content that ends up back in the Pod's YAML. The emphasis here is on "short" - the termination message is limited to only a few thousand bytes, which in turn means that the JSON object Steps return needs to fit within this.
Several fields in the emitted JSON object are "InternalTektonResults", meaning that they are metadata which the Tekton Controller needs but that doesn't need to be surfaced in typically user-facing fields like Task Results. Each of these "internal-only" fields is given a "type" member in the Termination Message that looks like this:
{"type": "InternalTektonResult"}

Each entry in the JSON object that uses this type consumes, at minimum, 28 bytes of the termination message allotment (6 for "type" and 22 for "InternalTektonResult"). As we add more metadata which needs to be passed around this overhead will grow.
For the longer term we've discussed methods of avoiding the Termination Message altogether. In the short term we could easily curtail this wastage by changing "type" from a string to an int. e.g. {"type": 0}. The int would then map to an enum of possible types (of which there are few). This would drop the overhead of each internal result from 28 bytes to 7. We could drop this even further by switching from "type" to (for example) "_" or "t".

/assign


how to train the model without OHEM?

When I train the model without OHTM, the loss will become Nan. Can you give more details on how to train the model without OHEM?
Why I want to train the model without OHEM?
Right now, I am developing a new algorithm based on your project. I wanna to train a baseline model with OHEM.

Can confirm that. The bbox deltas drop to Nan after a short while. Tried to figure it out but I gave up, just enabled it for now. Should be added to the readme I think, it actually took me quite a while to figure out that my error came from the OHEM setting. VOC btw doesn't use OHEM, that adds to the initial confusion because "false" works with VOC.


Incorrect handling of newlines in docstrings

When tsify generates a doctring for a type definition, any newlines in the Rust doc comments appear as "\n" (two characters, "\" and "n") in the TypeScript docstring.

Hmmm. I know this was an issue with an old version of rust-analyzer. Can you verify that the produced .d.ts files have the \ and n in them?

I can confirm that I'm seeing the characters \ and n in sequence in the generated .d.ts file for this type. That many-line doc-comment ends up as a one-line doc-string with the newlines appearing as \n.
Sorry, I know that's not a very self-contained example. I can try to produce a minimal example if it's helpful.

Can you test whether it still happens if you use triple slash /// for the doc comments?

That fixed it!

Great! I will leave this issue open because it really should work with /** */ style comments as well, but at least there is a workaround.
Let me know if you're interested in looking into a PR for this.


Updating AppName to replace looser matchers first

Description of Problem
Right now when you run php artisan app:name MyApp on a fresh Laravel 5 installation you will find several instances of MyMyApp around the application.  This can be verified by running git diff | grep MyMyApp after running the task inside of a git project.  This occurs because the looser matches were at the end of the search/replace array.
Solution
Putting them at the beginning will replace the general instances first, then cascade through all namespace declarations to make sure everything is covered.
Steps to Reproduce

Clone Laravel 5 and run composer install or branch your own code if the namespace is App
Run php artisan app:name MyApp
Run git diff | grep MyMyApp and notice several instances exist
Run git reset --hard && composer install to return your app to the previous state

Caveat: This is still broken.
Let's say you named your app namespace with something else that is declared inside of the existing namespaces.  For example Console, then change it again to App.  You will still change all instances of Console throughout the app to be App.  Perhaps the loosest match on just <namespace>\\ should ultimately be removed?  This will have its own side effects but I'm just presenting it as an option.
The broken state is still broken in the original version, so this is a less broken version.  Not perfect, but while using simple string replacement, this should at least prevent a few more edge cases.

There been 1 or 2 PR, check #7226

Wow - using Cmd-F with the open issues didn't find it, forgot to check PR's separately.  Welp, whichever solution seems better I'm good with


Closes #17 | Implement dataloader for Philippine Fake News Corpus

Closes #17.
Checkbox

[x] Confirm that this PR is linked to the dataset issue.
[x] Create the dataloader script seacrowd/sea_datasets/my_dataset/my_dataset.py (please use only lowercase and underscore for dataset naming).
[x] Provide values for the _CITATION, _DATASETNAME, _DESCRIPTION, _HOMEPAGE, _LICENSE, _URLs, _SUPPORTED_TASKS, _SOURCE_VERSION, and _SEACROWD_VERSION variables.
[x] Implement _info(), _split_generators() and _generate_examples() in dataloader script.
[x] Make sure that the BUILDER_CONFIGS class attribute is a list with at least one SEACrowdConfig for the source schema and one for a seacrowd schema.
[x] Confirm dataloader script works with datasets.load_dataset function.
[x] Confirm that your dataloader script passes the test suite run with python -m tests.test_seacrowd seacrowd/sea_datasets/<my_dataset>/<my_dataset>.py.
[ ] If my dataset is local, I have provided an output of the unit-tests in the PR (please copy paste). This is OPTIONAL for public datasets, as we can test these without access to the data files.

@ljvmiranda921
This dataset was tagged as source-only, running tests.test_seacrowd_source_only should work.

Ah okay got it. I didn't look into the convo thread in the Issues. LGTM!


Add missing nil pointer check

Fixes #189.
This fixes some unchecked pointer accesses in the rest package.
Independently of how errors are handled, these checks should be in place.
Proper error handling as discussed in #189 and #181 can happen separately.
Signed-off-by: Tobias Guggenmos<EMAIL_ADDRESS>

To be clear, merging this PR doesn't mean #189 won't be merged, once it's ready.
#189 does a different thing and addresses a wider issue.


dcos job kill: Attribute error in normalize_marathon_id_path

Please answer the following questions before submitting your issue. Thanks!
What version of DC/OS + DC/OS CLI are you using (dcos --version)?
C:\Users\niroy>dcos --version
dcoscli.version=0.5.7
dcos.version=1.9.0
dcos.commit=0ce03387884523f02624d3fb56c7fbe2e06e181b
dcos.bootstrap-id=58fd0833ce81b6244fc73bf65b5deb43217b0bd7

What operating system and version are you using?
Windows 10 build 1703
What did you do?
I create a simple job and schedule, let it run then tried to kill the job.
dcos job show sleep2
{
  "id": "sleep2",
  "labels": {},
  "run": {
    "artifacts": [],
    "cmd": "echo \"Hello world\"\ndate\nsleep 1000\necho \"Bye\"\n",
    "cpus": 0.01,
    "disk": 0,
    "env": {},
    "maxLaunchDelay": 3600,
    "mem": 32,
    "placement": {
      "constraints": []
    },
    "restart": {
      "policy": "NEVER"
    },
    "volumes": []
  }
}

dcos job schedule show sleep2 --json
[
  {
    "concurrencyPolicy": "ALLOW",
    "cron": "*/5 * * * *",
    "enabled": true,
    "id": "default",
    "nextRunAt": "2018-01-26T23:10:00.000+0000",
    "startingDeadlineSeconds": 900,
    "timezone": "UTC"
  }
]

What did you expect to see?
Job gets killed
What did you see instead?
dcos job kill sleep2
Traceback (most recent call last):
  File "dcoscli\subcommand.py", line 103, in run_and_capture
  File "dcoscli\job\main.py", line 40, in main
  File "dcoscli\util.py", line 24, in wrapper
  File "dcoscli\job\main.py", line 61, in _main
  File "dcos\cmds.py", line 43, in execute
  File "dcoscli\job\main.py", line 247, in _kill
  File "dcos\metronome.py", line 343, in kill_run
  File "dcos\util.py", line 730, in normalize_marathon_id_path
AttributeError: 'NoneType' object has no attribute 'strip'

I could reproduce this, @kensipe any idea?

I got the same issue but with dcos job kill job-id --all


feat: Caddy

WIP

Certbot's autorenewal seems to fail because of a new interactive prompt, likely introduced in recent versions.

/ok-to-test


Why is babel-pundle a plugin instead of a loader?

From my understanding of conventions created by Webpack, loaders are String -> String transformations of the contents of a file, and plugins modify the behavior of the bundler itself. For example, CSS -> JS (style-loader) is a loader, and CommonChunksPlugin, which does static analysis of your project as a whole is a plugin.
Given this definition, since babel does ES6/7 -> ES5, which boils down to String -> String, shouldn't it be classified as a loader?

Also as an addendum to this, what's the convention for naming pundle packages in the future going to be?
Currently you have pundle, babel-pundle, and pundle-dev, and then for the loaders (which don't exist?) you have pundle-less and pundle-css.
It seems like babel-pundle is the odd one out.

We could rename babel-pundle to pundle-plugin-babel. The main difference between a plugin and a loader is that the plugin just transforms the text, the loader is responsible for renaming all calls to require with __require() and asking pundle to resolve the require requests and wrapping the content of the module with vars like __dirname and __filename.
But because Babel is just a step in processing javascript modules and shouldn't need to define those variables mentioned above, it's just a loader. Similar to how postcss would be a plugin and css would be a loader

Ok so really a module bundler is analogous to an ETL framework for JavaScript.
extract = require
transform = plugins
load = loaders
Makes more sense than the Webpack approach tbh. However, it might be smart to make transformation separate from the concept of a plugin such that a plugin can have impure side-effects or change the behavior of pundle globally, but a transform is only a pure String -> String transform.


lcov-diff produces too much data

Trying to make this run with an example:
/home/user/.cargo/lcov-diff test2.txt test1.txt  -o textDiff.txt
The resulting textDiff.txt contains all the lines from test2 and some more instead of a true diff.
test1.txt
test2.txt

I can't say I am an expert in lcov file format, but these are kind of strange files, I mean there are lines with zero coverage. So there is no coverage in both of your inputs.
The tool doesn't remove the lines with 0 hits, just left them as is. So if you try to diff2 from diff1 as in your example all 4 DA records will be in the result. That's the first reason for the increased size.
The second is the LF and LH records, which were missed in your inputs and the tool added them.
If you generate HTML reports with genhtml the result will be correct.
Maybe it's a good idea to remove DA:x:0 lines, but the original reports usually contain them. I believe this is to distinguish the lines which can be possibly covered from lines without any code (blank lines, comments, etc.)
So you probably should just look at genhtml results and don't worry about the actual file sizes.

Is it possible to add support for the diff file as that allows us to get text based summaries versus HTML?


One hot encoder

This PR implements OneHotEncoder for single class per label (#58 (comment))
Encoder supports any Hash+Clone types as labels and prduces RealNumber vectors.
Will wait for feedback before implementing multi-class encoders.

@VolodymyrOrlov Thank You for taking the time to review this, appreciate the feed-back!
Let me make sure that I fully understand your comments:
We do want to have a consistent api with fit and transform for one-hot encoding, but those must act on entire dataframes as input.  My implementation acts on a series (if we stick to the dataframe analogy), so I'll rename those. and make the fit and transform  methods to act on a dataframe.
I'm a bit confused about what the dataframe should be in this case.  smartcore models act on vectors and matricies of RealNumber traits. I attempted to replicate Scikit's implementation where by and large you can encode any type of category.
The simplest solution I can think of is to make OneHotEncoder support only RealNumber categories and have the user define which ones are categorical.
What do you think?

@VolodymyrOrlov Thank You for taking the time to review this, appreciate the feed-back!
Let me make sure that I fully understand your comments:
We do want to have a consistent api with fit and transform for one-hot encoding, but those must act on entire dataframes as input.  My implementation acts on a series (if we stick to the dataframe analogy), so I'll rename those. and make the fit and transform  methods to act on a dataframe.
I'm a bit confused about what the dataframe should be in this case.  smartcore models act on vectors and matricies of RealNumber traits. I attempted to replicate Scikit's implementation where by and large you can encode any type of category.
The simplest solution I can think of is to make OneHotEncoder support only RealNumber categories and have the user define which ones are categorical.
What do you think?

@VolodymyrOrlov Thank You for taking the time to review this, appreciate the feed-back!
Let me make sure that I fully understand your comments:
We do want to have a consistent api with fit and transform for one-hot encoding, but those must act on entire dataframes as input. My implementation acts on a series (if we stick to the dataframe analogy), so I'll rename those. and make the fit and transform methods to act on a dataframe.
I'm a bit confused about what the dataframe should be in this case. smartcore models act on vectors and matricies of RealNumber traits. I attempted to replicate Scikit's implementation where by and large you can encode any type of category.
The simplest solution I can think of is to make OneHotEncoder support only RealNumber categories and have the user define which ones are categorical.
What do you think?

@gaxler This is correct, we'd like to support both, series and dataframes.
I agree with you. OneHotEncoder should one-hot-encode not only floats but also other types, like integers and strings. The problem is that current design is not flexible enough to support all these types.
There is an easy and hard paths, depending on how much time you have to work on this feature.
The easy path is to convert only nominal and ordinal floats. In this case you don't have to refactor Smartcore interfaces. You can either infer categorical columns from data (e.g. by taking a random sample and looking at values in the sample), or take a list of column IDs that should be encoded from user, as you've suggested. In fact, I think we should do both.
The hard path is to introduce two new types: dataframe and series. These types will be modeled after Pandas DataFrame and Series and will encapsulate data as a matrix (or as a vector) and additional metadata, like column names and types. Later we can switch all Smartcore algorithms to use these new types instead of matrices and vectors. Since most algorithms cannot work with categorical data directly we'll have to implement internal converters that will extract data from dataframe and series and transform these values into floats or throw an error, if it is not possible.
Let me know what you think about these options. Also, feel free to connect with me in Discord, if you have an account there. My ID is volodymyr.orlov#7062

@VolodymyrOrlov Thank You for taking the time to review this, appreciate the feed-back!
Let me make sure that I fully understand your comments:
We do want to have a consistent api with fit and transform for one-hot encoding, but those must act on entire dataframes as input. My implementation acts on a series (if we stick to the dataframe analogy), so I'll rename those. and make the fit and transform methods to act on a dataframe.
I'm a bit confused about what the dataframe should be in this case. smartcore models act on vectors and matricies of RealNumber traits. I attempted to replicate Scikit's implementation where by and large you can encode any type of category.
The simplest solution I can think of is to make OneHotEncoder support only RealNumber categories and have the user define which ones are categorical.
What do you think?

@gaxler This is correct, we'd like to support both, series and dataframes.
I agree with you. OneHotEncoder should one-hot-encode not only floats but also other types, like integers and strings. The problem is that current design is not flexible enough to support all these types.
There is an easy and hard paths, depending on how much time you have to work on this feature.
The easy path is to convert only nominal and ordinal floats. In this case you don't have to refactor Smartcore interfaces. You can either infer categorical columns from data (e.g. by taking a random sample and looking at values in the sample), or take a list of column IDs that should be encoded from user, as you've suggested. In fact, I think we should do both.
The hard path is to introduce two new types: dataframe and series. These types will be modeled after Pandas DataFrame and Series and will encapsulate data as a matrix (or as a vector) and additional metadata, like column names and types. Later we can switch all Smartcore algorithms to use these new types instead of matrices and vectors. Since most algorithms cannot work with categorical data directly we'll have to implement internal converters that will extract data from dataframe and series and transform these values into floats or throw an error, if it is not possible.
Let me know what you think about these options. Also, feel free to connect with me in Discord, if you have an account there. My ID is volodymyr.orlov#7062

I think I'll start with the easy path, just to make it usable. Afterwards, would love to work on the new data types.

I think I'll start with the easy path, just to make it usable. Afterwards, would love to work on the new data types.

Thanks!
It seems that we are really close to also doing https://github.com/smartcorelib/smartcore/issues/59
Maybe we can do it in a single PR?


CSS

CSS
User Story
As a boot camp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
AC
GIVEN a Prework Study Guide website
WHEN I view the study guide
THEN I see a dark blue header and footer, and four boxes with a shadow

User Story (US) & Acceptance Criteria (AC) abbreviated hereon for efficiency.


Can't run Onsets and Frames Colab notebook

I'm not sure if I'm doing something wrong but I've just opened the onsets and frames colab notebook, but there are import issues.
---> 10 from magenta.models.onsets_frames_transcription import audio_label_data_utils
     11 from magenta.models.onsets_frames_transcription import configs
     12 from magenta.models.onsets_frames_transcription import constants

ImportError: cannot import name audio_label_data_utils

Same problem with any other imports from onsets_frames_transcription.

Sorry about that. I think there's a mismatch between the notebook and some recent changes we made. A fixed version should be live on Monday or Tuesday.

Should be fixed now!

Hey, it still has the same problem, I just checked this version and it's still failing.
https://colab.research.google.com/notebooks/magenta/onsets_frames_transcription/onsets_frames_transcription.ipynb
Also I think it's a good idea save the versions of the notebook into the repo, so everyone can see previous versions and maybe find when it broke.
Greetings

Solved adding this line just added this line before installing pyfluydsynth.

!apt-get install portaudio19-dev python-pyaudio python3-pyaudio

Sorry about that, should be fixed now. In 270a9b94100270a673a69889b51c1f8b4132b22f, I removed the extra dependencies from the default magenta package setup that were causing the problems.


Update vendored Molinillo to 0.5.3

See https://github.com/CocoaPods/Molinillo/releases/0.5.3

@indirect r?
This also probably deserves a patch release

@segiddins yeah, let's do it.
@homu r+

:pushpin: Commit f3836b6 has been approved by indirect

:zap: Test exempted - status


Added support for sub and superscripts

I added support for Sub- and Superscript markers. I used the new org-togglers branch, since the new infrastructure made it very easy to implement. (Thank you for that!)
I only encountered one small error: When you have something like word1_{sub1} word2 and then remove the {, the first letter of sub1 and the space between the words is hidden. (Since then it technically isn't a valid fragment anymore, so it hides the "previous" fragment, not knowing, that the brace disappeared and the characters moved.)
I haven't found an elegant way to fix this. The only solution that I could think of, is to, instead of saving the element context in org-appear--prev-frag, we simply save the previous point and reevaluate the element context in the post-command-hook. This way we can react to changes in the current fragment.

Thank you for the PR!
I don't see an easy fix for that bug as well - notably, it also affects other fragments, not only your contribution. If you delete stuff inside a fragment and then exit it, the package doesn't know that the fragment "boundaries" changed and deletes everything. Your suggestion on saving (point) to reevaluate prev-frag upon exit seems good, I'll think about how to implement this in an efficient way.
That being said, I noticed your sub/superscript handler in parse-elem checks if element has brackets and hard-codes visible-start/end based on that. This is unnecessary because all sub/superscript elements are guaranteed to have contents-begin and contents-end properties, which tell you the locations of visible-start/end. Would you mind changing that? You can refer to the code for links in parse-elem, except you don't have to check that contents-begin and contents-end exist.

Thank you for the PR!
I don't see an easy fix for that bug as well - notably, it also affects other fragments, not only your contribution. If you delete stuff inside a fragment and then exit it, the package doesn't know that the fragment "boundaries" changed and deletes everything. Your suggestion on saving (point) to reevaluate prev-frag upon exit seems good, I'll think about how to implement this in an efficient way.
That being said, I noticed your sub/superscript handler in parse-elem checks if element has brackets and hard-codes visible-start/end based on that. This is unnecessary because all sub/superscript elements are guaranteed to have contents-begin and contents-end properties, which tell you the locations of visible-start/end. Would you mind changing that? You can refer to the code for links in parse-elem, except you don't have to check that contents-begin and contents-end exist.

Done.
I have a question though: Why don't we use contents-begin / end for the emphasized words (bold, underline, ...) too? I noticed these have also the contents-begin / end set.

Done.
I have a question though: Why don't we use contents-begin / end for the emphasized words (bold, underline, ...) too? I noticed these have also the contents-begin / end set.

Merged! Thanks again.
As for your question, this is because code and verbatim fragments (for some unknown reason) don't have the contents-begin/end properties. We could, of course, process emphasised and verbatim fragments separately but I decided that one less call to member is preferable.
P.S. I'm experimenting with allowing users to edit text inside fragments in the feature branch. If you could play around with it and leave your thoughts here, that would be great! I feel that simply disabling all modifications when inside a fragment is bad code but maybe it works?

Merged! Thanks again.
As for your question, this is because code and verbatim fragments (for some unknown reason) don't have the contents-begin/end properties. We could, of course, process emphasised and verbatim fragments separately but I decided that one less call to member is preferable.
P.S. I'm experimenting with allowing users to edit text inside fragments in the feature branch. If you could play around with it and leave your thoughts here, that would be great! I feel that simply disabling all modifications when inside a fragment is bad code but maybe it works?


ZKS

Issue Report
Description
[Provide a brief description of the issue]
Steps to Reproduce
[Outline the steps to reproduce the issue]
Expected Behavior
[Describe what you expected to happen]
Actual Behavior
[Describe what actually happened]
Additional Information (if applicable)
[Include any additional information, logs, or screenshots]

Issue Report
Description
[Provide a brief description of the issue]
Steps to Reproduce
[Outline the steps to reproduce the issue]
Expected Behavior
[Describe what you expected to happen]
Actual Behavior
[Describe what actually happened]
Additional Information (if applicable)
[Include any additional information, logs, or screenshots]

It's just a template, not any issue...


PROTON-2398 Fix typos in production and test code found by running a spellchecker

The tests fail because I enabled two tests that were not running by mistake. And they fail.

The tests fail because I enabled two tests that were not running by mistake. And they fail.

The tests fail because I enabled two tests that were not running by mistake. And they fail.

The tests for the receiver unblock behaviour are out of date with the current implementation which is why those fail.  Need a separate issue to update that test plus the "doTtestBlockingReceiveCancelledWhenReceiverClosedOrDetached" also should be updated to reflect current exception expectations which probably should be done in an issue that isn't proposing to address spelling errors.

The tests for the receiver unblock behaviour are out of date with the current implementation which is why those fail. Need a separate issue to update that test plus the "doTtestBlockingReceiveCancelledWhenReceiverClosedOrDetached" also should be updated to reflect current exception expectations which probably should be done in an issue that isn't proposing to address spelling errors.

Resolved in https://issues.apache.org/jira/browse/PROTON-2418, https://github.com/apache/qpid-protonj2/commit/f0d8d2d8b25f3f908156500138ec864380ab8e28.

Right, e.g. Artemis uses the merge commits approach too, and it seems to work for them ok.

Each to their own. I know the Artemis folks often rebase things themselves and thats part of why they use merge commits, as they need to, in order to make the PR close in such cases. Except in projects with such frequent commits and numbers of collaborators that it simply isnt feasible to avoid the merge commits, I typically prefer to avoid them.


How can I open the project in Xcode?

I did not find any .xcodeproj file

I don't support xcode as this was built with theos only, all from scratch. If you'd like to add support for it, please feel free to create a pull request.


Panic on record -i

Description of the bug
Not sure what is happening here, I get this panic for record -i
❯ RUST_BACKTRACE=full git record -i
The application panicked (crashed).
Message:  A fatal error occurred:
   0: could not read index: unsupported mandatory extension: 'link'; class=Index (10)
   1: unsupported mandatory extension: 'link'; class=Index (10)

Location:
   git-branchless/src/commands/record.rs:34

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 3 frames hidden ⋮
   4: git_branchless::commands::record::record::hf95eb9a48d11fafe
      at <unknown source file>:<unknown line>
   5: git_branchless::commands::do_main_and_drop_locals::h67a6c33c87e9eaef
      at <unknown source file>:<unknown line>
   6: git_branchless::commands::main::h7b15b6b3e2074e92
      at <unknown source file>:<unknown line>
   7: std::sys_common::backtrace::__rust_begin_short_backtrace::hbdf730c526cfc3ae
      at <unknown source file>:<unknown line>
   8: std::rt::lang_start::{{closure}}::h186a8a31cdb73caf
      at <unknown source file>:<unknown line>
   9: std::rt::lang_start_internal::h09bcc9fffd987f14
      at <unknown source file>:<unknown line>
  10: main<unknown>
      at <unknown source file>:<unknown line>
  11: __libc_start_call_main<unknown>
      at <unknown source file>:<unknown line>
  12: __libc_start_main_alias_1<unknown>
      at <unknown source file>:<unknown line>
  13: _start<unknown>
      at <unknown source file>:<unknown line>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Location: git-branchless/src/commands/mod.rs:464

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 8 frames hidden ⋮
   9: core::result::Result<T,E>::expect::h65a001aa1ac658bb
      at <unknown source file>:<unknown line>
  10: git_branchless::commands::main::h7b15b6b3e2074e92
      at <unknown source file>:<unknown line>
  11: std::sys_common::backtrace::__rust_begin_short_backtrace::hbdf730c526cfc3ae
      at <unknown source file>:<unknown line>
  12: std::rt::lang_start::{{closure}}::h186a8a31cdb73caf
      at <unknown source file>:<unknown line>
  13: std::rt::lang_start_internal::h09bcc9fffd987f14
      at <unknown source file>:<unknown line>
  14: main<unknown>
      at <unknown source file>:<unknown line>
  15: __libc_start_call_main<unknown>
      at <unknown source file>:<unknown line>
  16: __libc_start_main_alias_1<unknown>
      at <unknown source file>:<unknown line>
  17: _start<unknown>
      at <unknown source file>:<unknown line>

Expected behavior
No response
Actual behavior
No response
Version of rustc
rustc 1.66.0 (69f9c33d7 2022-12-12)
Automated bug report
No response
Version of git-branchless
git-branchless 0.6.0
Version of git
git version 2.38.3

Hi @GregBowyer, thanks for reporting. The error message here is referring to the "split index" extension to the index format. libgit2 apparently doesn't implement support for the split index, so we can't operate on it.
The easiest fix is for you to disable the split index, if feasible. Otherwise, we'd have to implement split index support in libgit2 (hard), or use a different Git implementation (gitoxide? I haven't looked into whether it supports split index) or remove the dependency on the index for this code path (easier).
The index is ultimately used here:
https://github.com/arxanas/git-branchless/blob/d11c8420aabfc8162e69bb919547f0263d14b127/git-branchless-lib/src/git/snapshot.rs#L391
The purpose is to get the blob OIDs from the index to construct the commit object. We actually also call git status to get the contents of the index (precisely because git status handles lots of extensions that libgit2 doesn't), but it seems like we drop the blob OID information in this regex (as it's not a capturing group):
https://github.com/arxanas/git-branchless/blob/d11c8420aabfc8162e69bb919547f0263d14b127/git-branchless-lib/src/git/status.rs#L198
So it's possible that the fix is to 1) capture the blob information; 2) use it instead of accessing the Index object; and 3) remove the call to repo.get_index() higher in the call stack. I don't plan to work on this, but feel free to try it yourself.

Relevant but not necessary to resolve this issue: this is the tracking issue for split index support in libgit2: https://github.com/libgit2/libgit2/issues/6132

Could I suggest that git-branchless informs the user that core.sparse-index and sparse-checkout are not currently supported. Either as before running a command or to handle the error and make the output more helpful for these cases.
Here is the error I got after disabling sparse-index (I'm fully up to date):
Command: git move -s 1f4cf75
Output:
Attempting rebase in-memory...
The application panicked (crashed).
	Message:  A fatal error occurred:
	0: could not cherry-pick commit dc3c08e52766dc2dbdd9398cf48c1b85cf7e69f8 onto 39a363890b13d544783580a3ba3ad28bdff36f1c: unsupported mandatory extension: 'sdir'; class=Index (10)
1: unsupported mandatory extension: 'sdir'; class=Index (10)

Location:
	git-branchless-lib/src/core/rewrite/execute.rs:661

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: branchless::core::rewrite::execute::in_memory::rebase_in_memory with effects=<Output fancy=true> repo=<Git repository at: ".../.git/"> rebase_plan=RebasePlan { first_dest_oid: NonZeroOid(f4a7449244c2d10804a27328de4c23102cd806e1), commands: [Reset { target: Oid(NonZeroOid(f4a7449244c2d10804a27328de4c23102cd806e1)) }, Pick { original_commit_oid: NonZeroOid(1f4cf759dad299b339dbf77a8ab50784b0b6b2e8), commits_to_apply_oids: [NonZeroOid(1f4cf759dad299b339dbf77a8ab50784b0b6b2e8)] }, DetectEmptyCommit { commit_oid: NonZeroOid(1f4cf759dad299b339dbf77a8ab50784b0b6b2e8) }, Pick { original_commit_oid: NonZeroOid(0429a1672536b4105c78280a51db6ac8bdc5b155), commits_to_apply_oids: [NonZeroOid(0429a1672536b4105c78280a51db6ac8bdc5b155)] }, DetectEmptyCommit { commit_oid: NonZeroOid(0429a1672536b4105c78280a51db6ac8bdc5b155) }, Pick { original_commit_oid: NonZeroOid(29ef0fdf896213b2ade55a5dc9fbccedfa05ebc7), commits_to_apply_oids: [NonZeroOid(29ef0fdf896213b2ade55a5dc9fbccedfa05ebc7)] }, DetectEmptyCommit { commit_oid: NonZeroOid(29ef0fdf896213b2ade55a5dc9fbccedfa05ebc7) }, RegisterExtraPostRewriteHook] } options=ExecuteRebasePlanOptions { now: SystemTime { tv_sec:<PHONE_NUMBER>, tv_nsec: 1378000 }, event_tx_id: Id(243), preserve_timestamps: false, force_in_memory: false, force_on_disk: false, resolve_merge_conflicts: false, check_out_commit_options: CheckOutCommitOptions { additional_args: [], reset: false, render_smartlog: true } }
	at git-branchless-lib/src/core/rewrite/execute.rs:460
	1: git_branchless_move::r#move with effects=<Output fancy=true> git_run_info=<GitRunInfo path_to_git="git" working_directory=".../platform" env=not shown> sources=[Revset("1f4cf75")] dest=None bases=[] exacts=[] resolve_revset_options=ResolveRevsetOptions { show_hidden_commits: false } move_options=MoveOptions { force_rewrite_public_commits: false, force_in_memory: false, force_on_disk: false, detect_duplicate_commits_via_patch_id: true, resolve_merge_conflicts: false, dump_rebase_constraints: false, dump_rebase_plan: false } fixup=false insert=false
		at git-branchless-move/src/lib.rs:65

		Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
		Run with RUST_BACKTRACE=full to include source snippets.
		Location: git-branchless/src/commands/mod.rs:235

		Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
		Run with RUST_BACKTRACE=full to include source snippets.

@mattrq feel free to open a PR for that


Implement a context management system

Ensure we've gotten away from the unclean passing of context objects (batch, run, vars) and instead implement a manageable system for injecting these based on caller context.
It's been a while since I reviewed this (and now I have the justification to spend time doing so) so will make sure this gets nailed down in the structure of the code.

Taken a first step by actually using the vars in the main configuration. Should now extract this out.


ion-refresher causes inner-scroll to form a stacking context

Bug Report
Ionic version:
[x] 4.x
Current behavior:
When I trigger a pulldown refresh, ionic adds 'transform' to '.inner-scroll', which is correct. But when I finish the pull-down refresh, 'transform' is not removed, which leads to the formation of stacking content.
Expected behavior:
No stacking context.
Remove 'transform' at the end.
Steps to reproduce:
default

pulling

end

Related code:
<ion-content>
  <ion-refresher slot="fixed">
    <ion-refresher-content></ion-refresher-content>
  </ion-refresher>
</ion-content>

Other information:
Ionic info:
Ionic:

   ionic (Ionic CLI)             : 4.12.0
   Ionic Framework               : @ionic/angular 4.2.0
   @angular-devkit/build-angular : 0.13.6
   @angular-devkit/schematics    : 7.2.4
   @angular/cli                  : 7.3.6
   @ionic/angular-toolkit        : 1.4.1

System:

   NodeJS : v10.13.0
   npm    : 6.4.1
   OS     : Windows 10

Hi there,
Thanks for the issue! Are you able to provide an example of the "stacking content" you described?
Thanks!

As long as the "transform: translateZ(0px)" is removed after the pulldown, it can be restored. #stacking-content
Current:
https://github.com/ionic-team/ionic/blob/48814ab134cfc5d8c54895e3a648d5dc8ef8ec28/core/src/components/refresher/refresher.tsx#L341
Expected:
style.transform = ((y > 0) ? `translateY(${y}px) translateZ(0px)` : null);

@liamdebeasi Thanks for your response.
As long as the "transform: translateZ(0px)" is removed after the pulldown, it can be restored. #stacking-content
Current:
https://github.com/ionic-team/ionic/blob/48814ab134cfc5d8c54895e3a648d5dc8ef8ec28/core/src/components/refresher/refresher.tsx#L341
Expected:
style.transform = ((y > 0) ? `translateY(${y}px) translateZ(0px)` : null);

Hi there,
Thanks for the follow up. What you described makes sense. Is this causing any issues with the framework, or is this more of a "best practice"?
Thanks!

However, by default, 'translateZ(0px)' does not exist, so it should be removed when the pulldown is triggered.
I'm not sure if 'stacking content' will cause more problems, but it won't be a problem if you don't create it, right?

Hi there,
I'm having a similar issue with the refresher.  I have a DOM element that is positioned fixed at the bottom of my ion-content.
The problem is that the left-over translateZ causes this fixed position element to no longer be positioned correctly. (some quirk of CSS, I don't fully get why...)
If I remove the translateZ then it positions correctly again.
This could the what the EXTIN is describing as "stacking content", not 100% sure.
Anyway, I agree, it would be better to remove translateZ once the refresher is finished. If it's not there before the refresh starts, why keep it around when it's done?  Seems inconsistent to me and is causing these display issues.
cheers
-adam

I am having the same issue. I have a fixed box  at the bottom of the ion-content, but after refreshing (using ion-refresher) the page, the box is not fixed any more.
Removing the transform: translateZ(0px);fixes it. the solution I have tested isstyle.transform = ((y > 0) ? translateY(${y}px) translateZ(0px) : unset);`

I'm also having the issue with transform: translateZ(0). Please remove it if there's no reason for it.

Up! Having problems with fixed elements, because of the 'translateZ(0px)'. It is such an easy fix. Why hasn't it been fixed for a year?

Thanks for the issue. This has been resolved via https://github.com/ionic-team/ionic/pull/20621 and will be available in an upcoming release of Ionic Framework.


Set pAI up with a static map the pAI can use

About the PR
Set pAI up with a static map the pAI can use. Hopefully will encourage players to pick them up more often or give them to newbies.
Why / Balance
As the personal AI is literally immobile as a ghost, it is useful to ensure that people carry one. With that idea in mind, it is worth giving the pAI a reason to be carried that doesn't depend on the player having a cracking selection of midi files on hand or a ripping wit.
Technical details
Extension of the existing pAI with a new UI class to disable the entity tracking by default. I expect this to be shredded in code review, but I'm working on C# for the first time and had no idea how to make it configurable.
Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

:cl: ZeroNationality

add: Added a static station map for the pAI to use to guide their owner

I quite like this idea, truly putting the pal in pAI


fix(probabilistic_occupancy_grid_map): fix robot_pose to refer base_link_frame

Description

As reported in https://github.com/orgs/autowarefoundation/discussions/4202#discussioncomment-8564823, we found probabilistic occupancy grid map is published not on the baes_link but on the raw pointcloud frame.
This PR fixes the issue.
Related links

https://github.com/orgs/autowarefoundation/discussions/4202#discussioncomment-8564823
Tests performed
@tseco15 Could you check this PR in your environment and check if the problem is fixed?

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[ ] I've confirmed the contribution guidelines.
[ ] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

Hi,
I can confirm that the PR solves the problem. I have tested it in my environment, and the occuppancy_grid_map is now published in base_link frame.


ISPAPI Backorder Module for blesta

The ISPAPI Backorder Addon supports the following functions and features:
High Performance Drop-Catching Technology
Supports gTLDs: .ASIA .BIZ .CLUB .COM .INFO .MOBI .NET .ORG .PRO .TOP
Supports many Afilias, CentralNic, Donuts and Rightside TLDs
Supports ccTLDs: AG .BZ .CC .CH .CO .DE .FM .LC .LI .ME .MN .SC .TV .US .VC
Extra Commissions are Paid to Resellers for Successful Backorders
Backorders are Free to Place and the Backorder Fee is Paid Only on Successful Registration
Multiple Currencies Supported
Multi-lingual (available in English, German with more to come)
Includes a Backorder Domain Name Search Engine
Automatic Drop List Update
Auction Platform for Handling Multiple Backorders
Easy to Install and Operate

Dear @domaingood,
for now, we have no backorder module for Blesta available. Based on current work load and project plan, we cannot provide you nor an ETA for a blesta backorder module neither if we will realize this in future. Reason is simply that the customer demand for this is right now very low and we have a lot more important projects to cover. In addition the most of our customers are using whmcs and therefore we have to give that development higher priority.
As we are currently restructuring things, it could of course be that we have in near future better technical approaches to realize such things easier.
Sorry that we cannot provide you better news. Still, many thanks for getting in touch with us and bringing this request up to our attention. We can understand the need for this, but still we cannot work on this right now for the reasons given. We have added your request to our queue of improvement/extension ideas for blesta.

Just a little update about what is going on at HEXONET / CentralNic regarding 3rd-party Software Integrations and especially regarding Blesta. We will focus this year on technically revamp in-house to improve in direction of 3rd-party Software Integrations to make new Integrations easier for us and leading to less maintenance effort / support on our end. When covered this will allow us to faster react on customer requests and to offer our products and services where ever we notice customer demand. Just a matter of time. We were also in touch with Blesta to get natively shipped with their software. Still, regarding your request - nothing we can promise, just wanted to keep you a bit informed.


Eroshare naming enhancements and XhamsterRipper method refactoring.

Eroshare naming now includes username, gallery-id, and gallery name.

e.g. for url=https://eroshare.com/aB12C3d uploaded by sample-user with
title="hello  there", the directory name will be:
eroshare_sample-user_aB12C3d_hello-there

The raw HTML page is also now saved for eroshare.

Refactored utility methods from XhamsterRipper into base AlbumRipper.

Hi @sleaze - it looks like you've got a bunch of PRs open that mostly are unrelated but include a lot of previous unrelated commits. Is it feasible to split them out into independent commits and PRs that I can test and merge one at a time?


Detect server capabilities fails with ZendServerBamboo addon installed

When clicking the 'Detect server capabilities' button with the addon installed I get the following exception.
com.atlassian.util.concurrent.LazyReference$InitializationException: java.lang.ClassNotFoundException: com.zend.zendserver.bamboo.GradleCapabilityHelper
Running Bamboo 5.7 on Mac OS X Server (Yosemite).

confirmed Ubuntu 14.04


[MB-1245 MB-1246 MB-3406] Domestic Fuel Surcharge Pricer

Description
This PR:

Allows us to price the FuelSurcharge service item
Implements the explicit lookup function for the weight based distance multiplier used in the FuelSurcharge pricer
Implements the explicit lookup function for the actual pickup date used in the FuelSurcharge pricer
Records the distance of the move directly on the MTOShipment in order to be used in the FuelSurcharge pricer

Reviewer Notes
@garrettqmartin8 has left some questions as comments on this PR.
Setup

make db_dev_e2e_populate
make server_run
make client_run
Follow the instructions in the Acceptance Testing Payment Requests Wiki
Verify fuel surcharge price was calculated correctly

Code Review Verification Steps

[ ] Code follows the guidelines for Logging
[ ] The requirements listed in
Querying the Database Safely
have been satisfied.
Any new migrations/schema changes:

[ ] Follow our guidelines for zero-downtime deploys (see Zero-Downtime Deploys)
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] User facing changes have been reviewed by design.
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

References

Jira MB-1245 for this change
Jira MB-1246 for this change
Jira MB-3406 for this change

Messages

:book:
:link: MB-1246

  Generated by :no_entry_sign: dangerJS against a6186d5a7c70c676183e49c188e023a539e78fd3

@harshamurthy @garrettqmartin8 A heads up I've tried with 2 different newly created moves and the pricer returned a 0 cents price

@harshamurthy @garrettqmartin8 A heads up I've tried with 2 different newly created moves and the pricer returned a 0 cents price

Resolved here: https://ustcdp3.slack.com/archives/CP6F568DC/p1596128230210600


Feature request: Reduce the update rate

It would be cool if the rate of the published updates was configurable

What do you mean by "published updates" ?

I am using colorPicker(:selectedColor) and colorPicker(:confirmedColor)
and would like to call less frequently sendColorUpdate. In order to avoid flooding this interface
extension ColorPickerViewController: ColorPickerDelegate {
    func colorPicker(_ colorPicker: ColorPickerController, selectedColor: UIColor, usingControl: ColorControl) {
        sendColorUpdate(color: selectedColor)
    }

    func colorPicker(_ colorPicker: ColorPickerController, confirmedColor: UIColor, usingControl: ColorControl) {
        sendColorUpdate(color: confirmedColor)
    }
}

ColorPickerDelegate is called every time a user makes an interaction (moves finger on control etc.). This is standard behaviour user also by Cocoa Touch controls like UISlider. If you need to trottle user interaction you should do it on your end, in the class that implements ColorPickerDelegate.

Also note that ColorPickerDelegate is ment for eobserving color updates as they are happening. If you only need the final selected color, you can read the value (property selectedColor of ColorPickerController or DefaultColorPickerViewController) when user leaves the screen, presses confirm button or similar.

Got it working with your advice on ColorPickerDelegate. I have a throttler now


BatchNormalization does not support integer inputs.

System information.

Have I written custom code (as opposed to using a stock example script provided in Keras): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS 10.15.7
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): v2.7.0-rc1-69-gc256c071bb2 2.7.0
Python version: 3.8.12
Bazel version (if compiling from source): N/A
GPU model and memory: N/A
Exact command to reproduce: see below

Describe the problem.
The following code fails because the BatchNormalization layer does not support inputs of type uint8 (see the full stacktrace below). You can run this code in this gist.
import tensorflow as tf

(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Dense(10, activation="softmax")
])
model.compile(loss="sparse_categorical_crossentropy", optimizer="sgd")
model.fit(X_train, y_train, epochs=2)

Describe the current behavior.
The code above raises the following exception: TypeError: Exception encountered when calling layer "batch_normalization_1" (type BatchNormalization). Input 'y' of 'AddV2' Op has type float32 that does not match type uint8 of argument 'x'.
See the full stacktrace below.
Describe the expected behavior.
The BatchNormalization layer should automatically cast integer inputs to floats.
If you remove the BatchNormalization, everything works fine, because the Dense layer casts its inputs to floats automatically. I expect these layers to behave in the same way: either both of them should cast integers to floats when needed, or neither of them should. IMO the first option is preferable.
Contributing.

Do you want to contribute a PR? (yes/no): yes
Briefly describe your candidate solution(if contributing): in the BatchNormalization layer, cast integer inputs to float32.

Source code / logs.
Full stack trace:

Epoch 1/2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-15a576831687> in <module>()
      7 ])
      8 model.compile(loss="sparse_categorical_crossentropy", optimizer="sgd")
----> 9 model.fit(X_train, y_train, epochs=2)

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
   1127           except Exception as e:  # pylint:disable=broad-except
   1128             if hasattr(e, "ag_error_metadata"):
-> 1129               raise e.ag_error_metadata.to_exception(e)
   1130             else:
   1131               raise

TypeError: in user code:

    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 878, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 867, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 860, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 808, in train_step
        y_pred = self(x, training=True)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
        raise e.with_traceback(filtered_tb) from None

    TypeError: Exception encountered when calling layer "batch_normalization" (type BatchNormalization).

    Input 'y' of 'AddV2' Op has type float32 that does not match type uint8 of argument 'x'.

    Call arguments received:
      • inputs=tf.Tensor(shape=(32, 784), dtype=uint8)
      • training=True

Hello @ageron. I think the gist file above isn't publicly shared and needs to send a request to get access.

I am able to reproduce the issue with Batch normalization as reported. Please find the gist here. Thanks!

Initial notes from triage:
we don't think we typically cast automatically for users, but perhaps this is a mistake.
@fchollet any thoughts on this?

Hi @LukeWood ,
Thanks for your feedback. I ran some tests by feeding uint8 inputs to almost every type of layer, and here are the results:
The following layers do not complain about the uint8 inputs, and they returnfloat32 outputs:

Average
Dense
Embedding
LeakyReLU
Normalization
RandomHeight
RandomWidth
Rescaling
Resizing

This one returns int64 outputs:

Hashing

These layers return uint8 outputs:

Activation
Add
AlphaDropout
CenterCrop
Concatenate
Cropping1D
Cropping2D
Cropping3D
Dropout
Flatten
GaussianDropout
GaussianNoise
GlobalAveragePooling1D
GlobalAveragePooling2D
GlobalAveragePooling3D
GlobalMaxPooling1D
GlobalMaxPooling2D
GlobalMaxPooling3D
Lambda
Maximum
MaxPooling1D
MaxPooling1D
MaxPooling2D
MaxPooling2D
Minimum
Multiply
Permute
RandomContrast
RandomCrop
RandomFlip
RandomRotation
RandomTranslation
RandomZoom
ReLU
RepeatVector
Reshape
SpatialDropout1D
SpatialDropout2D
SpatialDropout3D
Subtract
ThresholdedReLU
TimeDistributed
UpSampling1D
UpSampling2D
UpSampling3D
ZeroPadding1D
ZeroPadding2D
ZeroPadding3D

These layers reject the uint8 input and raise an exception:

AdditiveAttention
Attention
AveragePooling1D
AveragePooling2D
AveragePooling3D
BatchNormalization
Conv1D
Conv1DTranspose
Conv2D
Conv2DTranspose
Conv3D
Conv3DTranspose
ConvLSTM1D
ConvLSTM2D
ConvLSTM3D
DepthwiseConv1D
DepthwiseConv2D
Discretization
Dot
ELU
GRU
LayerNormalization
LocallyConnected1D
LocallyConnected2D
LSTM
LSTMCell
MaxPooling3D
MaxPooling3D
PReLU
SeparableConv1D
SeparableConv2D
SimpleRNNCell
Softmax

I did not test the following layers:

ActivityRegularization
CategoryEncoding
DenseFeatures
GRUCell
IntegerLookup
Masking
MultiHeadAttention
StackedRNNCells
StringLookup
TextVectorization

So it looks like not casting is indeed the default, with some important exceptions, including Dense, LeakyReLU, Normalization, RandomHeight, RandomWidth, Rescaling, and Resizing.
So perhaps the decision should be done on a case-by-case basis. Regarding BatchNormalization, it would be nice to be able to use it as the first layer, with uint8 images as input.
Wdyt?

Thanks for the analysis. I think casting the int input to floats will make sense for BN layer.
For those layer that raises error when input is int, I think majority of them just cast the input to backend.floatx() (unless we have a good reason to raise an error about the invalid input type).
Feel free to send a PR for this issue if you would like to contribute, and we can apply it to all the layers that are applicable.

The BatchNormalization layer should automatically cast integer inputs to floats.

The way casting current works in Keras layers, is that each layer has a "dtype policy" which contains a "variable dtype" and a "compute dtype". By default both are equal to float32, but they have have different values (e.g. in mixed precision you'd use a policy with a float32 variable dtype and a float16 compute dtype.
All layers will cast their inputs to their compute dtype. BUT this only happens for floating point inputs (e.g. casting float64 to float32). In your case no casting happens because the input is integer type.
We probably have two options here:

Extend the rule above to cast all numerical dtypes to the compute dtype. This may well be ok I think?
Doing the casting by hand on a case by case basis.

I'm trying to think if there are cases where 1) would be obviously incorrect. Maybe image preprocessing layers? But even then the rule "cast to compute dtype" is simple and consistent. @mattdangerw I remember we look at this in the context of KPL, do you remember what our conclusion was?
I'd favor doing 1) (at the level of the base layer) unless we find a significant reason why this would be incorrect. However, backwards compatibility constraints might prevent us from doing so for layers where uint8 is currently accepted and returns uint8 outputs.

Casting all numerical types to compute dtype would be incorrect for layers that need to handle categorical/discrete integer inputs. Layers that would fall into that camp are Embedding, IntegerLookup, Hashing, CategoryEncoding, among others.
We do have an option to disable the casting entirely (kwargs['autocast'] = False). Embedding is a good example, it turns off the autocast option, and casts the outputs, rather than the inputs, to the global compute dtype. That's the general pattern I have been trying to follow in preprocessing layers--cast to compute dtype as soon as possible, which is often after some initial categorical computations. For image preprocessing, I actually think casting early is generally ok.
The place I would most worry 1) would be breaking is for custom embedding layers. I think most of the model garden NLP models would fall into this camp. They use custom embedding layers that do not set the autocast option, but would fail on non-integer inputs.
Possibly 2) might be the more practical option. There are probably only so many of our layers that will actually be used at the top of a model. We could explicitly cast ints to compute dtype for the layers that only operate on floats.
If we go with 1), we need to make sure we leave an option to turn it off (autocast), and make sure that is well know and documented for users. I worry it could be frustrating for a developer who has a legitimate need for integer inputs (there are many), and who can't figure out why inputs are magically changing under the hood.

Thanks for the detailed explainer, Matt!

Possibly 2) might be the more practical option. There are probably only so many of our layers that will actually be used at the top of a model. We could explicitly cast ints to compute dtype for the layers that only operate on floats.

Doing 2) is correct in any case and does not preclude doing 1) in the future, so I would recommend doing 2) right now (at least for BatchNorm and possibly a few more), and open a ticket for future investigation of a more generalized behavior.

I have submitted a change that makes all normalization layers cast their input to their compute_dtype, which resolves this issue. We can do this in additional layers too if there's demand.


Removing the PATH= from evmserver.sh

This is not necessary and clobbers the default PATH environment variable on containers.
Additionally the default path on the appliance is /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin which covers all the locations listed here except for /usr/local which contains no executable files.

/cc @bdunne


route handling

add the ability to define your own routes in AWS
change the route handling

fixes #39