Response map HTTP filter for custom upstream responses

Support custom upstream responses using a new response_map filter.
The response map filter behaves like the local reply code, but on the HTTP filter chain instead. This allows for modification of responses generated both locally and upstream.
The motivation and implementation strategy for this change is captured in https://docs.google.com/document/d/1THyRnepx89FbK2VVaqoEHXv2knbm_lsLx2XgDZ7FxDw/edit?usp=sharing
Commit Message:
Additional Description:
Risk Level: medium (new filter, refactored common code)
Testing: unit tests done, integration tests still needed.
Docs Changes: NEEDED
Release Notes: NEEDED
Platform Specific Features:
Fixes #12883

can we split the matcher / substitution part to another PR?

Yes, will do. I will also look into replacing empty_format with fixed_text and passing that text to PlainStringFormatterImpl.

Could we just add an option to the HCM response mapper to also apply to upstream responses?

Could we just add an option to the HCM response mapper to also apply to upstream responses?

Yeah I'm leaning to this as well.

Yeah I'm leaning to this as well.

+1. I was just about to start looking through the doc, but I'm strongly against this amount of duplication and it seems like we can just extend the mapper to work with upstream responses also. Can we potentially re-orient the doc to that solution and then talk through any issues that come up in the doc?

Yeah I'm leaning to this as well.

+1. I was just about to start looking through the doc, but I'm strongly against this amount of duplication and it seems like we can just extend the mapper to work with upstream responses also. Can we potentially re-orient the doc to that solution and then talk through any issues that come up in the doc?

I'll look into how we could do this in the HCM and update the doc with the new research. When I originally designed this code, I was thinking that the filter chain would be invoked for all sendLocalReply calls but @lizan showed me that this isn't true in all cases (in particular, stream timeouts etc).

I updated the doc. It now tries to focus much more on the problem statement and possible solutions and less on the particulars of this pull request or even code in general. I will now start doing more code research and will update the doc with those findings as they become relevant.

I think that having the HCM matcher be able to alter upstream responses is worth doing, especially for the common case where some server has global response tweaks they want to apply which can't be done via existing config knobs, but there's still the problem of per-route config to deal with.  That feels like a better fit for the RouteEntryImplBase::perFilterConfig which currently requires an explicit filter, though if the @envoyproxy/api-shepherds have a way to generalize that it could address that particular concern.

That feels like a better fit for the RouteEntryImplBase::perFilterConfig which currently requires an explicit filter, though if the @envoyproxy/api-shepherds have a way to generalize that it could address that particular concern.

Yeah agreed that we should have per-route config in addition to generic matching, though I think this is also an issue for local replies, since in many cases a local reply may be generated when a route is available. I would need to look into it in more detail but I wonder if we could just have a non-filter key for override config that could be looked up?

If you're OK with a non-filter key that works for me.  Alternately @esmet suggested offline that (with some possible tweaking to the per-filter-config code) we could arguably use the HCM network filter name (it is a filter, just not L7 =P) instead of an arbitrary string.  WDYT?

If you're OK with a non-filter key that works for me. Alternately @esmet suggested offline that (with some possible tweaking to the per-filter-config code) we could arguably use the HCM network filter name (it is a filter, just not L7 =P) instead of an arbitrary string. WDYT?

Either SGTM. I agree the latter sounds interesting and we could have a generic config message that might eventually allow overriding various pieces of HCM behavior on a per-route basis.

I think that having the HCM matcher be able to alter upstream responses is worth doing, especially for the common case where some server has global response tweaks they want to apply which can't be done via existing config knobs, but there's still the problem of per-route config to deal with. That feels like a better fit for the RouteEntryImplBase::perFilterConfig which currently requires an explicit filter, though if the @envoyproxy/api-shepherds have a way to generalize that it could address that particular concern.

If we're not implementing the response mapper in a filter, then it can be a field in Route message and shouldn't be part of typed_per_filter_config anyway? Then it won't use RouteEntryImplBase::perFilterConfig either.

if we're not doing it in a filter, don't we want it all in the HCM/FM for behavioral consistency? it's not clear to me if we'd need the route specific logic if Envoy sent a local reply late in the response process, but the FM is the only entity which sees the direct responses.

This is still on my plate. The next thing that I plan to do is to try moving the response mapping code out of an HTTP extension and into the HCM itself.

This is still on my plate. The next thing that I plan to do is to try moving the response mapping code out of an HTTP extension and into the HCM itself.

I submitted a new draft PR that implements this behavior in the FilterManager https://github.com/envoyproxy/envoy/pull/14926 feedback welcome.
I'm keeping branch/PR unchanged so we have the HTTP filter implementation to compare to, and preserve history.

@esmet : The way i look at it is that your work is still in progress, and there is no way to control the error pages for example, returned by the upstream.
As we are using envoy as a proxy to the Istio Mesh, my understanding is we'd have to wait a little bitt longer, am i correct to assume that ?
Thanks !

@alexandru-ersenie yes, this work is still in progress and I need to get it to a place where we achieve consensus around how it will be implemented. I will hopefully be able to pick this up again soon.


IsRegex error with bundle and last version of atoum

see https://github.com/atoum/atoum/issues/538 for more informations

https://github.com/atoum/AtoumBundle/releases/tag/1.2.1

Now we have to fix this :-).


Sent current time to device, so we can show it

Use same time processor that monitor time for time events


devops / docker : Fix syntax and style

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

Hi @n1b0r . Congrats on your first pull request (PR)! 🎉
Thank you for your contribution to the page! 👍
We're happy to accept these changes, and look forward to future contributions. 📝


⚠ Bofrost.com has degraded performance

In 050e734, Bofrost.com (https://www.bofrost.com) experienced degraded performance:

HTTP code: 200
Response time: 1374 ms

Resolved: Bofrost.com performance has improved in b1e7098 after 8 minutes.


update README.md to reflect #53

although the PR #53 introduces the Core.App(), and using app.root.map idiom, it doesn't show to use the "mapped" stack to existing CDK code (as in using new new cdk.Stack([what goes here now]) and what goes in new Bucket([here instead of Build<cdk.Stack>]).
Everything should go inside map? Or the return value from map should be used again using map((s) => new Bucket(s)) on the rest of the code? When to use chain? What does ap do?
to be clear: the array method map gives the wrong impression that root.map is an iterator with side effects, although it's just one item. but after reading the code, learned that map is actually a function that provides a param and outputs another wrapped Build.
Also, would be good to export the inner Build type, just in case. (from the index main export)
EDIT: after reading the internals in more-depth, I noticed that Build is really powerful, and while the abstractions of the AWS services are awesome, I think it deserves a "docs" for him, because it needs to get used to it to juggle around CDK constructs and Punchcard abstractions

Oh hey! Sorry, I haven't been checking this in a while - thanks for trying punchcard and providing  feedback.
You're absolutely right that Build needs more documentation. For now, Build is built on fp-ts's Monad: https://gcanti.github.io/fp-ts/introduction/core-concepts.html so perhaps their documentation can help.
The way I think about Build is that it lazily expresses computations that happen in the Build-time phase of an application. So, to interact with existing CDK code, you only need to "map into" existing Build context or create one for yourself.
E.g. without using `Core.App:
const app = Build.lazy(() => new cdk.App())
const stack = app.map((app: cdk.App) => new cdk.Stack(app, 'my-stack'));

This is a lot like ordinary CDK code:
const app = new cdk.App();
const stack = new cdk.Stack(app, 'my-stack');

Except that execution of the CDK code is delayed until the entire Build tree is executed during CDK synth. This layer is designed so that arbitrarily complex CDK code can be used without impacting the code that runs in Lamdba - e.g. interacting with the OS to create files or build code and Docker images etc. Before Build existed, everything was executed all the time - so your lambda function would be calling things like new iam.Role() and table.grantRead(role), which are wasteful and fragile when running within the Lambda Function. Things liek CDK Assets, for example, would break punchcard because they synchronously call out to the file system :(
In other words: Build and Run improve the portability and interoperability between Punchcard and the CDK by encapsulating computations lazily.
RE: the difference between map and chain:
class Build<T> {
  <U> map(f: (t: T) => U): Build<U>;
  <U> chain(f: (t: T) => Build<U>): Build<U>;
  // chain is also known as flatMap in other functional languages, like Scala or Haskell
  // even Java's Stream (which is also a Monad) has flatMap: https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#flatMap-java.util.function.Function-
  <U> flatMap(f: (t: T) => Build<U>): Build<U>;
}

Use map when you don't need to interact with any other Build instances. E.g. creating the Stack from the App (above) didn't need chainbecause it can create the Stack from the App instance passed to its map function.
Use chain when you need to link with other lazily constructed values.
const myTable: Build<dynamodb.Table>;
const myRole: Build<iam.Role>;

const myFunction: Build<lambda.Function> myRole.chain(role => myTable.map(table =>
  const f = new lambda.Function(..,..,{
    role
  });
  table.grantRead(f);
  return f;
))

If we used map here instead of chain, then the type of myFunction would be Build<Build<lambda.Function>>. chain flattened that for us, which is also why it's often called flatMap.

awesome, thanks, I'm already publishing stuff about punchcard even if it's considered unstable and testing it for an MVP like I said (I just can't make it public atm, but I'll eventually), but that's how eagerly excited I am about the whole concept (CDK itself already kick-started my interest). the Build / Run is pure genius, I might add.

okay, I tried to do the following:
import getResource from './mycode'

/*...*/

const api = stack.map((s) => {
  const certificate = Certificate.fromCertificateArn(
    s,
    'certificate',
    certificateArn
  )

  return new p.ApiGateway.Api(stack, 'api', {
    domainName: {
      domainName: 'domain',
      certificate
    }
  })
})

const res = Build.resolve(api).addResource('res')

res.setGetMethod({
  integration: endpoint,
  responses: {
    /*...*/
  },
  request: {
    /*...*/
  },
  handle: async (r) => {
    try {
      return p.ApiGateway.response(p.ApiGateway.StatusCode.Ok, {
        result: await getResource(r.code)
      })
    } catch (e) {
      return p.ApiGateway.response(p.ApiGateway.StatusCode.InternalError, {
        result: null,
        error: e.message
      })
    }
  }
})

obviously didn't work, because I can't use Build.resolve outside of map/chain (and fails when deployed with attempted to resolve a Build value at runtime which is totally true (awesome error reporting by the way 👍 ). should I put everything inside? Because if I use chain, it gives me

to make it happy, add a Build.of, but the end result is the same for when using map. The reason I need the "api" variable is to assign it as an alias for a subdomain as a RecordSet using AddressRecordTarget.fromAlias(new ApiGateway(Build.resolve(api.api))) which works, for Route 53, it's in another stack.map later on the code, but doesn't work for the Build value error above
my first attempt was putting Certificate.fromCertificateArn(Build.resolve(stack)) outside, alongside new p.ApiGateway.Api, but the error is the same (and justifiably)

not related directly to the issue, but I ran across a documentation website generator, https://github.com/facebook/docusaurus might help with documenting this gem of a framework
Related: https://blog.axlight.com/posts/react-tracked-documentation-website-with-docusaurus-v2/

Sorry I haven't responded to your issue yet. It looks like you're getting confused by the Punchcard constructs p.ApiGateway.Api and the CDK's. p.ApiGateway.Api should be constructed outside of the Build and refrain from ever calling Build.resolve, especially in the global module-load scope.
I don't have a better solution at this time though - I hope you don't mind waiting until I get round to re-implementing the API gateway stuff. It is by far the most neglected part of Punchcard right now.


Merge Dev into Main

Note: https://data-sources-v2.pdap.dev is not expected to properly search; we should be routing to the appropriate URL, but the search results page is not built yet.

Issues addressed:

https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/404
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/399
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/398
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/395
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/394
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/393
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/392
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/387
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/384
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/383
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/376
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/374
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/364
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/357
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/353
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/352
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/351
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/345
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/341
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/322
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/249
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/391
https://github.com/Police-Data-Accessibility-Project/data-sources-app/issues/405

Note: I'm not sure if @joshuagraber 's issues have the fixed_in_dev tag, which is what I used to identify the relevant issues.

The issues for v2 include the API work and client work, so I think everything's included here @maxachis

@maxachis nice, this will close a bunch of stuff! I would prefer not to merge anything with failing checks into main, so that we don't get in the habit of ignoring those—should we do a linting pass to make that pass, or remove the test?

@maxachis nice, this will close a bunch of stuff! I would prefer not to merge anything with failing checks into main, so that we don't get in the habit of ignoring those—should we do a linting pass to make that pass, or remove the test?

Just had everything reformatted with black! We are good to go ⚫⬛◼◾


PulseEffecet is replaced with EasyEffects

As PulseEffecet replaced with EasyEffects, please fix configs with EasyEffects.
https://github.com/wwmm/easyeffects

Oh I just checked it now! I was aware that Wallace replaced the PulseAudio interface with the PipeWire one, but I didn't know that he also changed the name!
I'm on Ubuntu 20.04 right now with a working PulseAudio stack + older PulseEffects 4. I see that I can upgrade to PipeWire through a PPA (though it's risky), so I'll see if I can upgrade and test everything in the weekend.

switch to arch bruh😂😂

Could you fix this issue?
I need to activate Dolby Atmos effects.

You can download the json files, run this script over them, and then install them manually. Making the installer do this will be a bit harder.
(Note that the manual method also doesn't copy over impulse files.)
#!/bin/sh

# Replace with 'input' if you want to convert input preset
section='output'

# Convert boolean and numeric strings + replace invalid empty blocklist
perl -i -pe 's/"(true|false|[\d\.-]+)"/$1/g; s/(?<="bl(a|o)cklist": )""/[]/g' "$@"

# Fix plugins order using v5 state field (your set up plugin order is preserved)
for f in "$@"; do

  # Extract active plugins in right order
  po_src=$(jq ".$section"' | . as $out | .plugins_order | .[] | . as $pn | select($out | to_entries | .[] | .key as $key | select(["blocklist", "blacklist", "plugins_order"] | any(. == $key) | not) | select(.value.state != false) | .key == $pn)' "$f" -r)

  # Replace order array in config with new one
  jq --arg po "$po_src" '($po | split("\n")) as $poa | '".$section.plugins_order"' = $poa' "$f" >tmp
  mv tmp "$f"
done

The original script is by AbsurdlySuspicious, I just tweaked it a bit to account for "blacklist" vs "blocklist".

Hey so yesterday I've installed PipeWire and I'm currently testing it with EasyEffetcs. There is an issue with chopping audio under heavy load (e.g. when I'm compiling) which didn't happened using PulseAudio, but I suspect this is more likely to be caused by the fact that Ubuntu 20.04 wasn't built with PipeWire by default (I installed it through the PPA).
Anyway now I'll rewrite the presets so that you can use them out-of-the-box :)

I've uploaded the updated presets and the updated script, let me know if everything works :)

Just installed it on an up-to-date ArchLinux installation with PipeWire and EasyEffects. Works as expected.


Update to latest cdt-gdb-vscode

This picks up many fixes and improvements that have been made to cdt-gdb-adapter

ping on this review.

Note: when opening the CDT blueprint on the Theia website the debug extension is active - but there's no gdb in PATH, which seems quite bad.

Note: when opening the CDT blueprint on the Theia website the debug extension is active - but there's no gdb in PATH, which seems quite bad.

100% - raised #46


cert manager operator inconsistent namespace

The namespace field in base/kustomization.yaml should be updated to match the namespace.yaml resource (cert-manager-operator).
https://github.com/redhat-cop/gitops-catalog/blob/main/cert-manager-operator/operator/base/kustomization.yaml
I believe the startingCSV field in the base/subscription.yaml is superfluous and may be removed.

Thank you for pointing this out.


Named import auto-completion adds additional } after import

Using Enter to accept an auto-completed named import (in JavaScript, where I tested) adds an additional } after the import, leading to a syntax error.
// expected, let | be the cursor
import { a| } from "./a";

// actual, let | be the cursor
import { a| } from "./a";}

Does this issue occur when all extensions are disabled?: Yes

Output of Code > About Visual Studio Code:

Property
Version

Version
1.86.2

Commit
903b1e9d8990623e3d7da1df3d33db3e42d80eda

Date
2024-02-13T19:42:12.210Z

Electron
27.2.3

ElectronBuildId
26908389

Chromium
118.0.5993.159

Node.js
18.17.1

V8
<IP_ADDRESS>-electron.0

OS
Darwin arm64 23.3.0

Steps to Reproduce:

open https://vscode.dev (happens on macOS as well, using vscode.dev just to make it easier to reproduce), open a folder
create two JavaScript files and a jsconfig.json:

 // jsconfig.json
 {
   "compilerOptions": {
     "checkJs": true,
     "moduleResolution": "node",
     "module": "ESNext",
     "target": "ESNext"
   }
 }

// a.js
export const a = 'a';

// b.js, let | be the cursor
import {a|} // vscode auto-completes the }

use the auto-completed import vscode suggests by pressing Enter// b.js, let | be the cursor
import { a| } from "./a";}

Notice the additional } at the end of the line.

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!


Display sh:message in output

There is support for sh:message in the specification so one can add human readable explanations of the constraint.
It seems like this message is not passed to the report at the moment. Would be great to have that to improve the understanding of the problem.

Please share an example of a shape and invalid graph. We've found that the message does get added but only in some scenarios

Now you mention it I noticed that on that particular node I did not had a message, sorry about!
I was expecting the followup message as it was on a different shape but that's obviously not the case because it did not even get there yet (on that particular shape). So all looks good at the moment, I will re-open in case I find a swallowed mesage.


Fallback to regular paste when no image in clipboard

Hi, I really like the extension, thanks for your work.
The only thing I found less intuitive was the different keyboard shortcut, cmd+alt+v
I was curious to map the command to cmd+v, but couldn't do it because when there is text and not an image in the clipboard, instead of pasting I get a notification.
Would it be possible to have Paste Image:

do the pasting if there is an image in the clipboard
fallback to the regular pasting command in case there is something else

This way we wouldn't need to remember a second key-combo, wdyt?

@riccardoferretti, also a fan of the extension and so are my colleagues.
Great idea and I think it is possible.
If that is helpful, you can get a good idea of where the external script in charge of findin data in the Clipboard and how it communicates with the javascript side of the plugin by staring at my pull request:
https://github.com/mushanshitiancai/vscode-paste-image/pull/85/files
So on windows for example the current powershell returns:
$img = [Windows.Clipboard]::GetImage()
if ($img -eq $null) {
    "no image"
    Exit 1
}

We would need to tweak this so we know when to fall back to the standard Paste.
We would need to do the same for each OS and their own external script.

hey @hmalphettes, I was wondering if you could use the vscode editor.action.clipboardPasteAction command instead and have it deal with the OS specific implementation, wdyt?

Hi @riccardoferretti
This repository has not seen any activity in over 3 years. There is an actively maintained fork here: https://github.com/telesoho/vscode-markdown-paste-image
With that said, I considered implementing the feature you are asking for and decided against it for the following reasons:

The way the pasting works here is slow (up to a couple seconds in some cases). The snappiness of the app would be hurt significantly if all pastes were to go through this process.
It may also cause an HTML paste when you wanted a plain text paste or vice-versa causing more issues than it solves.

I recommend treating ctrl+v as simple paste and ctrl+alt+v as rich paste.

Thanks for the reply @knyzorg - you are bringing up good points


playground: update when v0.4.3 is out

What version of CUE are you using (cue version)?
Current playground (https://cuelan.org/play), site says v0.4.2 . Tested with latest Firefox and Chrome on Windows.
Does this issue reproduce with the latest release?
Currently yes.
What did you do?
Entered a backtick character (`) in the playground editor.
What did you expect to see?
An error or something.
What did you see instead?
The page hangs, and is unresponsive. I can't edit the text in the input textbox. Firefox says that "This page is slowing down Firefox. To speed up your browser, stop this page".

v0.4.3 is deployed, and I verified that the bug is now fixed.

Thanks for flagging this, @antong!


Add .bazelrc to .gitignore

Allows devs to specify settings for all build targets. Example use-case:
$ cat .bazelrc
build --define=ij_product=intellij-ue-oss-stable

cc/ @mai93 @vsiva please take a look

It looks like the "better" way to do this is to add build_flags to .bazelproject
build_flags:
  --define=ij_product=intellij-ue-oss-stable

I would propose, instead of this approach, perhaps adding some documentation for how to import and run sources to the README.

@abhillman I see that we mention this in the repo README here and instructions about using existing .bazelproject files are mentioned in the last line of Building the plugin section
"You can import the project into IntelliJ (with the Bazel plugin) via importing the ijwb/ijwb.bazelproject file."

@abhillman Could you let me know if the instructions pointed to by Mai (plus the dev instance instructions) are enough?


cannot select an item during the open animation

The opening animation takes time (like 2 seconds), and this is not possible to select anything during this time. It can be frustrating when the animation is almost finished (the animation seems finished) but the selection does not work.

As suggested by @mishamoe in #84 you have to change animation options to allowUserInteraction in showMenu() method to solve the problem.


request object should be made it's own class instead of a mash

In certain scenarios, like when used with chef, Mash does not work as expected. The request object should be moved to it's own class instead

https://github.com/gilmour-libs/gilmour/pull/18


Update to Cake.Core 0.26?

I just tried to use this addin and my Cake tells me:
Error: The assembly 'Cake.Newman, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'  is referencing an older version of Cake.Core (0.17.0).  This assembly need to reference at least Cake.Core version 0.26.0.
Does this mean that this addin needs to be updated? I'd like to use it and downgrading my Cake version to 0.17 isn't an option for me.

@AndyBarnettCTM yes, due to the breaking changes that were made in 0.26.0, this addin would need to be updated for it to work, without manual intervention.
You have the option of trying the suggestion that you would have seen in the error message that was outputted, as there is a good chance that the addin will just work.

Correct me if I'm wrong, but if it requires 0.17.0, does that mean the November 2016 release of Cake? (https://cakebuild.net/blog/archive/2016/11/) I'm confused because there have been further commits to this repo in Jan 2017

That question would best be answered by @agc93

Sorry about the delays everyone! I somehow missed the notifications for the conversations 😢
I'm going to be updating this repo to use 0.26 as part of #9 , but it still needs some work (build and packaging are broken).
I'll try and get some time, but it's a holiday weekend here, so may not be until next week.

@agc93 I have continued #9 in my pull request of #11

@AndyBarnettCTM @agc93 @nmbro with the release of 0.3.0 I believe that this has been done, and can therefore be closed.

cheers for the hard work everyone!


bugfix/OpenAITopP

Before:

After:

Caution:
We generally recommend altering temperature or top_p but not both
Reference:
OpenAI Temperature and TopP

Before:  After:
Caution: We generally recommend altering temperature or top_p but not both Reference: OpenAI Temperature and TopP


deprecation of kg v1 results in numerous v1/v2 endpoints returning 500 error

https://siibra-api-stable.apps.hbp.eu/v1_0/atlases/minds%2Fcore%2Fparcellationatlas%2Fv1.0.0%2F522b368e-49a3-49fa-88d3-0870a307974a/parcellations returns a 500 error.

should be fixed by https://github.com/FZJ-INM1-BDA/siibra-api/commit/2791148011a31cd7864e524d07e1bc1d07e67db0
Feel free to reopen ticket/open new ticket if other endpoints are found.


[JOSS REVIEW] State of the field

The joss paper is missing a section about the State of the  field where you compare your software to other commonly-used packages.
This is related to https://github.com/openjournals/joss-reviews/issues/5744

Thanks, I've updated the paper.md file accordingly, and included a paragraph under the "Statement of need" section on other open source solvers tailored towards cardiac mechanics. Please let me know if you feel that I've forgotten to mention any other important package(s).


val_acc error

python3.7 main_path.py
Using TensorFlow backend.
2020-05-24 10:46:51.949622: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-05-24 10:46:51.977264: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:51.978080: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: GeForce GTX 1650 computeCapability: 7.5
coreClock: 1.86GHz coreCount: 14 deviceMemorySize: 3.81GiB deviceMemoryBandwidth: 119.24GiB/s
2020-05-24 10:46:51.978264: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-24 10:46:51.979267: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-05-24 10:46:51.980501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-05-24 10:46:51.980684: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-05-24 10:46:51.981722: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-05-24 10:46:51.982196: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-05-24 10:46:51.984400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-05-24 10:46:51.984513: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:51.984834: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:51.985087: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-05-24 10:46:51.985336: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-05-24 10:46:52.012351: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency:<PHONE_NUMBER> Hz
2020-05-24 10:46:52.012836: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7efadc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-05-24 10:46:52.012865: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-05-24 10:46:52.176158: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:52.176517: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56555ad18a10 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-05-24 10:46:52.176540: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1650, Compute Capability 7.5
2020-05-24 10:46:52.176731: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:52.177115: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: GeForce GTX 1650 computeCapability: 7.5
coreClock: 1.86GHz coreCount: 14 deviceMemorySize: 3.81GiB deviceMemoryBandwidth: 119.24GiB/s
2020-05-24 10:46:52.177179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-24 10:46:52.177200: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-05-24 10:46:52.177219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-05-24 10:46:52.177237: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-05-24 10:46:52.177255: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-05-24 10:46:52.177273: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-05-24 10:46:52.177291: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-05-24 10:46:52.177356: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:52.177760: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:52.178219: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-05-24 10:46:52.178259: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-24 10:46:52.178789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-05-24 10:46:52.178804: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0
2020-05-24 10:46:52.178812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N
2020-05-24 10:46:52.178931: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:52.179338: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-24 10:46:52.179714: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2255 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1650, pci bus id: 0000:2d:00.0, compute capability: 7.5)
Train on 20346 samples, validate on 1071 samples
Epoch 1/1000
2020-05-24 10:46:52.840229: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
32/20346 [..............................] - ETA: 2:34 - loss: 45284.4805 - mean_squared_error: 45284.4805 - accuracy: 0.45  992/20346 [>.............................] - ETA: 5s - loss: 19887.5697 - mean_squared_error: 19887.5703 - accuracy: 0.949420346/20346 [==============================] - 1s 70us/step - loss: 10088.8819 - mean_squared_error: 10088.8818 - accuracy: 0.9930 - val_loss: 3886.5988 - val_mean_squared_error: 3886.5986 - val_accuracy: 0.9961
Traceback (most recent call last):
File "main_path.py", line 17, in
history = model.fit(train_inputs, train_paths, epochs=1000, verbose=1, validation_split=0.05, callbacks=callbacks_list)
File "/home/nubonix/.local/lib/python3.7/site-packages/keras/engine/training.py", line 1239, in fit
validation_freq=validation_freq)
File "/home/nubonix/.local/lib/python3.7/site-packages/keras/engine/training_arrays.py", line 216, in fit_loop
callbacks.on_epoch_end(epoch, epoch_logs)
File "/home/nubonix/.local/lib/python3.7/site-packages/keras/callbacks/callbacks.py", line 152, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "/home/nubonix/.local/lib/python3.7/site-packages/keras/callbacks/callbacks.py", line 702, in on_epoch_end
filepath = self.filepath.format(epoch=epoch + 1, **logs)
KeyError: 'val_acc'

Changing val_acc to val_accuracy in main_*.py AND
By removing the callback to the checkpoints, as it attempts to use the checkpoints compiled in an earlier tensorflow/keras version.
I am using tensorflow 2.2.0
Solved


Add check for upgrade scenario

As discussed add a check for those upgrading from the old monolithic package

Thanks!  Fix is checked in, will be in rc2.


feat: take into count of multi-width characters on offset of highlights and jump

solves #121
I added util.offset_pos() and used it to calculate correct offset.
If you want to relocate or rename it, give me a comment or feel free to do it!

converting draft as operator pending mode is giving wrong offset

should be okay now

thanks!


what can be used as {query} in searches?

I can't find an explanation of what can be given as {query} in the documentation.
e.g. GET: anime/search/{query}
can I use wildcards? simple filename wildcards "?" "" or regexp e.g. "." "[abc]" etc.? SQL LIKE-Syntax with "%"? Is it possible to match on specific database fields?
Will a plain string be taken case-insensitive? Which database field(s) will it be matched against? will it match any substring or only the beginning of that field?
Judging by my tests with GET: studio/search/{query} e.g. "a" seems to match all studios whose title begins with [Aa] (case-insensitive). But I would rather find it documented than guess.

The anime/manga searches work a little differently then the rest but for all of them its just plain text, case-insensitive, no like % or regex is supported currently. It should match any substring but this dosen't work too well for the non anime/manga searches at the moment.
For anime/manga the following fields are matched: title romaji, title english, title japanese, and all the synonyms.
For staff/characters first name and last name are matched.

Many thanks for the explanation!


[$1000] Web - Search - Arrow keys don’t work after page refresh

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open the test website https://staging.new.expensify.com/
Log in
Press Ctrl+K for Search open
Type in a search query
Refresh page
Type in a search query again
Use the arrow keys to go through the search results

Expected Result:
I can use the arrow keys to go through the search results after page refresh
Actual Result:
I can't use the arrow keys to go through the search results after page refresh
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/78819774/c0a14d96-426c-4704-a517-7f36beb05699
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~017143e6351525ef91
Upwork Job ID:<PHONE_NUMBER>637136896
Last Price Increase: 2023-06-19

Reproduced, added an external label

Thanks for the proposal @hoangzinh.

we also have arrow key events listener ThreeDotsMenu component in the ReportScreen.

Can we navigate through items in ThreeDotsMenu with arrow? I might be missing which ThreeDotsMenu you are talking about.

Yup. No longer reproducible.
cc: @anmurali


#270 - Add middleware to refresh access token

Hi,
as mentioned in #270 , I think this would be a great addition to the library and the other PR ( #278 ) is stale, so I am trying to take this over the finish line.
I tried to address your comments as far as possible.
You mentioned in the review for the stale PR that you would not add the access token to the session "just for the sake" of it.
I kept  this part of the code intentionally, because I would argue that having access to the access token within the request(-session) is an important feature.
I think the most important use case is to be able to use the access token for subsequent requests on behalf of the user to other services.
In our specific situation, we would like to use django-auth-adfs in a situation, where we have a Django Frontend that needs to  perform the user sign-in (via Azure AD) and then request multiple downstream services on behalf of the user to acquire data to show. As all of these downstream services are in-turn OAuth2 protected, we need to send a valid access token with each request, which is only possible if we can access it outside of the AuthBackend e.g. via the session.
I also checked the solution proposed in #267 but I believe that extending the User model and thus saving the access token in the database leads to a security vulnerability as in this case anyone having access to the database could perform requests on behalf of every logged-in user.
Let's discuss and find a solution :-)

I kept this part of the code intentionally, because I would argue that having access to the access token within the request(-session) is an important feature. I think the most important use case is to be able to use the access token for subsequent requests on behalf of the user to other services.

If you didn't have the access token from here, how would you implement it otherwise in the application (not this library)? I'm not convinced that this package should handle anything other than authentication (not authorization or integration with Azure web services).

Thanks for you swift reply!

I kept this part of the code intentionally, because I would argue that having access to the access token within the request(-session) is an important feature. I think the most important use case is to be able to use the access token for subsequent requests on behalf of the user to other services.

If you didn't have the access token from here, how would you implement it otherwise in the application (not this library)? I'm not convinced that this package should handle anything other than authentication (not authorization or integration with Azure web services).

You would also have to carry out one of the OAuth2 authentication flows to retrieve an access token and thus authenticate the users, e.g. using the MSAL library (https://github.com/AzureAD/microsoft-authentication-library-for-python) in case of Azure AD. But this is exactly what you already do within the library and would mean duplicating the whole token exchange logic etc.
I think the point here is not yet about authorisation or integration yet as exposing the access token is simply the "source of truth" for authenticating the user. This is also what the library uses internally to derive the exposed user information.
However, in some cases (e.g. when one wants to forward the authentication to some other service) the exposed information, which you extract from the token, is not sufficient (e.g. because it is not signed and not verifiable by an external service).
Authorisation (i.e. if you are actually allowed to do something) and integrating with other services would still be out of scope for the library but would require to have this "pure" authentication available in some cases.

The only reason that I can come up with to not store the token is to avoid allowing the library to do more with the API as a part of the library. But I don't think that's a good enough reason to make things more difficult for users. That said, I'm okay with storing it. cc @JonasKs

And now I'm having second thoughts around security concerns. If we start storing the access token in the session, we're making a choice for all users of the library. That shouldn't be something we decide for users unilaterally. Additionally, we can't use the session is the user is using cookie-based sessions.
The changes I'd like to see are:

[ ] Move as much of the new changes to the middleware and out of the backend
[ ] Only store the token information in the session if the refresh token middleware is being used (not sure the best way here)
[ ] Prevent the library from storing tokens in the session if cookie based sessions are being used
[ ] There needs to be some documentation

Thanks for the feedback!
I have some vacation coming up but will look into it again once I am back.


Update checklist.blade.php

Add option to set attributes config

A new inspection was created.

Sure thank you @PiahDoNeumann ! It's been merged into master, and it'll be tagged shortly.
Cheers!

How does this work?

I didn't check, but all in almost all other fields you can define attributes to specify whatever you want: https://github.com/Laravel-Backpack/CRUD/blob/master/src/resources/views/crud/fields/text.blade.php#L12
So if you want to add somethink like a style attribute, you can do that. This should work similarly. @PiahDoNeumann please correct me if I'm wrong. The reason it does not use that view we use in all other fields (I assumed) is because it shouldn't have the default class all other fields have.

I needed to show the checklist disabled, I checked in the checkbox field and had the code there, copied and it worked the way I wanted. This code "'attributes' => ['disabled' => 'disabled'],".
Thank @tabacitu for accepting the PR.


Start Airy Core Platform from the CLI

In order to facilitate the installation and deployment process, we want to explore the possibility of running the whole Airy Platform from the CLI.
As the Airy Platform currently depends on Kubernetes, this needs to do two things:

create a new Kubernetes cluster
deploy all the needed components

Currently we are deploying the platform locally with Vagrant and for running in production we have have charts to start the Kubernetes applications. Installing with the CLI should unify these two installation processes and simplify the installation instructions.
Example command:
airy init
airy start --type local --kafka yes --postgres yes --domainname mycore.com --config ./airy.yaml

For the first step (starting a new Kubernetes cluster) it would be much simpler if we can run it on docker or containerd, so that we don't depend on VirtualBox and Vagrant. This approach will simplify the local installations on the machine, as we would depend on fewer local installations. Should be used only if we want to achieve very fast start and for some simple testing. Having the Kubernetes cluster in a VM (example Vagrant or Minikube) will be more stable.
For the second step (deploying the Airy components), we can take few approaches: create Airy CRD, deploy everything with helm as we are doing now from a pod running in docker (so that we don't install helm locally), talk directly to the kubernetes cluster with Go and create the resources.

As a first comment, I'll leave out the discussion of the actual command name and its flags( while I like airy start, I'm a lot less convinced about the flags UI, the names, the granularity). I suspect this issue will turn into a small milestone so I will keep my comments at that level of detail for now.
I like the idea of depending on kubernetes only (I don't think depending on docker/containerd is enough or maybe even correct). It would indeed bring some benefits. But if we're doing this, we should explore the implications a little more:

the current bootstrap script configures the CLI by adding a kube.conf and the CLI expects the file to be there. Obviously this doesn't work anymore if the we don't have a bootstrap script and interact directly with a kubernetes cluster. It's ok but it impacts docs/code/etc..
vagrant does the hosts configuration locally for us. How do we go about this? Maybe we depend even on minikube, do a loadbalancer? (still need to change the hosts file I think)
what about ngrok? Right now that's also solved at vagrant level

One thing I'm also not sure about is if we're assuming that introducing the airy start command also means getting rid of the bootstrap script. If so, that also has implications: How do we choose the version? The script now depends on the fact it's inside the repo.
As for the second step, I would like to dig deeper into the options. I agree the idea of not depending on helm is appealing. But I can't say I understand what "creating Airy CRD" means in practise (also would that mean the CLI would still need to depend on the GIT repo? That's actually a very general question we should settle). Gut feel makes me think we should discard the last option as I think that would imply the airy cli knows everything (which is a good advantage of the helm approach: whatever is in the chart give or take the config gets there automatically)

I'd like to wrap-up of the conversation that we had yesterday, as well as try to answer on some of the questions that have arisen.
In terms of the name of the flag, probably create suites better then start, because the command will actually create an instance of Airy Core, while start and stop can be used for managing the state of the instance.
The question whether the local instance will run on top of docker or Vagrant or VirtualBox we would still like to leave open for the moment, until we have all the data points and requirements. One mandatory requirement is that Airy Core must run on top of Kubernetes.
As the deployment to different cloud providers becomes more important, we are strongly thinking about reverting to the initial plan of having Airy Core packaged in an image, instead of starting an empty Alpine or Linux instance and provisioning it on the fly. This will also facilitate the local deployment with the cli command, as we would probably need to get rid of the bootstrap.sh script and probably all of the bash provisioning scripts.
Answers to some of the questions:

If we don't have the bootstrap script, true that we would need an alternative way to retrieve the kube.conf file. For this we first need to decide how do we create the local instance.
For the /etc/hosts file, if we don't use Vagrant, we would need to add/remove records there with our code. If we use docker however, all the urls can probably become localhost. But then the port cannot be persistent, so it is a similar problem.
The loadbalancer is currently part of the k3s distribution. If we don't use k3s, we would need to start a loadbalancer process inside the instance, to have access to the nodeport services.
Ngrok is currently bundled inside the components, so as long as the kubernetes cluster can access the internet - Ngrok will work. Also worth noting maybe that if we use Ngrok for the loadbalancer service, then we don't need to modify the /etc/hosts file and we can give public endpoints for the instance that runs locally. I'm not suggesting it as an approach, just as a technical observation.
The ingress controller question I think also depends on how do we deploy Kubernetes. Me personally, I would prefer to have the ingress controller inside Kubernetes, then ro run it separately.
For the version, I think that becomes bit easier, because we would have the version bundled in the cli. So airy create would create the release with the same version of the command itself, there would be no need to read it from anywhere. And in the airy.yaml we can specify different version tag and/or registry.
Introducing CRD at this moment is bit overwhelming I think and that only affects the part we do with helm, so not directly related to this project. At the moment we can use helm to provision kubernetes, in the future we can switch to mapping the airy.yaml file into an Airy CRD and things get created by themselves.

For us to be able to fully picture the project and create the proper issues, these are the first open questions from my perspective:

How do we deploy Kubernetes locally?
How do we make Packer or the way we build the image part of the CI/CD pipeline?
Do we rely on terraform to deploy to the cloud?

Thanks for the write-up @ljupcovangelski! Is the port not being persistent for a local docker installation a problem? As far as I can tell we could do with localhost setup for the local version because no routes intersect. We would only have to add a prefix path for the UI container.

Good point @chrismatix . I was thinking in a sense that we cannot force a particular port for the loadbalancer, because the user might have something running on port 80 or port 8000 already. But we can make this configurable for the user and then when one runs:
airyh create --type local --port 8000

We create a loadbalancer container listening on port 8000. All the ingress routes then should be
localhost:8000/ui
localhsot:8000/api
localhost:8000/chatplugin
localhost:8000/webhooks
...

Best if we can only configure the routing inside the ingress (remove the prefix-path from the request we send to the individual services), so that we don't need to change the code in the components.

Few things we need to consider if we switch to the image deployment concept:

If we have Kubernetes packaged in the image, we need to make sure that certificates and the kube.conf file changes for every different instance
Currently with the provisioning we also introduce a way we start our applications. For example we wait for Zookeeper to be there, to start Kafka. We wait for Kafka to be up and running before we scale up the Schema registry and all the other components of Airy Core. We need to consider how do we orchestrate the start and stop of the instances. One way might be that we introduce our entry-point command for the containers, where they wait for a list of services to be up, before starting the code that needs to run on the container.
Do we package the docker images in the box or not. If we orchestrate the startup, then not having the images packaged in offers a better experience (things are downloaded when they are needed). If we start all at once, better to have the images in the box, as things will start faster. Now the Kafka images are much smaller, so that is also an option.

Also something that comes to my mind is that perhaps we should do this step by step, so that we can meassure the effect on the user experience better:

Strip the provisioning of the prerequisites and the components as much as we can.
Introduce an image, but with the same bootstrap.sh script
Get rid of the bootstrap script in favour of the airy create CLI option.

Possible options for running Kubernetes locally:

Minikube - https://github.com/kubernetes/minikube
K3s - https://rancher.com/docs/k3s/latest/en/
Microk8s - https://microk8s.io/
Kind - https://github.com/kubernetes-sigs/kind
K0s - https://k0sproject.io/
Kops - https://github.com/kubernetes/kops. More intended for deploying to production environments, but has very good integration with Terraform and also offers some helm-like template capabilities, so worth considering.

Also something that comes to my mind is that perhaps we should do this step by step, so that we can meassure the effect on the user experience better:
* Strip the provisioning of the prerequisites and the components as much as we can.

* Introduce an image, but with the same `bootstrap.sh` script

* Get rid of the bootstrap script in favour of the `airy create` CLI option.

I love this suggestion and I think the outcome of this issue will be a milestone that will guide us through this process.
I'm not sure I understand how moving "back" to an image approach plays with running kubernetes locally. Isn't the assumption that we "run an image" locally via virtualbox? (Virtualbox being an example... but also the most reasonable solution imo).
As I understand it, we agree on:

airy create is a better name
introducing images so airy create can refer to that to, well, create an airy core instance
eliminate the bootstrap.sh all together (even though we may want to do this in steps)

Now some clarification/further question

the CLI already bundles the current version so in theory airy create can refer to that to fetch the right images
I'm not sure what's the concrete proposal for the networking. I'm working with the assumption that all the base images we will produce (virtualbox, aws, gcloud etc..) will use k3s which means we'll have access to treafik and can "save" what we already have (I see no good reason to change this but maybe I'm missing something). It's still not clear to me what we're proposing for the hosts when you're creating a local instance.
Do we package the docker images in the box or not I would say definitely not for the airy components. We have the controller that can start/stop things accordingly to a config and there's no reason to make the base image bigger than it should be. But I do agree we should probably have kafka/schema registry/redis already backend in. They're "hard requirements" for the system after all (you could make an argument for redis... but the image is pretty small)
Currently with the provisioning we also introduce a way we start our applications I think we could get away with an init script in the image itself. init-container may be too hard. After all, we want to start airy components once kafka is there. Maybe we can even make the controller responsible for this? (as in, wait for kafka to be there to start things). It would "just work" as we'd start kafka and the controller and the controller will start the rest once kafka is there.
How do we make Packer or the way we build the image part of the CI/CD pipeline? that is also something I'm not sure how to picture but very central to the problem we have. The "real question" to me is where would we host these images? I can picture the pipeline (have an action that runs packer). Maybe s3 like we do with the CLI

Almost there I think. Very exciting work! After another round of questions/answers, I feel we will be ready to create the milestone (it's also my personal hope :D)

I'm not sure I understand how moving "back" to an image approach plays with running kubernetes locally. Isn't the assumption that we "run an image" locally via virtualbox? (Virtualbox being an example... but also the most reasonable solution imo).

If we have made a decision for switching to Virtualbox/Vagrant image - then I think we should stick to the existing k3s, if there is a way to change the kubernetes certificates when we run a new instance of the image (after the image and the kubernetes cluster inside are created).

As I understand it, we agree on:
airy create is a better name
introducing images so airy create can refer to that to, well, create an airy core instance
eliminate the bootstrap.sh all together (even though we may want to do this in steps)

I agree

the CLI already bundles the current version so in theory airy create can refer to that to fetch the right images

I also think this is the best approach

    I'm not sure what's the concrete proposal for the networking. I'm working with the assumption that all the base images we will produce (virtualbox, aws, gcloud etc..) will use k3s which means we'll have access to treafik and can "save" what we already have (I see no good reason to change this but maybe I'm missing something). It's still not clear to me what we're proposing for the hosts when you're creating a local instance.

Correct. If we use an image with k3s, the traefik ingress that comes with it - would be the best option

    Do we package the docker images in the box or not I would say definitely not for the airy components. We have the controller that can start/stop things accordingly to a config and there's no reason to make the base image bigger than it should be. But I do agree we should probably have kafka/schema registry/redis already backend in. They're "hard requirements" for the system after all (you could make an argument for redis... but the image is pretty small)

:+1: Should we have two images: one for local deployment (with kafka, postgresql, redis bundled) and also another one for when people decide to run Airy core with their own setup of the prerequisites?

    Currently with the provisioning we also introduce a way we start our applications I think we could get away with an init script in the image itself. init-container may be too hard. After all, we want to start airy components once kafka is there. Maybe we can even make the controller responsible for this? (as in, wait for kafka to be there to start things). It would "just work" as we'd start kafka and the controller and the controller will start the rest once kafka is there.

I vote for the init-container option, because keeps the image clean and the problem is purely put in ops domain how do we want to run that image. Also would be visible from kubectl get pods: If kafka is not there, the pod will be stuck in the Init stage. Perhaps we will also have some other logic in the future, when an app waits for two services and a particular config-map to be there?

    How do we make Packer or the way we build the image part of the CI/CD pipeline? that is also something I'm not sure how to picture but very central to the problem we have. The "real question" to me is where would we host these images? I can picture the pipeline (have an action that runs packer). Maybe s3 like we do with the CLI

Yes, s3 is good I think.

I would like to keep the local and cloud deployment as similar as possible and this Vagrant provider could be the solution  (also exists for AWS). Because all the cli has to do to switch between local and cloud deployment is to pass a couple arguments to vagrant up.
The vagrant box(image) itself can also stored in the Vagrant cloud  for free.

We are wrapping-up this issue after a final discussion, with the following design decisions:

Naming should be airy create
The only component that needs to be there/created is a Kubernetes cluster
No more Terraform, we talk directly to AWS/cloud-provider API
No more Vagrant,  we talk directly to VirtualBox

We have two major steps:

Strip the provisioning of the prerequisites and the components as much as we can, so that we don't use bash to provision stuff in the Kubernetes cluster.
Decide how we create Kubernetes cluster, both locally and in the cloud.

After this, we can get rid of the scripts/bootstrap.sh script.


Invariant Violation: requireNativeComponent: "RNSketchCanvas" was not found in the UIManager

I am using react-native version 0.73.6 and  Expo 50.0.17

Sorry you are encountering that. As our sample application is presently on RN73.x and works perfectly fine - I cannot replicate this.
I encourage you to attempt building the sample application and see if you replicate the issue - https://github.com/sourcetoad/react-native-sketch-canvas/tree/master/example


add colon(:)

fix #569

LGTM once CLA is done

@yancai Please could you sign the CLA and then let us know? I can then close and reopen this PR. PMC members should hold off approving this for now as we will have to do it again.

@yancai can you please sign the CLA? If not someone else can do this fix for you - just let us know

@duglin Sorry, I am currently on a business trip until July 23. I can't sign the CLA now. If you want to fix this urgently, you can find someone else to fix this bug.

No, I don't think its urgent, we can wait - thanks for the update

@duglin I have signed CLA and sent to<EMAIL_ADDRESS>

@yancai thanks!

really easy review!  3 more needed

CLA signing success!


Hide "promotion available" banner for faction heroes if the player learns an ability on one of the previous ranks instead.

This is the issue: https://imgur.com/a/HQ0xGMq
Note how the banner is there even after there is no ability points to unlock any new abilities.

Issue resolved in the relevant commit.


Add ValidatePhoneNumberRequest to UI Request Creation

Tied to #1680
For whoever ends up working on this, please reach out to me with any questions

@mgmccarthy You want to do the same thing you did in RequestApiController with ImportController am I correct? if so, what should I do about not having the CountryCode? I assumed I should validate against List<ImportRequestViewModel> and I only have Phone in ImportRequestViewModel, I saw @stevejgordon's todo for converting country to country code, should I wait for that or do it myself or go ahead without it? sorry for so many questions :)

@HamidMosalla I am sure @mgmccarthy will have something more/better to say here but in short you can assume CountryCode is 1 (US) for this initial release, note that in the PR and then post this initial release we will fix/adapt that to support additional countries.  I recall this being an approach we used somewhere else ( @stevejgordon @mgmccarthy might remember where)

Also no apologies for asking questions @HamidMosalla it's awesome you care to know what is wanted!

@tonysurma Thank you, I can go ahead with the current info, but it would've been better if I had more detail, for example in the RequestApiController we have only one phone number, we validate it, and if it wasn't valid we return BadRequest();, but in the case of the current issue we have a list of phone numbers, should I return with an error message and do nothing else even if one of them wasn't valid? or I only accept the valid ones and show a message that says theses numbers weren't valid, more important than that is PhoneNumberResource.FetchAsync a remote call? if it is validating so many numbers over the wire is problematic, there should be some kind of method that accepts a list of ValidatePhoneNumberRequest, is there any thing like that exist in the project that I don't know about?

@tonysurma Looks like this issue has been fixed even before I made a comment, I should've paid more attention ():

@hamidmosalla, I already have PR  #1706 done and waiting to be merged for this issue.

@HamidMosalla, it's my fault about the confusion. I never assigned this Issue to myself  and I should have. Sorry about that.


[Concurrency] Only auto-import _Concurrency when auto-importing Swift.

This allows the standard library to build with concurrency enabled.
Fixes rdar://70700689.

@swift-ci please smoke test

Bah, forgot to update test cases that were missing the explicit import.

@swift-ci please smoke test


Add staged cleanup for calls such as: mountMappedDirectories

We should likely add staged cleanup for multi level operations if a later operation fails. IE: Remove the directory we created if the mount later fails, etc.

We actually don't need that if these calls fail on create docker will nuke the container anyways.


Fix empty logfile with no removed programs

The if statement used to determine if the before and after programs list
are identical was flawed. %ERRORLEVEL% should be used instead of
!ERRORLEVEL!. You may want to check your other statements to make sure they execute correctly.

Closing to fix up my codebase


Add an option to enable LoadReportingSerivce client in Envoy

Provide an option to make the Envoy report load stats to the LRS (LoadReportingService) server via LRS.

@ingwonsong: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
b2b3404f9990272600b2315bcb4c18b89df8a79e
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ingwonsong: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
b2b3404f9990272600b2315bcb4c18b89df8a79e
link
true
/test lint

gencheck_istio
b2b3404f9990272600b2315bcb4c18b89df8a79e
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ingwonsong: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
a624fd2c48cbb08fc8d7fc8b717dbfd372330f15
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ingwonsong: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
1b344eed1e69d59ec07d0c293d822fe5cd4d0e2a
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/cherrypick release-1.18


bumping js-slang to 0.4.69

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update
[ ] Code quality improvements

How to Test
parse("- 1;"); should give
operator combination with -unary as operator
Checklist
Please delete options that are not relevant.

[x] I have tested this code
[x] I have updated the documentation

Pull Request Test Coverage Report for Build 7748

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 32.627%

Totals

Change from base Build 7745:
0.0%

Covered Lines:
2284

Relevant Lines:
6444

💛  - Coveralls


Html markers

Hi, this is not really an issue, but I couldn't find sample from playground samples.
What I'm looking for, is how clustering HTML markers work?
I see, its done on azure maps web SDK but cant figure out, how to archieve it with react-azure-maps

Thanks for your feedback, we have missing logic with adding HTMLMarker to dataSource. It will be fixed in next release.
Anyway you can still cluster Features and Shapes and is more effectible and have better performance instead of HTMLMarkers
@QPRocky  To be sure we are on the same page: can you add exactly what you want and how you done it on pure azure maps sdk?

Thanks for reply.
Actually I would like to cluster points like here:
[https://azuremapscodesamples.azurewebsites.net/HTML Markers/HtmlMarkerLayer/HTML Marker Layer.html](HTML Marker Layer)
But instead using build-it markers, I would like to use my own custom images. (at moment managed to use own images with  component)

Ok now I see what we need to update. I will update exactly the same example on our playground. I think it should be available tomorrow. Thanks again for feedback and feel free to create another issue if we still have missing some features or you need help

Wow, very nice!
Thanks!

Update: We still working on that but facing a couple of issues with this custom layer implementation which coming from AzureMapsCodeSamples we will try to solve it somehow.

@QPRocky feature is done in 0.0.7 example of using this is in https://github.com/WiredSolutions/react-azure-maps-playground/commit/3099420a0d0d1fdc14fdba2125a76dcea33c2150


disable send_anonymous_usage_stats for CI runs?

@jtcohen6 is this something we need to do for this adapter and dbt-sqlserver? or is it already taken care of by dbt-adapter-tests?

Taken care of in dbt-adapter-tests


draw.war / external references conflict with company privacy policy

Today I saw after installing the latest draw.war (8.6.8) in our NextCloud installation that my browser script blocker showes me two external references: cloudflare.com and jgraph.com.
This stays in conflict with our privacy policy. Please can you remove these references to external servers/services? This might be also important for others because of the new DSGVO in Germany and the GDPR in Europe. Thx.

Cloudflare is the CDN draw.io is delivered on and jgraph.com is our company web site. We are GDPR compliance and we have a signed GDPR agreement with Cloudflare.

What you can also try is in the NextCloud field where the draw.io server is specified, add &stealth=1 to the URL parameters

Thank you for your comment. But when we include draw.war in our site, we need some agreement and documentation with you and have to inform our users about this. In addition our service would need external services/servers to run. Why are these external references in the code? If it's not really needed it produces huge bureaucracy.

I will try the &stealth=1. Thx.

jgraph.com is just the dynamic footer, it's not needed. Can you screenshot the network traffic tab showing the cloudflare calls please?

I've added https://github.com/pawelrojek/nextcloud-drawio/issues/42 in the NextCloud integration project, hopefully the author might implement this.

Just adding the &stealth=1 to the draw.io URL field in the NextCloud setup did not work. Draw.io did not start with this parameter.
I wanted to add the screen shot but allways see "something went really wrong .." Sorry, I'm not a github expert. So I put it on the nextcloud: https://nc.artcom.de/index.php/s/XrSZEzpkTYC6rgC

Do you have existing URL parameters? If not, the first character has to be a question mark, the & is only for later parameters, e.g. https://www.draw.io/?stealth=1.

If I use it in the apache reverse proxy setup (ProxyPass /draw http://localhost:8080/draw?stealth=1) it gives me:
HTTP Status 404 - /draw%3Fstealth=1
type Status report
message /draw%3Fstealth=1
description The requested resource is not available.
Apache Tomcat/8.0.32 (Ubuntu)

The parameter can not be added in the Nextcloud app. The app itself has to deal with it to add this parameter - otherwise the URL will be malformed, since the Nextcloud app will always build in in the form
?
If you set the URL in the configuration to https://www.draw.io/?stealth=1 the result will be something like https://www.draw.io/?stealth=1?parameter

PS: According to https://desk.draw.io/support/solutions/articles/16000042546-what-url-parameters-are-supported- the parameter should not be stealth=1 but a combination of offline=1&stealth=1.


Rework the RAFT project to be more incremental

It's a bit much as it stands.
I can see two ways to make it more reasonable

provide a starter with all the mechanics of rpcs etc defined, but some methods to be filled in
provide a full implementation but without the otel tracing element. include some bug or bugs that can be tracked down with tracing.

Planning to work on the idea 1 above in April. The to-be-implemented parts should reference which sections of the raft paper are necessary

We should sequence the project a bit as well - e.g. do leader election first, then the k/v
I think experienced folks would do this but harder for newbies to make a plan of attack
also, see if we can add distributed trace to an earlier project eg tracing


Object literals in README

They're not implicit since they're explicitly instantiated via
a literal.

The {}s are implicit

This syntax is taken from https://livescript.net. And they named it implicit objects though it's actually another literal form of objects. I think this description is also acceptable.


Consolidate policy types into a spoofed type for simplified list and creation

Fix #833 Fix #679
WIP
This is the initial work for consolidating the policy creation flow.
Namespace scoped

Cluster scoped

Closing as this is no longer planned.


Fixed typos

@jackHedaya Thank you for your PR!
I should more careful about writing😅

@all-contributors please add @jackHedaya for doc


Unable to use hotreload with --framework flag in command line

Hello,
when using this plugin, the hotreload functionality for aspnetcore and blazor is not working.
Using the verbose flag i see this:
 Did not find a HotReloadProfile or running a non-default command. Watching with legacy behavior.
If i launch dotnet watch manually from my terminal, everything works fine.
I noticed that adding the --framework flag breaks the hot-reload functionality in dotnet watch.
So if i try:
"C:\Program Files\dotnet\dotnet.exe" watch run  --project D:/Test/Test.csproj --framework net6.0
hotreload doest not work, but with:
"C:\Program Files\dotnet\dotnet.exe" watch run  --project D:/Test/Test.csproj
it works!
Maybe you can change the settings to add the --framework flag only if the user opts in, and remove it from default behaviour (so dotnet watch will take the default configured for the project).

Fresh version uploaded, will be available in Rider soon.


Corrected issue in case of change of month

In reference to:
https://github.com/JWGmeligMeyling/jqm-calendar/issues/36#issuecomment-138086164
Plus:
can change background color of event
changed from "a" to "button"
PS: Test needed, please... logic is if this month has 31 days and next 30 then I go to 30, next month is 29, then I go to 29 (feb 2016) then I stay on 29 whatever..

:+1:

:blush:


Test rolling with axis=1, win_type=, and center=True once pandas bug 46135 is fixed.

Describe the problem

In my fix for #4185, I'm going to skip testing for this behavior due to https://github.com/pandas-dev/pandas/issues/46135. Once that bug is fixed, we should test this behavior.

@mvashishtha that bug in pandas is marked as fixed, we should proceed with testing.

@vnlitvinov looks like we have to wait till at least version 1.5. The fix isn't included in pandas 1.4.4, which was cut after https://github.com/pandas-dev/pandas/issues/46135 was merged.


Add resolve feature

Now we can control Components interface for loading async data

Add a test and I'll merge. Thanks :)

@ryanflorence Added

@ryanflorence check it please...
Also, please, have a look at https://github.com/Rezonans/redux-async-connect
Do you like this idea - to sync props with state?..


Update QuickStart.rst

Fixing a spelling mistake done in the Quickstart tutorial. Changed from "report_intermeidate_result" to "report_intermediate_result".

Hey, not sure why those tests failed. Is it relevant for my changes?


In the traffic ops API, deliveryservice.rangeRequestHandling should be a boolean rather than a string

api/1.1/deliveryservices/:id.json returns
rangeRequestHandling: "0" or
rangeRequestHandling: "1"
rather than
rangeRequestHandling: false
rangeRequestHandling: true

change made, issue closed

this is not true. should be a string and not a boolean. reverting change previously applied. per the docs:
How to treat range requests:
0 Do not cache (ranges requested from files taht are already cached due to a non range request will be a HIT)
1 Use the background_fetch plugin.
2 Use the cache_range_requests plugin.

change reverted. issue closed.


I've downloaded everything I possibly can for this and then I press start and it says connection break retrying...

Let me start by saying that i know absolutely nothing about coding, im just trying to use the neje laser engraver for projects at my job. ive followed every instruction i could possibly find, downloaded everything for it that i could possibly find to download. i followed this entire thread on the EZgraver and i finally at least got it to the point where i could send the image to the machine but then it says image downloaded and then i press start and it immediately says connection break retrying and then i get nothing! i just want to use the machine that i just spent 80 damn dollars on! please help a girl out!!!!!!

http://www.trusfer.com/2017-dk-8-kz.htm


Bug-fix -- navigation bar

Description of Changes
Purpose

[ ] Adding a new file
[ ] Changing an existing file

Affected Files
List the files that are being added or changed:

path/to/file1
path/to/file2

Proposed Changes
Provide a brief description of the changes you are proposing:
Screenshots (if applicable)
Please attach any screenshots that will help explain your changes:

Rtarun3606k

assign me

nav bar ok