ROX-13557: Make Operator central.db.isEnabled Enabled by default and drop the flag

Description
A detailed explanation of the changes in your PR.
Feel free to remove this section if it is overkill for your PR, and the title of your PR is sufficiently descriptive.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
TODO(replace-me)
Use this space to explain how you tested your PR, or, if you didn't test it, why
you did not do so. Valid reasons include, for example, "CI is sufficient",
"No testable changes". Feel free to attach JSON snippets, curl commands,
screenshots.
In addition to reviewing your code, reviewers must also review your testing
instructions and make sure they are sufficient.

/test all

/retest

/retest

/test grouped-static-checks

/test grouped-static-checks

/retest

/retest

/retest

/retest

/retest

/retest

/test openshift-newest-operator-e2e-tests

/test openshift-oldest-operator-e2e-tests

/test openshift-oldest-operator-e2e-tests

/test gke-qa-e2e-tests

Just a quick note based on the PR description before I dive in:
I don't think I understand this sentence:

The central.db.isEnabled feature flag remains to ease migration to 3.74 if the flag is already presented in the CRD.

And regarding this:

The flag will drop after 3.74 release.

I'd have to refresh my memory but I do not think it is possible to remove a field in a given apiVersion. So maybe say something along the lines of "this field has no effect and will be removed in a future API version"?

Also, the title "Make Operator central.db.isEnabled Enabled by default" sounds much weaker than what actually seems to be happening here?

Unfortunately I won't have time to review today, but perhaps you can fix the failing operator e2e tests in the meantime @kurlov ?

/retest

@kurlov in the light of discussion yesterday, please check with the postgres migration team whether this flip is still ok to merge now.

/retest

/test go-unit-tests


Fixed typo

@paulkel : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Verified the update.
@paulkel - Thanks for submitting the fix.


Info.plist: file does not exist or is not readable or is not a regular file

I'm getting the below error when I run "ignite new MyApp"
...
...
✔ Installing Ignite dependencies (~ 1 minute-ish)
✔ Linking external libs
⠋ Linking external libs./MyApp/ios/MyApp/Info.plist: file does not exist or is not readable or is not a regular file (Error Domain=NSCocoaErrorDomain Code=260 "The file “Info.plist” couldn’t be opened because there is no such file." UserInfo={NSFilePath=./MyApp/ios/MyApp/Info.plist, NSUnderlyingError=0x7fe499609ce0 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}})
✔ Updating PList file
⠋ Cleaning up after messy guestscd: no such file or directory: MyApp/android
/bin/sh: ./gradlew: No such file or directory
✔ Cleaning up after messy guests

Time to get cooking! 🍽

To run in iOS:
  cd MyApp
  react-native run-ios

To run in Android:
  cd MyApp
  react-native run-android

$ cd MyApp
$ react-native run-ios

ENOENT: no such file or directory, uv_chdir

$

Here's the output of ignite doctor:
$ ignite doctor

Computer
  Platform: darwin

Ignite
  Version: 1.9.0
  Path: /usr/local/bin/ignite

Node
  Version: v6.6.0
  Path: /usr/local/bin/node

NPM
  Version: 3.10.3
  Path: /usr/local/bin/npm

Yeoman
  Version: 1.8.5

React Native CLI
  Version: 1.0.0

App
  React Native Version: ¯\_(ツ)_/¯

I can create "normal" React Native apps with react-native init MyApp
Am I missing something? I'm fairly new to Node and RN, although I have already been able to build part of my app from scratch and using existing libraries I have found in the web. This Ignite project though I'm starting it on a different folder, like, my existing partial app is under /dev/mobile/react and this Ignite app I'm trying to install and run under /dev/mobile/ignite
Any help will be greatly appreciated, looking forward to start using this amazing starter tool.

We'll be releasing ignite 1.9.1 soon.  I'd like to see if the problem will persist with that.
Just doing finishing touches.

Great, I'll keep an eye here for when it's released

please try now:  https://github.com/infinitered/ignite/releases/tag/1.9.1

Thanks, @GantMan but unfortunately I'm getting the same error...

Computer
  Platform: darwin

Ignite
  Version: 1.9.1
  Path: /usr/local/bin/ignite

Node
  Version: v6.6.0
  Path: /usr/local/bin/node

NPM
  Version: 3.10.3
  Path: /usr/local/bin/npm

Yeoman
  Version: 1.8.5

React Native CLI
  Version: 1.0.0

App
  React Native Version: ¯\_(ツ)_/¯

looks like your android and your ios folders are not getting created.   We're not sure why this happens for some people.  Are you sure your NPM permissions are right?
https://docs.npmjs.com/getting-started/fixing-npm-permissions
If you're sure, I'd like to schedule a screen share and see if I can ascertain the source of this issue.  It comes around for random users.

I did use the brew install node command when I first installed node, so the permissions should be correct.
Let me know when you want to do the screen share, I'd be more than glad to help you find the cause of this issue.

Can you point me in any direction to troubleshoot this error myself? I uninstalled most of the node modules I had installed before I even tried react native just in case something there was causing conflicts, but it didn't help. I have tried to find the text in the errors to see if it could give me some pointers but I have found nothing.

sorry, yes I can.  Do you have a chance to catch me on our gitter chat?
You can run the ignite commands by hand and possibly get more feedback.

spawnCommand should identify if a command failed.
https://github.com/infinitered/ignite/blob/master/ignite-generator/src/app/index.es#L217
solution exic code?   https://github.com/mmalecki/spawn-command

Thanks, I'm currently trying one of those permission fixes after trying uninstalling/reinstalling node and getting the same results.
I'll report back with more findings

Well, running the react-native init command with root seemed to work fine, so after a bit more of researching, I found this post with a lot of information about the problem: http://stackoverflow.com/questions/16151018/npm-throws-error-without-sudo/24404451#24404451
Then I ran this command:
sudo rm -rf ~/.npm
Apparently, sometime when I was starting out with node I ran npm using sudo, and it might have broken something, and for some reason the permissions in that directory were still wrong even after trying reinstalling node or cleaning cache and what not.
Anyway, I still have the problem when running ignite new appname but not when I run react-native init appname --version0.34.1 two times in a row with the same app name
I'm not sure if this "second run" fix would have worked or not without rf-ing my ~./npm directory
I will keep troubleshooting

Ok, a lot has been going on, the last run I did after modifying the react-native init call inside your app/index.js file, just by adding another arg after the version number, I added --verbose
Now, it created the directories and all, but I get this message when it finishes:
...
...
   create verboseapp/fastlane/Fastfile
   create verboseapp/fastlane/Matchfile
✔ Installing Ignite dependencies (~ 1 minute-ish)
✔ Linking external libs
✔ Updating PList file
⠋ Cleaning up after messy guests
FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':app'.
> failed to find Build Tools revision 23.0.1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 6.515 secs
✔ Cleaning up after messy guests

Time to get cooking! 🍽

To run in iOS:
  cd verboseapp
  react-native run-ios

To run in Android:
  cd verboseapp
  react-native run-android

ahh!  No build tools revision 23.0.1!
If you type android and get the downloads, please grab it.  If you type android and see nothing, that means you probably don't have it.
Instructions for adding it here:
https://facebook.github.io/react-native/docs/getting-started.html

@GantMan it finally worked! I did npm update and I also installed yarn... I noticed this time it said something about using yarn or npm and it preferred yarn. I also noticed a warning about android SDK 23 not present, but it did work all the way this time!
Thanks for fixing this man, now time to enjoy and get started on a couple of apps I have in mind.
Cheers


General code cleanliness pass

I took a brief pass to check for:

Outstanding "# TODO's"

@snehalchaudhari98 @aninda-ghosh -- there is one "# TODO : generalizing the patch mapping" that I think you wrote.
There are also remaining "TODO: Change me"s in the config files

Dead code

We still had some references to boundary loss in dataset/trainer even though we didn't pursue this in comprehensive experiments
Some code blocks in the morphological feature calculations I commented out a longggggg time ago

Referencing where some code comes from -- main.py is from torchgeo

How do people feel about linters?

There are certain conflicts due to changes in the folder and file structure. Would require some more time to fix this PR

No worries, I'll rebase


Allow global install

It would be useful if setup-pixi allowed the installation of global executables (ie like pixi global).
This could either be by referring to a global manifest, or maybe providing a list of package to pass to global install for simpler cases.
This could offer better options than existing specialised GH actions. For instance, setup-cloud does not support caching natively, and take a fair bit of time (20s).

This could help address, amongst other use cases, https://github.com/prefix-dev/setup-pixi/issues/126


Support AVX/AVX2

Hello, please support in future releases AVX/AVX2 instructions. It will allow guest OS to run much faster using SIMD features on somes customs builds, if available on host cpus. Thank you.
R. Chebah.

Summary:
No AVX and/or AVX2 features available and seen for guest OS  (thanks to cat /proc/cpuinfo under linux).
The details:

HAXM version:  v7.5.6

Host OS version:  Windows 10 v1803

Host OS architecture: x86_64

Host CPU model: See below

Host RAM size:  16GB

(Optional) Host computer model:

Guest Environment

Android Emulator or QEMU version: QEMU emulator version 4.2.0 (v4.2.0-11797-g2890edc853-dirty)

Guest OS version: Debian Buster (10.x)

Guest OS architecture:  Debian Linux x86_64 (32 bits userland using by selection at boot x86 or x86_64 kernel)

To Reproduce

Steps to reproduce the behavior:

qemu-system-x86_64.exe -hda i386_debian.img --accel hax -m 4096 -smp cores=2,threads=2 -net nic -net "user,hostfwd=tcp::5022-:22"
cat /proc/cpuinfo
see features of cpuid replies

Expected Behavior

Expectation:
AVX/AVX2 features please on Guest OS (cpuinfo by example). Thank you.
Reproducibility

Diagnostic Information

Host crash dump:

HAXM log:

Android Emulator or QEMU log:

Screenshots:

The screenshot above show Guest OS after requesting cpuinfo and show the kernel version. Cpuz tool is running on host OS (Windows x86_64) showing AVX/AVX2 available instructions.
Additional context

None

Thanks for the input. We'll evaluate the impact to current products first and setup plan accordingly.

Thanks for the input. We'll evaluate the impact to current products first and setup plan accordingly.

I think it's easier to allow setting supported CPU features with API and not hardcode them in haxm. This will allow Android studio to use the old, hardcoded, CPU feature set and QEMU users would set the required features through -cpu command line switch.
To add full AVX support these bits should be set:
cpuid 1 - ecx bit 28 = avx
cpuid 7 - ebx bit 5 = avx2, 16 = avx512f, 17 = avx512dq, 21 = avx512_ifma

Hello,
Does theses -cpu options use a softcoded AVX emulation thanks to the
QEMU API and command? Because I initially requested a full support of
AVX registers thanks the HAXM driver on windows in order to increase the
speed when AVX SIMDs are used.
Thanks.
Le 23/12/2020 à 10:17, nevilad a écrit :

I think it's easier to allow setting supported CPU features with API
and not hardcode them in haxm. This will allow Android studio to use
the old, hardcoded, CPU feature set and QEMU users would set the
required features through |-cpu| command line switch.
To add full AVX support these bits should be set:
cpuid 1 - ecx bit 28 = avx
cpuid 7 - ebx bit 5 = avx2, 16 = avx512f, 17 = avx512dq, 21 = avx512_ifma
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/intel/haxm/issues/290#issuecomment-750029999, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACS3ZLJCFVKVBXPR4I3JKR3SWGYSPANCNFSM4MGSGKUQ.

The short answer:
To use a feature it must be supported by your hardware and returned as supported by the hypervisor. A good written guest first checks the feature existence and when it is supported uses it. Haxm does not return avx features when asked for supported features and there is no way to change this without modifying haxm. So a good guest will not use it.
In your case the easiest way is to add these features to the returned list. When building a modified haxm version and using it (it's signed by a debug certificate!) is a feasible option for you, I can send you modified code.

The long answer:
I explain it on qemu+kvm example, as these support all these features and describe the differences in qemu+haxm.
Qemu allows to set cpu features through -cpu command line switch, for example -cpu Broadwell. Qemu has builtin info about many CPUs, including their supported cpu features. It finds the cpu (Broadwell in our example), gets its cpu features and sets these to KVM through it's API.
This is not supported in qemu+haxm - haxm support code in Qemu ignores -cpu command line switch. There was a patch to support it for older cpus, but I don't know if it was accepted.
When the guest asks for supported cpu features, KVM returns the list without checking if the underlying hardware actually supports it. When the hardware does not support it and the guest trys to use these instructions, it will get an exception. For example the guest should check avx before using avx instructions. When these are returned as supported by KVM and not supported by the hardware, the guest gets an exception.
Your CPU supports avx, modifying haxm to return these to the guest is the only change needed.

Does theses -cpu options use a softcoded AVX emulation thanks to the QEMU API and command? Because I initially requested a full support of AVX registers thanks the HAXM driver on windows in order to increase the
speed when AVX SIMDs are used.

No, not softcoded. It uses real CPU.


Access to Gamecube settings - RG353M - ArkOS

Evening Christian.
I've had a scout on the wiki and reddits , tried all button inputs. is there something I am not doing correctly ?
I'm trying to access the settings from within a gamecube rom to try frameskip
RG353M latest ArkOS revision - It's managed surprisingly well in a few titles and I want to try messing with frameskip settings etc
Thanks again for all the hard work, I'm using ArkOS exclusively now

The gamecube emulator is dolphin standalone so there are no settings to access from a menu like retroarch.  frameskip is already on but if you'd like to change settings, you can do so via ssh by modifying /home/ark/.local/share/dolphin-emu/Config/Dolphin.ini.  You can also try changing some options in /home/ark/.local/share/dolphin-emu/Config/GFX.ini.  If you find some setting changes that seems beneficial, feel free to share here.  Thanks,

thankyou Christian I will continue testing (mainly 2d) and I will post configs for the arkOS crew once finished !


Fix concurrency issue in source change detection

Ref #845

To validate this pull request against other sbt modules, please visit this link.

Both Travis CI and App Veyor passed on the first try.


Escape Characters are not evaluated

What OS?

Web-Version
Windows 10 / VS-Code:

Version: 1.51 .1 (user setup)
Commit: e5a624b788d92b8d34d1392e4c4d9789406efe8f
Date: 2020-11-10T23:34:32.027Z
Electron: 9.3.3
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Windows _NT x64 10.0.17763

Version?

vs code plugin 0.5.2
browser (https://edit-csv.net/) v1.1.1

Description
I set up escape characters (first try (a): \ second (b): =).
My expectation ist, that a,b\,c=,d will render
(a) a | b,c= | d
(b) a | b\ | c,d
Steps to reproduce

load following csv

Name,field-with-escape,field-without-escape
foo,1=,2345,0
bar,98\,765,1
baz,456\,0,2

Set EscapeChar to \
a. expected foo|1=|2345|0, next line: bar|98,765|1
b. got foo|1=|2345|0, next line: bar|98\|765|1
Set EscapeChar to =
a. expected foo|1,2345|0, next line: bar|98\|765|1
b. got foo|1=|2345|0, next line: bar|98\|765|1

My "render" do you mean writing out the csv oder rendered in the table?
The EscapeChar option is used to escape the QuoteChar in the cell value... (we use the option escapeChar from papaparse: https://www.papaparse.com/docs)
I see this is not obvious from the ui, I should add a ? hint with tooltip.
If you want to escape the ,, you need to quote the field/cell:
Name,field-with-escape,field-without-escape
foo,1=,2345,0
bar,"98,765",1
baz,"456,0",2

(I think this is what you want?)

Yes, in deed that is what I am looking for, tank you!

I've added the hint but as this is a very minor change I won't release an update yet.
This will ship with the next update (e.g. next fixed issue).
We can leave this open as a reminder :)


Add a test for failure when using v4 signatures.

Adds a unit test with the Amazon Java SDK for failures when submitting
v4-signed requests.

Updated to use the ExpectedException rule.

Commit message should mention that we use the AWS SDK instead of jclouds due to JCLOUDS-480.

Please address Checkstyle nits then we can merge this.


Fix #2051, Resolve doxygen doc warnings and enforce in CI

Describe the contribution

Fix #2051

Testing performed
Built doc standalone and confirmed with all the merges it built without warnings
Expected behavior changes
None except warnings squashed
System(s) tested on

Hardware: i5/wsl2
OS: Ubuntu 18.04
Versions: Bundle main + dependent commits

Additional context
Needs nasa/osal#1219
Third party code
None
Contributor Info - All information REQUIRED for consideration of pull request
Jacob Hageman - NASA/GSFC

CCB:2022-02-23 APPROVED


Add a CONTRIBUTING.md File

Add a CONTRIBUTING.md File explaining all the contribution guidelines.

Hey bro, there is no need to create a CONTRIBUTING.md file, the README file covers everything needed to contribute to the project already.

Hey bro, could you add the following labels "hacktoberfest" and "hacktoberfest-accepted" and maybe I will give it a try.


Remove trailing slash in proxyUrl

Infura has trouble handling requests with a trailing slash in API endpoint. https://github.com/INFURA/infura/issues/221
Credit to 🍯 for the find

Cleaned up the fix a bit by directly sanitizing trailing slashes from the proxyUrl instead of changing pathname.

Cleaned up the fix a bit by directly sanitizing trailing slashes from the proxyUrl instead of changing pathname.

Nevermind, reverted back to the original change since it 👨🍳 pointed out that it's actually more correct. We'll leave pathname untouched if it's actually defined as something other than "/".


More safelist/blocklist refactoring

Add Rack::Attack namespace to deprecation warning.
Add deprecated Rack::Attack.blacklisted_response attr methods.
Drop support for end-of-life ActiveSupport/Dalli versions

Builds on #181


Cut-down developer-focused version of the CLI

Is your feature request related to a problem? Please describe.
Some Lagoon operators would like to provide developers with a simplified version of the CLI so that there is less to learn / break. For example it would be nice to facilitate common developer workflows without advertising the ability to update (break) user/group permissions etc.
Describe the solution you'd like
I'm not sure what actions would be required. Probably:

Edit environment variables
Trigger Deployments (on non-prod, this will be enforced by RBAC)
??? (please request in the comments!)

Describe alternatives you've considered
The current CLI is great but has functionality which we might not want to advertise to avoid breaking certain bespoke Lagoon installs.
Additional context
n/a

Potentially there should be a UI way to achieve these more basic actions. I for one wish env vars where possible to view/edit in the UI.

Will look into feasibility as part of the rewrite.


http://playerjs.io/test.html is broken for short videos

I'm testing an embed for a gif that's been converted to mp4. It's really short, less than 3 seconds, so the setCurrentTime and getCurrentTime tests fail. I'm using the HTML5Adapter.
Example: http://playerjs.io/test.html?url=https%3A%2F%2Fstark.riffsy.com%2Fdev%2Fsean%2Frx.fcgi%2Fembed%2F4989370%3Fplaypause%3D1%26autoplay%3D1%26muted%3D1

Why was this closed? The bug hasn't been fixed.

Sorry, should have marked as won't fix. Happy to accept a pull request, but using a longer video solves the problem nicely.


Why there's always a .cargo-ok in git status?

Steps to Reproduce:

cargo check

And this path: ~/.cargo is totally wrong. My project(.git) is under ~/Documents/xxx

pls see also https://github.com/rust-lang/cargo/issues/6340

pls see also rust-lang/cargo#6340

But it shouldn't appear here.
And the git status works well.  But vscode's integrate git not. Even I add it into the .gitignore, It still there


Trying to run a simple Vue App

After cloning a repository with a fresh Vue 3 created app.
I'm trying to run "npm install" to get the packages, but im receiving the message:
"Program interrupted. This could be caused by a memory limit or an error in your code."
Im trying to run it in an iPad (6th generation) 128 GB. Any help?

my ipad pro 2020 A12Z 6GB memory has the same problem

in ipad 9gen, and i have the same problem

Same here. Will this be fixed?

I have the same thing. Npm does work okay for me (installs all libraries); but when running node with one of my own javascript files, I get the Program Interrupted message. Is there a way of seeing the underlying error? I am on an M1 iPad.

@tomkerswill The latest version (1.4.1) should improve the error handling. Can you try it?


Allow for React-based layouts

This adds a layoutType option, which can be either blaze(the default) or blaze-to-react.
When blaze-to-react is specified the routes are rendered using ReactLayout instead of BlazeLayout. The user may specify the layout and content regions in an almost identical way to the standard Blaze configuration.
The User Account templates themselves are rendered into the React layout using the
Blaze-To-React package.
Dependencies upon React, BlazeLayout, ReactLayout and BlazeToReact are all weak so that they aren’t pulled in to the user’s code base unnecessarily.

No one interested in this PR?

sure!
It's busy time, that's all :-(
I was actually discussing about this PR with @jshimko a few hours ago, we'll try to get it merged asap.
nice work, btw :-)

This is great! @splendido when can we expect the next release so I can grab this package officially via meteor add?

@adrianmc : a few hours?


Using jimp instead of gm/GraphicsMagick

Think it would be possible? https://github.com/oliver-moran/jimp
I think more people would use this plugin if they didn't have to install GraphicsMagick.

Nice find!
Avoiding external dependencies makes definitely sense when there is a equivalent replacement.
I'll look into it.

Released as 0.2.0 (#10).

Woot!


DSD-493: changelog update for SimpleGrid component

Fixes JIRA ticket DSD-493
This PR does the following:

adds changelog entry for SimpleGrid component

How has this been tested?

Checklist:

[ ] I have updated the Storybook documentation accordingly.
[ ] All new and existing tests passed.

Front End Review:

[ ] View the example in Storybook

Tugboat has finished building the preview for this pull request!
Link:

https://pr696-pfgwlopoayuqxsde2oyrg2pdbh3zsgxo.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/615223104807c0dc3f3cc462


WIP: Case study 2

@theoxo can we close this now that case_study_2 is not even a branch we use?

@theoxo can we close this now that case_study_2 is not even a branch we use?

Oh yah definitely. I think this is so out of date it's not even funny. Let's not delete the branch though; this branch has some test/train split code in it that I don't think we ever merged into main (and which might prove useful in the future).


add support for preserving header key case

Adds support for preserving header key case in HTTP 1.1
Fixes https://github.com/istio/istio/issues/32008
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[ ] Does not have any user-facing changes. This may include CLI changes, API changes, behavior changes, performance improvements, etc.

@su225 Sorry for hijacking your PR and API changes. But we have a high severity customer issue because of this and we need a fix immediately. Hence pushed this PR to fast forward the work. We were not able to do it via Envoy filter because of proto.Merge limitation on nested typed configs.

@ramaraochavali: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-security-multicluster-tests_istio
2f72c1db99fdc0bd7ba36413c44fb69b5a1ab904
link
/test integ-security-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-security-multicluster-tests_istio

@ramaraochavali - No problem :) I'll close mine.

I don't think we should do it but want to make sure we get some feedback on this. I think it may be fine but TOC has asked to be looped into any behavioral changes.
I think in the meantime you can use EnvoyFilter?
 apiVersion: networking.istio.io/v1alpha3
 kind: EnvoyFilter
 metadata:
   name: h2-control
 spec:
   configPatches:
   - applyTo: NETWORK_FILTER
     match:
       listener:
         filterChain:
           filter:
             name: "envoy.filters.network.http_connection_manager"
     patch:
       operation: MERGE
       value:
         typed_config:
           "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
           http_protocol_options:
             header_key_format:
               stateful_formatter:
                 name: preserve_case
                 typed_config:
                   "@type": type.googleapis.com/envoy.extensions.http.header_formatters.preserve_case.v3.PreserveCaseFormatterConfig

I would also like to know if there is any performance cost associated - the word "stateful" may be scaring me a bit

/test all

I think it may be fine but TOC has asked to be looped into any behavioral changes.

Fair enough.  We can default it to false if there is a concern for behavior change.

I would also like to know if there is any performance cost associated - the word "stateful" may be scaring me a bit

There will be some memory overhead as it stores the keys
https://github.com/envoyproxy/envoy/blob/main/source/extensions/http/header_formatters/preserve_case/preserve_case_formatter.h#L20 and minimal perf overhead to look at this. This totally depends on how many headers  a request has , but the general case may be fine.

I think in the meantime you can use EnvoyFilter?

With that Envoy filter Istio sending this to Envoy and Envoy complaining about header_format missing - So I was wondering it is a proto.Merge issue.
http_protocol_options {
  header_key_format {
  }
}

Closing the loop on the filter discrepancy between what @howardjohn  is seeing on master and what I am seeing on 26614eba5d4485965d9ed26ddea8779aa591d86f - the SHA I am testing does not have the updated go control plane and hence does not know about this type github.com/envoyproxy/go-control-plane/envoy/extensions/http/header_formatters/preserve_case/v3.
`

@ramaraochavali: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
e5d7ab43650559f1c553b9a435f718498e2c9f5a
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-pilot-k8s-tests_istio

I asked for this to not be merged until we look into this more. I already have it on the TOC agenda for today, depending on the outcome we may need to revert - just a heads up.
I talked with Rama and got the EnvoyFilter solution so I think they are unblocked anyhow (?)

Yes. If TOC feels change in behaviour is not acceptable, we can change the default to false?

My vote is to preserve the old behavior by default and expose this as an opt-in

We didn't get a chance to discuss with the full TOC but since the initial vote is -1 I think we may want to revert. There is an envoyfilter solution for those that require applications that do not follow the HTTP spec to use istio's HTTP support.

John when you mean revert - you meant default value right? I prefer having
this default off .
On Tue, 25 May 2021 at 1:06 AM, John Howard @.***>
wrote:

We didn't get a chance to discuss with the full TOC but since the initial
vote is -1 I think we may want to revert. There is an envoyfilter solution
for those that require applications that do not follow the HTTP spec to use
istio's HTTP support.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/33030#issuecomment-847285065, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AEDINKGDROVKD2M63I5YR3LTPKTFHANCNFSM45KM35DA
.

Agree about EnvoyFilter -> MeshConfig or first class API if it makes sense.
@ramaraochavali i didn’t get a satisfactory answer before but are there any security concerns for users who turn this on w.r.t header based AuthZ matches?
I’m still debating default on/off but keeping an ENV var is not optimal either way.

I am not against MeshConfig API.  However I do think this is an interim option (not a longer term option for many because of legacy clients migrating to newer stack) and also could not think of any downside of defaulting this to true in longer term in Istio (except for the behaviour change if any for existing users - which I really doubt because the newer http1.1 clients would any ways be sending headers in lower case already so there is no issue with them and it does not apply http2 at all).
So I felt env var approach may be better here so that we can just remove it when we default to true for few releases and no one complains about it. However if every one thinks we should go via API - Let us do that.

i didn’t get a satisfactory answer before but are there any security concerns for users who turn this on w.r.t header based AuthZ matches?

@nrjpoddar  I could not think of any.

This can be a temp solution before the  API lands on,  need to turn it off by default to make it safe.

If API is preferred and since we all agree that defaulting to false is better option, let us quickly converge on API https://github.com/istio/api/pull/2004. I can change the current implementation to use API instead of completely reverting it.

I am also seeing ACK ERROR 764f7f92e61f.vm-namespace-12 Internal:Error adding/updating listener(s) <IP_ADDRESS>_80: Proto constraint validation failed (HttpConnectionManagerValidationError.HttpProtocolOptions: ["embedded message failed validation"] | caused by Http1ProtocolOptionsValidationError.HeaderKeyFormat: ["embedded message failed validation"] | caused by field: "header_format", reason: is required): stat_prefix: "outbound_<IP_ADDRESS>_80" as well

Is that VM running old version of proxy?


Ability to use PUT requests for bot stats

AFAIK BotBlock will only allow POST requests for bot stats. dlist.gg uses PUT requests, resulting in a 405: Method not allowed.

Moving this over to api-worker as the first step will be implementing support for this in the code before we can start tracking data for it.


Conditions localization and updates

Fixed issue with stunned and fatigued at the same time by making the labels different. Applied to Poisoned too because it's the same penalty.

Added/Restored localization to condition dialog labels.
Added an effect to Engaged: The actor can't make ranged attacks unless the weapon has Pistol quality.

I believe this falls outside of how I'm willing to localize effects, as generally the scripts are redefined by the module if the localization keys needed aren't present

Just localized it because it was how it used to be. No problem.

Reverted localizations. Also added an activateScript to Poisoned.


Re-name syllabus sessions to "topics", drop numbers and re-order

Just some clean-up before we get the team looking at the topic list. Belongs to epic #23

This and #24 are now done, and the topic list seems fairly stable - enough to get the projects started! ( #50 )


feat: redesign env switcher

Fix #241

:tada: This PR is included in version 4.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Pin sass to version 1.64.1

Description

Issue Ticket Number

Fixes
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Fix or improve the documentation
[ ] This change requires a documentation update

Checklist

[ ] I have followed the contributing guidelines of this project as mentioned in CONTRIBUTING.md
[ ] I have created an issue ticket for this PR
[ ] I have checked to ensure there aren't other open Pull Requests for the same update/change?
[ ] I have performed a self-review of my own code
[ ] I have run tests locally to ensure they all pass
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes needed to the documentation

👷 Deploy Preview for radiant-gecko-9518c7 processing.

Name
Link

🔨 Latest commit
ab2c20835ca6d166e7daa7cc443c765e818a3958

🔍 Latest deploy log
https://app.netlify.com/sites/radiant-gecko-9518c7/deploys/64bfbf83386cff0008e22240


Google Cloud Platform - check if import map exist and create if it does not

Part of #98
Tested that, initially, if you do not have an import map in your bucket the server breaks.
After adding the changes, the server run fine, import map is created and additional runs do not reset the import map.

Released in https://github.com/single-spa/import-map-deployer/releases/tag/v4.3.0


support esm, cjs modules #164 and remove bower support

is there a way we can do this without duplicating the code?

what code duplication are you referring to?

.cjs.js, .esm.js, are all basically clones of the main js file. Can we do this as a build step?

So more like a umd bundle? We could do this, though the downside is environments that support es modules natively won't actually have an es module. We could do 2 builds maybe? umd and esm ?

This is looking good. Do we have to leave dialog-polyfill.js as part of the source, or could we instead only build it when we're deploying to NPM?
Mostly again I worry about duplication. At least we'd like to maybe put a comment at the top of dialog-polyfill.js saying that this is not canonical, edit index.js for actual changes.

another thing we could do is restructure the project like this:

src/index.js  <- intuitively seems like the entry point
dist/dialog-polyfill.js <- the built version used in browser and node environments.

dist/ seems to be a common convention for "this is a built file"

@samthor if you have follow up thoughts happy to make changes to move this along. 👍

I'm sorry for dragging my heels on this.
My only major comment is that I'd either like:

to note in the README somewhere that dialog-polyfill.js is purely generated as part of a build process
to not include it at all—since users of this are now getting it from NPM, the artefact will appear there.

I've pushed a commit which I believe resolves all of the above feedback. I've also signed the cla (though I'm not sure why it's still showing that I haven't. Maybe it just takes a moment to update.)
I think it might be valuable to outline all of the supported install vectors:

directly include the script as a global in an html document

<script src="node_modules/dialog-polyfill/dialog-polyfill.js"></script>

from a script module

<script type="module">
  import dialogPolyfill from './node_modules/dialog-polyfill/dialog-polyfill.esm.js';
  // ...
</script>

a nodejs script, or a bundler that operates on commonjs (browserify)

const dialogPolyfill = require('dialog-polyfill');

a script run through a bundler capable of operating on esm (rollup, webpack, node -r esm script.js)

import dialogPolyfill from 'dialog-polyfill';

deno (which is awesome by the way https://deno.land )

import dialogPolyfill from `https://raw.githubusercontent.com/GoogleChrome/dialog-polyfill/master/dialog-polyfill.esm.js`;

The current state of this PR should support all of these (and I'm assuming we want to.) Open for suggestions, I just wanted to make these use cases more up-front and explicit.

There's now three copies of the same code again and no clear indication in the README which one is the source of truth.
Again, I'd much rather just have one file here and only generate the needed output as an artefact of we upload to NPM.

I'd much rather just have one file here and only generate the needed output
as an artefact of we upload to NPM.

I understand the utility of having one source file so it's clear what the "source of truth" is. The problem is, there are some install vectors that have nothing to do with npm.  script global inclusion for example. People that embed dialog-polyfill in their website without using any package manager are likely to just clone your repo, or just copy the file directly off of the github page. In that case no npm build step runs, and the source of truth file can't be included.
See my above comment that attempts to list all known install vectors. Are some of these ones we don't care about?
@samthor again, I'm happy to make changes but the problem is there are a lot of people weighing in with feedback, and some of it is conflicting. If you give me explicit changes that you want, I'll make them.

some possible solutions to the multiple source file copy concern:

dialog-polyfill.esm.js can be eliminated. It is a byte-for-byte copy of index.js
add a CONTRIBUTING section to the readme saying something along the lines of "please modify index.js, it is the source of truth that other files are generated from"
add a commented banner at the top of the generated files saying something along the lines of // this file is generated from index.js, please modify that instead and rebuild with "npm run build"

I'm happy with the first two changes. The last is redundant as folks will
hopefully see the README.
On Wed, 27 Feb 2019 at 13:59, Mike Reinstein<EMAIL_ADDRESS>wrote:

some possible solutions to the multiple source file copy concern:

dialog-polyfill.esm.js can be eliminated. It is a byte-for-byte copy
of index.js
add a CONTRIBUTING section to the readme saying something along the
lines of "please modify index.js, it is the source of truth that other
files are generated from"
add a commented banner at the top of the generated files saying
something along the lines of // this file is generated from index.js,
please modify that instead and rebuild with "npm run build"

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/GoogleChrome/dialog-polyfill/pull/176#issuecomment-467704377,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHRkHq6yLr9gGdn6RoXBE56TFPi72bAks5vRfSOgaJpZM4agcOP
.

we could also create a dist/ directory and put all the built files in that. I feel like dist/ is pretty well understood to mean "these files are the result of a build process", and we could put something in the readme saying that explicitly

Yes, that's even clearer. I'd be very happy with that. 👍
On Wed, 27 Feb 2019 at 14:12, Mike Reinstein<EMAIL_ADDRESS>wrote:

we could also create a dist/ directory and put all the built files in that. I feel like dist/ is pretty well understood to mean "these files are the result of a build process", and we could put something in the readme saying that explicitly
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

As per above, I think the dist solution is best even if it ends up being a byte-for-byte copy in the case of the ESM copy because again, if the source ever splits into multiple files or comes to rely on third party dependencies, consumers of the ESM distribution won't have a breaking change to change paths to point to a bundled distribution script. It is already anticipated and there.

Yeah, if we have a dist folder I'm happy to have the duplication if you
think it's important. It's a clear artefact at that point.
On Wed, 27 Feb 2019 at 14:17, Brett Zamir<EMAIL_ADDRESS>wrote:

As per above, I think the dist solution is best even if it ends up being
a byte-for-byte copy in the case of the ESM copy because again, if the
source ever splits into multiple files or comes to rely on third party
dependencies, consumers of the ESM distribution won't have a breaking
change to change paths to point to a bundled distribution script. It is
already anticipated and there.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/GoogleChrome/dialog-polyfill/pull/176#issuecomment-467707787,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHRkMDrHcziho06g8p3fwzKzVN2yUUwks5vRfjWgaJpZM4agcOP
.

do we want to build the contents of dist/ only before publishing to npm, or would we also like to do it just before committing? I could add a git pre-commit hook if this is desirable.

Before publishing. I don't want dev work to generate artefacts in PRs.
Unless this is breaking with traditional precent.
On Wed, 27 Feb 2019 at 14:23, Mike Reinstein<EMAIL_ADDRESS>wrote:

do we want to build the contents of dist/ only before publishing to npm, or would we also like to do it just before committing? I could add a git pre-commit hook if this is desirable.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

it's up to you, different projects handle it differently. Though I will say the one nice thing about doing it on pre-commit; if the build step fails, the commit also fails.

I personally don't like pre-commit hooks in case I am working on a fork which is partially complete, but I'm not working on the source.
I do, however, like including the dist files in source at least for publish time, as I think it can be a good guard against inadvertent changes which affect the build--e.g., if a change shouldn't affect the build but it does.
But feel free to ignore--just my own prefs...

there are definitely pros and cons to the pre-commit hook(s). I don't feel that strongly about it either way.
btw when we get these details sorted let me know and I'll re-submit a clean PR.

have been going back and forth between dist/dialog-polyfill.js vs dist/dialog-polyfill.umd.js.
Also, dialog-polyfill.css vs dist/dialog-polyfill.css
thoughts?

Since this is such a big change (and will probably be a 0.x bump) let's
copy (?) the CSS as part of build, to dist (giving us more options in
future). It matches the model of mirroring the module code.
And I don't see the value in ".umd", it's confusing for folks using it the
old-style way.
On Wed, 27 Feb 2019 at 14:37, Mike Reinstein<EMAIL_ADDRESS>wrote:

have been going back and forth between dist/dialog-polyfill.js vs
dist/dialog-polyfill.umd.js.
Also, dialog-polyfill.css vs dist/dialog-polyfill.css
thoughts?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/GoogleChrome/dialog-polyfill/pull/176#issuecomment-467711310,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHRkJD8fjel5uv77YcZ3yzqx3ecf2g2ks5vRf2VgaJpZM4agcOP
.

I believe this should resolve all the open items. Let me know if I've missed anything.

The build:esm and build:umd steps are the wrong way around. Otherwise LGTM


[Enhancement] array_function support all type [PR-1]

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #

array_slice support all type, keep the original implementation
array_distinct support decimalV3
Fix var function check type bug

Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[ ] 2.5
[ ] 2.4
[ ] 2.3
[ ] 2.2

run starrocks_fe_unittest

[FE PR Coverage Check]
:heart_eyes: pass : 13 / 13 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/catalog/FunctionSet.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
9
9
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 13 / 13 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/catalog/FunctionSet.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
9
9
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 13 / 13 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/catalog/FunctionSet.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
9
9
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 18 / 18 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/catalog/FunctionSet.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/catalog/Type.java
3
3
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
11
11
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 18 / 18 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/catalog/FunctionSet.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/catalog/Type.java
3
3
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
11
11
100.00%
[]

Ok, boys, I recorde your questions in https://github.com/StarRocks/starrocks/issues/17669, and will handle it later

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test


Deadlock when updating SagaInstance using EntityFramework

Hello,
I'm getting deadlocks in db for updates to the sagainstance. Using MassTransit 3.3.3.
Deadlock report from Sql Server:
<deadlock>
 <victim-list>
  <victimProcess id="process1c4089c28" />
 </victim-list>
 <process-list>
  <process id="process1c4089c28" taskpriority="0" logused="0" waitresource="KEY: 54:72057594044219392 (d9e2fd965031)" waittime="4887" ownerId="142311259" transactionname="user_transaction" lasttranstarted="2016-06-22T15:46:14.060" XDES="0xed112d90" lockMode="X" schedulerid="2" kpid="11132" status="suspended" spid="124" sbid="0" ecid="0" priority="0" trancount="2" lastbatchstarted="2016-06-22T15:46:14.083" lastbatchcompleted="2016-06-22T15:46:14.060" lastattention="1900-01-01T00:00:00.060" clientapp=".Net SqlClient Data Provider" hostname="" hostpid="10736" loginname="" isolationlevel="serializable (4)" xactid="142311259" currentdb="54" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
   <executionStack>
    <frame procname="adhoc" line="1" stmtstart="158" stmtend="434" sqlhandle="0x0200000017aafc19faaded28d9137caa67d512180bd4d2030000000000000000000000000000000000000000">
unknown    </frame>
    <frame procname="unknown" line="1" sqlhandle="0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000">
unknown    </frame>
   </executionStack>
   <inputbuf>
(@0 varbinary(max) ,@1 varbinary(max) ,@2 uniqueidentifier,@3 uniqueidentifier)UPDATE [dbo].[ShopIndexingSagaInstances]
SET [FetchedPages] = @0, [IndexedPages] = @1, [TimeoutTokenId] = @2
WHERE ([CorrelationId] = @3)
   </inputbuf>
  </process>
  <process id="process1ee480108" taskpriority="0" logused="0" waitresource="KEY: 54:72057594044219392 (d9e2fd965031)" waittime="4898" ownerId="142311258" transactionname="user_transaction" lasttranstarted="2016-06-22T15:46:14.053" XDES="0x1b9bd2d90" lockMode="X" schedulerid="1" kpid="4152" status="suspended" spid="56" sbid="0" ecid="0" priority="0" trancount="2" lastbatchstarted="2016-06-22T15:46:14.073" lastbatchcompleted="2016-06-22T15:46:14.060" lastattention="1900-01-01T00:00:00.060" clientapp=".Net SqlClient Data Provider" hostname="" hostpid="10736" loginname="" isolationlevel="serializable (4)" xactid="142311258" currentdb="54" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
   <executionStack>
    <frame procname="adhoc" line="1" stmtstart="158" stmtend="434" sqlhandle="0x0200000017aafc19faaded28d9137caa67d512180bd4d2030000000000000000000000000000000000000000">
unknown    </frame>
    <frame procname="unknown" line="1" sqlhandle="0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000">
unknown    </frame>
   </executionStack>
   <inputbuf>
(@0 varbinary(max) ,@1 varbinary(max) ,@2 uniqueidentifier,@3 uniqueidentifier)UPDATE [dbo].[ShopIndexingSagaInstances]
SET [FetchedPages] = @0, [IndexedPages] = @1, [TimeoutTokenId] = @2
WHERE ([CorrelationId] = @3)
   </inputbuf>
  </process>
 </process-list>
 <resource-list>
  <keylock hobtid="72057594044219392" dbid="54" objectname="MessageQueueCoordinator.dbo.ShopIndexingSagaInstances" indexname="PK_dbo.ShopIndexingSagaInstances" id="lock16f6fb380" mode="S" associatedObjectId="72057594044219392">
   <owner-list>
    <owner id="process1ee480108" mode="S" />
    <owner id="process1ee480108" mode="X" requestType="convert" />
   </owner-list>
   <waiter-list>
    <waiter id="process1c4089c28" mode="X" requestType="convert" />
   </waiter-list>
  </keylock>
  <keylock hobtid="72057594044219392" dbid="54" objectname="MessageQueueCoordinator.dbo.ShopIndexingSagaInstances" indexname="PK_dbo.ShopIndexingSagaInstances" id="lock16f6fb380" mode="S" associatedObjectId="72057594044219392">
   <owner-list>
    <owner id="process1c4089c28" mode="S" />
    <owner id="process1c4089c28" mode="X" requestType="convert" />
   </owner-list>
   <waiter-list>
    <waiter id="process1ee480108" mode="X" requestType="convert" />
   </waiter-list>
  </keylock>
 </resource-list>
</deadlock>

Here is my configuration for the state this is happening in:
this.During(this.IndexingProducts,
	this.When(this.ProductsFetched)
		.Then(this.CopyMessageDataToInstance)
		.Schedule(this.SagaTimeout, this.CreateSagaTimeoutEvent)
		.ThenAsync(this.SendIndexProductsCommand),
	this.When(this.ProductsIndexed, this.HasNotIndexedAllPages)
		.Schedule(this.SagaTimeout, this.CreateSagaTimeoutEvent),
	this.When(this.ProductsIndexed, this.HasIndexedAllPages)
		.TransitionTo(this.ActivatingIndex)
		.Then(this.UpdateStateChangedTimestamp)
		.Schedule(this.SagaTimeout, this.CreateSagaTimeoutEvent)
		.ThenAsync(this.SendActivateIndexCommand));

The only visible problems this is generating is that some of the scheduled timeouts is not canceled.
Please inform me if you need any more information from me.
EDIT:
I've seem to have found the potential issue. In EntityFrameworkSagaRepository on line 96 it will take a shared lock on the select. Both transactions will then keep the lock and block each other from committing. Shouldn't this lock be shared?

How are both getting locks on the same row? That database should be using serializable isolation level, to avoid this.

It is using serializable. Our DBA said that line 96 will make a deadlock. Im not really an expert, but I'll see if he can fill in some more info.

According to the deadlock graph it looks like both processes hold the shared lock (and then block each other from taking an X-lock) but i can't reproduce this when running a simple example in two sessions in SSMS.
From MSDN:
SERIALIZABLE
Specifies the following:
Statements cannot read data that has been modified but not yet committed by other transactions.
No other transactions can modify data that has been read by the current transaction until the current transaction completes.
Other transactions cannot insert new rows with key values that would fall in the range of keys read by any statements in the current transaction until the current transaction completes.

I managed to reproduce it quite easy now by running the following in two different sessions
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRAN

SELECT * FROM Person where Id = 1;

WAITFOR DELAY '00:00:10';

Update Person set Active = 1 where Id = 1;

COMMIT TRAN

CREATE TABLE [Person](
	[Id] [bigint] NOT NULL,
	[Active] [bit] NOT NULL
 CONSTRAINT [PK_Person] PRIMARY KEY CLUSTERED
(
  [Id] ASC
) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
INSERT INTO [Person] (Id,Active) VALUES (1, 0);
INSERT INTO [Person] (Id,Active) VALUES (6, 0);
GO

One of them will deadlock
(1 row(s) affected)
Msg 1205, Level 13, State 51, Line 8
Transaction (Process ID 53) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

I tested it out with a simple select to lock the row like this:
var tableName = ((IObjectContextAdapter)dbContext).ObjectContext.CreateObjectSet<TSaga>().EntitySet.Name;
dbContext.Set<TSaga>().SqlQuery($"select * from {tableName} WITH (UPDLOCK, ROWLOCK) WHERE CorrelationId = @p0", sagaId).ToList();

This removed all of the deadlocks.

Some additional info about this
http://stackoverflow.com/questions/27347730/serializable-transaction-deadlock
http://stackoverflow.com/questions/33811505/deadlock-with-only-one-resource-and-isolation-level-serializable

@stromblom isn't it ms sql server specific only hack?

@NikGovorov yes it is. I forgot that sql differ from vendor to vendor. Which sql vendors are supported out of the box for MT? Maybe @phatboyg can pitch in here? =)

@stromblom I assume that the whole idea of supporting NHibernate and EntityFramework is to support any database that are supported by the ORMs.

@NikGovorov I assume as much myself. But the only thing EntityFramework supports without tinkering with it is MS SQL. So my question was more in the line of "when I fix my code, which db types should be supported out of the box."

Got a new deadlock going now.  But I'm having problems finding the code that generates the lock.

hm, curious is that is related to the issue I'm describing in here: https://groups.google.com/d/msg/masstransit-discuss/NIfam7VqTPw/NKRis0A8AAAJ

@snekbaev Yeah, it seems like there is a problem with concurrency inside the same saga. Could you find the deadlock report perhaps?

Looking into this a bit more, it very much seems like the second type of lock I'm getting is in EntityFrameworkSagaRepository.SendQuery. And this is consisten with what @snekbaev is experiencing as well. I propose adding a full table lock here instead of the existing range lock the .Where will generate. I'll look into it tomorrow if I got time.

Isn't EF really limited on how you can actually lock things? It seems the workarounds are all along the lines of "just use this SQL and it will make it do the right thing."

Yes. But, other than using nhibernate I see no other solution, and to be honest it looks like you should have the same deadlock situation there as well.
I was thinking about making a dapper implementation. But that would probably take quite some time.

@stromblom, I've side tracked to another functionality and wasn't experimenting with sagas much lately. However, in my last tests I switched to MongoDB persistence and still was experiencing issues. Retry helps to some extent but still not as reliable as I wished and message processing order issue remains. Basically what I wanted to achieve is an ordered and reliable handling of sagas inside a single saga processing service. Let me elaborate. For the test case I queued up around 60 messages: 30 ItemAddedToCart and 30 OrderSubmitted. Each pair had the same correlation id. I ran the saga consumer with various settings, obviously if I set concurrency to 1 everything is good, but that's not what I wanted. What I wanted is a parallel processing of non related saga messages, however, only 1 message belonging to the same saga should be processed at the same time. I didn't want OrderSubmitted message to be processed before or during ItemAddedToCart was being processed. Queue had the ItemAddedToCart added first, so process it and only then move to the next message correlated to that saga. I tried the partioner, but to my results the same bin contents is still processed in parallel. So I either misconfigured or my use case is not supported. Guess only @phatboyg could really tell :)

@phatboyg I'm currently working on a solution of locking the whole table in SendQuery.
What are your thoughts on having MsSql specific things in the EF repo?

I'm pretty sure EF is mostly SQL Server, anyway.

So I've tried my code out in our test environment, went over around 25000 sagas and got no deadlocks. So looks like it is working as intended.

So the code related to this has been merged, should we consider this issue closed?


[BUG]: VM Launch Mods #3 unhandled exception

When I try to start a freshly cloned copy of the latest commit (45b01082db79cb9803648a5350ffa225b37790a0) on Debian, the VMM fails to launch.
Make output:
make -C /lib/modules/4.9.0-1-amd64/build M=/home/carter/hypervisor/bfdrivers/src/arch/linux clean
rm -f ../../../../bfelf_loader/src/bfelf_loader.o
rm -f ../../../../bfelf_loader/src/.bfelf_loader.o.cmd
rm -f ../../common.o
rm -f ../../.common.o.cmd
make -C /lib/modules/4.9.0-1-amd64/build M=/home/carter/hypervisor/bfdrivers/src/arch/linux modules
  CC [M]  /home/carter/hypervisor/bfdrivers/src/arch/linux/entry.o
nasm -f elf64 -o /home/carter/hypervisor/bfdrivers/src/arch/linux/vmcall.o /home/carter/hypervisor/bfdrivers/src/arch/linux/vmcall.asm
  CC [M]  /home/carter/hypervisor/bfdrivers/src/arch/linux/platform.o
  CC [M]  /home/carter/hypervisor/bfdrivers/src/arch/linux/../../common.o
  CC [M]  /home/carter/hypervisor/bfdrivers/src/arch/linux/../../../../bfelf_loader/src/bfelf_loader.o
  LD [M]  /home/carter/hypervisor/bfdrivers/src/arch/linux/bareflank.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/carter/hypervisor/bfdrivers/src/arch/linux/bareflank.mod.o
  LD [M]  /home/carter/hypervisor/bfdrivers/src/arch/linux/bareflank.ko
insmod ./bareflank.ko
Caught unhandled exception:
    - what(): parse error - unexpected ']'
Makefile:181: recipe for target 'load' failed
make[1]: *** [load] Error 1
Makefile:5: recipe for target 'load' failed
make: *** [load] Error 2

029d20b54c88692258ea0a5ad2468a8f179a6caf works fine.

Yes.... this is my bad. I have a  fix for this. Change your vmm.modules to the following:
{
    "modules" :
    [
        "%BUILD_ABS%/sysroot_vmm/x86_64-vmm-elf/lib/libc++.so.1.0",
        "%BUILD_ABS%/sysroot_vmm/x86_64-vmm-elf/lib/libc++abi.so.1.0",
        "%BUILD_ABS%/sysroot_vmm/x86_64-vmm-elf/lib/libc.so",
        "%BUILD_ABS%/makefiles/bfunwind/bin/cross/libbfunwind.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/crt/bin/cross/libcrt.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/debug_ring/bin/cross/libdebug_ring.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/entry/bin/cross/libentry.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/exit_handler/bin/cross/libexit_handler.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/intrinsics/bin/cross/libintrinsics.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/memory_manager/bin/cross/libmemory_manager.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/misc/bin/cross/misc",
        "%BUILD_ABS%/makefiles/bfvmm/src/pthread/bin/cross/libpthread.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/serial/bin/cross/libserial.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/syscall/bin/cross/libsyscall.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/vcpu/bin/cross/libvcpu.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/vcpu_factory/bin/cross/libvcpu_factory.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/vmcs/bin/cross/libvmcs.so",
        "%BUILD_ABS%/makefiles/bfvmm/src/vmxon/bin/cross/libvmxon.so"
    ]
}

Sorry about that, we do not have a way to test this part out with Travis so it snuck through

FYI, If you have your own extension, you will need to update it to match what I have here. The reason this changed is we no longer rely on the libbfc repo, and instead, the syscall / pthread code is now in our main repo which is a LOT cleaner.

Oh... and just for reference, the reason it did that is I had a , at the end of the JSON array, which the parser does not like since you cannot have trailing ,. If you fix that you will notice that I also messed up the pthread line as it will complain that it cannot find pthread as it should have been called libpthread.so

Works. Thanks for the quick response.

@carter-yagemann No problem. Sorry I committed that error. A colleague of mine is working on a Jenkins server + VMWare Fusion (with nested enabled) to test the hypervisor and ensure all of the extensions compile and run, so hopefully in the future an issue like this will be caught quickly and fixed.

Fixed


shapelib: 1.4.0 -> 1.4.1

Semi-automatic update. These checks were performed:

built on NixOS
ran /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/dbfcreate -h got 0 exit code
ran /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/dbfcreate --help got 0 exit code
ran /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/dbfcreate help got 0 exit code
ran /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/dbfinfo -h got 0 exit code
ran /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/dbfinfo --help got 0 exit code
ran /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/dbfinfo help got 0 exit code
found 1.4.1 with grep in /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1
found 1.4.1 in filename of file in /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1

cc "@ehmry"

Success on x86_64-linux (full log)
Partial log (click to expand)
shrinking /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/Shape_PointInPoly
shrinking /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/shpcat
shrinking /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/shptreedump
shrinking /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin/shpadd
shrinking /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/lib/libshp.so.2.0.2
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/lib  /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1/bin
patching script interpreter paths in /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1
checking for references to /tmp/nix-build-shapelib-1.4.1.drv-0 in /nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1...
/nix/store/dc39wljsqrwrdnz2c0072wsch1qdf03v-shapelib-1.4.1

Success on x86_64-darwin (full log)
Partial log (click to expand)
libtool: install: /nix/store/6b3jzg40vibmrjg85i46vvidyy915428-coreutils-8.29/bin/install -c .libs/shpproj /nix/store/28f9ibd0cl17x9ys3yph6mxjkn3g346f-shapelib-1.4.1/bin/shpproj
libtool: install: /nix/store/6b3jzg40vibmrjg85i46vvidyy915428-coreutils-8.29/bin/install -c .libs/shpwkb /nix/store/28f9ibd0cl17x9ys3yph6mxjkn3g346f-shapelib-1.4.1/bin/shpwkb
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/private/tmp/nix-build-shapelib-1.4.1.drv-0/shapelib-1.4.1/contrib'
make[1]: Leaving directory '/private/tmp/nix-build-shapelib-1.4.1.drv-0/shapelib-1.4.1/contrib'
post-installation fixup
strip is /nix/store/4sdh09gmvl15cy0zb6i7mbvxh5syz206-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/28f9ibd0cl17x9ys3yph6mxjkn3g346f-shapelib-1.4.1/lib  /nix/store/28f9ibd0cl17x9ys3yph6mxjkn3g346f-shapelib-1.4.1/bin
patching script interpreter paths in /nix/store/28f9ibd0cl17x9ys3yph6mxjkn3g346f-shapelib-1.4.1
/nix/store/28f9ibd0cl17x9ys3yph6mxjkn3g346f-shapelib-1.4.1

Success on aarch64-linux (full log)
Partial log (click to expand)
shrinking /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1/bin/shpadd
shrinking /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1/bin/dbfdump
shrinking /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1/bin/dbfcreate
shrinking /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1/bin/dbfadd
shrinking /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1/lib/libshp.so.2.0.2
strip is /nix/store/lvx1acn1ig1j2km8jds5x3ggh3f2wa8v-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1/lib  /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1/bin
patching script interpreter paths in /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1
checking for references to /build in /nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1...
/nix/store/w6gi1d802bqngg7abzml9g88z1vp3nfn-shapelib-1.4.1


more map options (satellite, 3d, etc.)

we should look in to more map display options such as satellite, flyover, 3d.... whatever else apple maps offers.

Began working on this here


[Feature] Basic external ref support

Closes #3
tl;dr Add basic support for "external" $ref fields that point to files.
When expanding fields, if field's ref is pointing to a .json file, resolve it, and merge all the types it references into the current parent Expander.
The changes in this PR do this by:

Adding field to Expander that holds the directory of the file from which its Schema is derived; this is re-used for reading relative referenced files.
When expanding a field that references an external file,  create a child Expander to expand the referenced Schema and merging its data (types and further referenced Schemas) into the current one. Do this recursively.
Maybe using a tad too many clone()s...

To be honest, I need this for another project I plan on working on, so mostly hacked around until the tests passed without paying too much attention to efficiency. Here to learn so suggestions are welcome :)
Things not handled thus far:

Circular dependencies (compiler stack overflows)
Namespacing of types (model name clashes just fail the compile)

A relatively easy solution would be to add the ability to hook into the generation and allow it to replace the type being generated for any given field. Then override the type of any fields holding a Schema and make it hold an Arc which would give us cheap clones.

Can you give a concrete example of how to do what you’re suggesting? You say easy but it doesn’t sounds trivial given the bootstrapping of the structs.
I could just wrap all the Schemas in Expander in Arc (or even Rc since we’re not threaded?) quite easily though I think and get cheap clones.

I gave the reference-counted Schema idea a shot: it turns out we also need to at least RefCell the Schema because merge_all_of needs a mutable Schema
https://github.com/Marwes/schemafy/blob/81ba451290820b421a8831afced0eccb3d220a9e/schemafy_lib/src/lib.rs#L140-L165
called from
https://github.com/Marwes/schemafy/blob/81ba451290820b421a8831afced0eccb3d220a9e/schemafy_lib/src/lib.rs#L323
Not sure at this point if I'm getting further and further away from what you're envisioning, so it's the last commit and can easily be reverted if you just want to drive bit that yourself.

I opened another PR that contains a commit https://github.com/Marwes/schemafy/pull/32/commits/23efd1a3bee0bdd57c4ee339286a6f4a792636ba that might be closer to what you envisioned, please take a look.

@Marwes I'm closing this because I think #32 is the way forward with cheaper clones due to the wrapping.


how to customize cluster domain

hi there,
Cool project, thanks for the effort done here.
Is there a way to customize cluster domain name via ansible playbooks?
Regards,
Julio

See https://github.com/alvistack/ansible-role-kube_master/blob/master/defaults/main.yml#L26-L27
# Kubernetes cluster name.
kube_cluster_name: "{{ hostvars[groups['kube_master'][0]].ansible_machine_id | to_uuid }}"

You could simply override it with any else static hard-coded value.


Interchange not working

I'm trying to use Interchange plugin to change the background image on a div dependant on the screen size but I'm unable to figure out why it's not working correctly.
Foundation is initialised fine and when I resize the screen data-uuid is updated, but there is no background-image style.
I've created a jsfiddle to show you what I'm doing:

https://jsfiddle.net/rb90qq08/6/
This is exactly how it should be according to the documentation.

The Foundation forum would be a great place to seek some help with troubleshooting what might be going on here. If it turns out to be a defect, you'll have some more info to provide here which will help prioritize and get it fixed.
We're trying to keep the issue list focused on confirmed defects in Foundation, so any help everyone can provide to differentiate between usage / troubleshooting and actual defects is appreciated!

I have posted there, but there's little to no activity. I added a (default) to data-interchange. But now the entire div just gets removed. Suppose I'll just have to look for an alternative to interchange.


Line 17: [Warning] Invalid file link:(~/concepts/bots/bot-conversations/bots-conv-channel).
Line 18: [Warning] Invalid file link:(~/concepts/bots/bot-conversations/bots-conv-personal).
Line 20: [Warning] Invalid file link:(~/resources/schema/manifest-schema).
Line 34: [Warning] Invalid file link:(~/concepts/bots/bots-notifications).
Line 48: [Warning] Invalid file link:(~/concepts/cards/cards-reference).
Line 56: [Warning] Invalid file link:(~/concepts/bots/bot-conversations/bots-message-format).
Line 59: [Warning] Invalid file link:(~/concepts/bots/bots-text-formats).
Line 61: [Warning] Invalid file link:(~/concepts/cards/cards-format).
Line 81: [Warning] Invalid file link:(~/concepts/bots/bots-notifications#channel-updates).
Line 142: [Warning] Invalid file link:(~/concepts/bots/bot-conversations/bots-conv-channel#mentions).
Line 152: [Warning] Invalid file link:(~/concepts/bots/bots-notifications#channel-updates).
Line 156: [Warning] Invalid file link:(~/concepts/bots/bots-notifications#team-name-updates).
Line 159: [Warning] Invalid file link:(~/concepts/bots/bots-notifications#channel-updates).
Line 196: [Warning] Invalid file link:(~/concepts/cards-actions).
Line 267: [Warning] Invalid file link:(~/concepts/bots/bot-conversations/bots-conv-proactive#starting-personal-conversations).
Line 268: [Warning] Invalid file link:(~/concepts/bots/bot-conversations/bots-conv-proactive#creating-a-channel-conversation).
Line 270: [Warning] Invalid file link:(~/concepts/bots/bot-conversations/bots-conv-proactive).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Error when trying siesta with Argo

Hi Paul,
I'm diving even deeper into siesta, and I try to make it work with Argo. But I get an error I couldn't work around. That's maybe not an issue, but merely my own failure to grasp all the details involved here, especially with the siesta blocks arguments.
Here is the code, in my subclass of Service:
    public init(withAPIVersion version: String = "1") {
        self.APIVersion = version
        super.init(baseURL: "http://api.arcsecond.io")

        self.configureTransformer("/\(self.APIVersion)/objects/*") {
            decode($0.content) as AstronomicalObject
        }
    }

I get the following at the line self.configureTransformer...:

Cannot invoke 'configureTransformer' with an argument list of type '(String, (_) -> _)'

The code for AstronomicalObject follows the exact same example given by Argo, in its simplest form:
public struct AstronomicalObject {
    public let name: String
}

public extension AstronomicalObject: Decodable {
    public static func decode(json: JSON) -> Decoded<AstronomicalObject> {
        return curry(AstronomicalObject.init)
            <^> json <| "name"
    }
}

I don't get this error when writing a "normal" init method for my struct, taking the json as parameter and doing all the parsing myself. But my goal is to use Argo precisely for that.
Any idea? Thanks in advance!

Ah, Swift type inference. It’s awesome … until it can’t figure something out.
I’m not familiar with Argo, but I’m guessing the problem is that its global decode(_:) method takes a type that is too broad for Swift to infer the input type of your transformer closure.
In these situations, my usual approach is to add explicit types until it works (or I get a more meaningful compile error), then trying removing them one by one to see which the compiler can infer. So maybe try something like this:
self.configureTransformer("/\(self.APIVersion)/objects/*") {
    decode($0.content as NSJSONConvertible) as AstronomicalObject
}

…or even the fully explicit version:
self.configureTransformer("/\(self.APIVersion)/objects/*") {
    (content: NSJSONConvertible, entity: Entity) -> AstronomicalObject
    return decode($0.content)
}

(Or should the content be JSON already? Not sure what Argo expects.)
This still may not work, but hopefully you’ll at least get a more helpful compiler error.

@onekiloparsec Did you ever get this working? If you did, let me know and we can find a place to drop an Argo example in the docs and/or Extensions/.

@pcantrell Sorry for the delay. The short answer is no. But I wanted to make a detailed answer after investigating a bit more.

Still struggling... I've updated to beta10, but I have an error message (also present with beta 8):
/Users/onekiloparsec/apps/arcsecond.swift/arcsecond/Arcsecond.swift:31:14: Cannot invoke 'configureTransformer' with an argument list of type '(String, () -> ())'
For some reason, subclassing Service and calling in the init method:
       self.configureTransformer("/\(self.APIVersion)/objects/*") {             //         }
produces this error (whatever I put in place of the //). Sorry for the trouble. I've re-read the documentation again, but I am probably missing something here.
The message says further more:
/Users/onekiloparsec/apps/arcsecond.swift/arcsecond/Arcsecond.swift:31:14: Expected an argument list of type '(ConfigurationPatternConvertible, requestMethods: [RequestMethod]?, atStage: PipelineStageKey, action: PipelineStage.MutationAction, onInputTypeMismatch: InputTypeMismatchAction, transformErrors: Bool, description: String?, contentTransform: (content: I, entity: Entity) throws -> O?)'

Down a Swift type rabbit hole!
I’ll have a chance to dig into this later this week. It certainly deserves attention — you won't be the last to hit it. If in the meantime you have time to prepare a minimal example project to reproduce the problem, that would be quite helpful.

Sure. here it is: SiestaSwiftTypeRabbitHole.zip

OK, digging into this…
This doesn’t work because the trailing closure doesn’t use any inputs and doesn’t return anything, so Swift can neither infer the input nor the output type of your transformer:
configureTransformer("/\(APIVersion)/objects/*") {
  //
}

Back to your original Argo example, there was clearly some problem going on where both Siesta and Argo’s use of generic left Swift unable to figure out what set of types would make the whole closure compile. I therefore tried isolating just the Argo call, like this:
let content = NSData()
let x: AstronomicalObject = decode(content)

…which gave a much more helpful compile error: decode(…) returns an optional. Aha! Armed with that knowledge, I tried this, which compiles:
configureTransformer("/\(APIVersion)/objects/*") {
    decode($0.content as NSData) as AstronomicalObject?
}

This version, which is more verbose but perhaps clearer about the structure of things, also compiles:
configureTransformer("/\(APIVersion)/objects/*") {
    (content: NSData, _) -> AstronomicalObject? in
    decode(content)
}

I only verified compilation here, not correct behavior. Let me know if this unties the type knot for you!

Thanks a lot for your insightful help. It compiles indeed. But somehow, I couldn't get anything out of it anymore. I mean, when I have the configureTransformer call uncommented, not only it is never called (hence decode is never called), but the resource observer closure isn't called either! Am I missing something?

Even stranger, whatever the presence of a custom transformer or not, the following makes my Xcode editor crashes:
    self.hd5980 = self.arcsecond.object("HD 5980")
    self.hd5980!.addObserver(owner: self) { (resource, event) in
        print("resource \(resource.typedContent())")
    }

where the object func is defined like this in the Service subclass:
public func object(name: String) -> Resource {
    return self.resource("/\(self.APIVersion)/objects/\(name)")
}

the resource observer closure isn't called either

This is a sign of something more that just contentTransformer trouble. Turn on the detailed logging and work through that line by line. It’s a little hard to wade through, but it will tell you exactly what’s going wrong.
self.hd5980!.addObserver(owner: self) { (resource, event) in
    print("resource \(resource.typedContent())")
}

Obviously the editor shouldn’t crash (!), but typedContent() will only work when its return type is constrained by its context. It means “conditional downcast to the type I expect.” That’s sort of the opposite of what you want; you’re looking for “just show me whatever type you have.” Try latestData?.content instead of typedContent().

Thanks for the typedContent(), it's a misuse of mine.
As for the contentTransformer, I see "The network connection was lost". I have no clue how to fix it. Maybe with a different value of keep-alive, but still, I don't see why this suddenly stop working just because I'm using a transformer in my client code.

I am not so sure how I did fix that issue, but I did. It seems that increasing TIMEOUT value in my server from 30 to 60 did the trick, and I now have something that goes through the decoding of Argo! The only additional change was to set the type of content in the closure to NSDictionary rather than NSData. Thanks for your help!

Glad you worked it out! It sounds like the network issue is outside of Siesta, but … at the very least, once the Swift 3 adventure is over, I'm keen to improve the readability of Siesta’s logging to help make these sorts of problems easier to diagnose.

The only additional change was to set the type of content in the closure to NSDictionary rather than NSData

Sounds like Siesta’s default JSON parsing is still in the parsing. No harm in that, although depending on what Argo does under the covers, it might parse a snerch faster if you let it do the parsing to. (Or maybe Argo uses NSJSONSerialization too and it’s all the same!)


"kfac.py"

thanks much for sharing the baselines. out of all the implementations, the kfac.py is really hard to follow. I was wondering if you can add some more high level comments to this file. Thanks again

There is also an implementation from the TensorFlow team here: https://github.com/tensorflow/kfac
Would it be an option for baselines to rely on their implementation instead?


Fix(Pipeline): Wait 30 seconds after deployment is done.

Based on a conversation with the ADT team, it has been suggested that we wait 30 seconds before making the first API call after the resources have been deployed and role-assignments have been performed to avoid permission issues during test run.

/azp run net - digitaltwins - tests


merge 'swift-DEVELOPMENT-SNAPSHOT-2019-05-26-a' into 'tensorflow'

Very few conflicts that I had to resolve manually:

They moved Serializer::writeDeclAttribute from one place in Serialization/Serialization.cpp to another place. We had added cases for differentiable and differentiating attribute. So I moved our cases into the new location.
They changed Serializer::writeType from a big switch into a visitor. We had made some modifications to some of the cases. So I ported our modifications to the visitor.

@swift-ci please clean test tensorflow


Linter: Dots after Return Consistency

Switch preference in linter and check for 100%

100% is probably not achievable, just follow the coding standard and check how applicable it is


No Desktop icon or app registration when disabling autostart

We are including Teams in our default Windows image however we do not want it to autostart automatically when a user first logs in.
This past week we updated the install to the latest MSI (Teams_windows_x64.msi - <IP_ADDRESS>57) and used the new option:
msiexec /i Teams_windows_x64.msi OPTIONS="noAutoStart=true"
Teams is installed for new users however Teams is not registered and cannot be found from via Win10 search and also there is no desktop icon for the user.
Ideally there needs to be a middle ground where we can install Teams per machine but choose to let the user initiate first launch.

@hippman257 Thank you for this feedback
It seems like a few things didn't work right. I want to get you over to Support. They have the resources to troubleshoot this with you and find a resolution.
Here is a quick video on opening a ticket: https://www.youtube.com/watch?v=puuOy0sEg1Y
If you don't mind please keep us posted here on the resolution and I really want to get this information you are discovering back into the docs.


native-base doesn't work

Seems to be a problem with react-native-vector-icons.
Somehow it gets rewritten to fetch the icons form an exponent repository, which then fails to find.

did you by chance run npm install @exponent/vector-icons --save-dev or yarn add @exponent/vector-icons inside your project?

@cyprusglobe I did install @exponent/vector-icons, although I had not done the --save-dev (i.e. it was not in package.json). However AFAIK that should not make a difference.
the error stems from the following line in Icon.js (part of native-base):
import Ionicons from 'react-native-vector-icons/Ionicons';

I repeated the same steps in a clean project and it still doesn't work, although I get a slightly different error:

Finally made these work. Several things were missing.
First, exp.json had to be modified to include "ttf" in assetExts.
That got me past the first few errors.
Next, the native-base package (the version listed in the original exponent blog post advertising working UI toolkits), depends on the roboto font for its default theme.
For that to work I needed the following wrapper on the app:
class Wrapper extends React.Component {
  state = { fontsAreLoaded: false };

  async componentWillMount() {
    await Exponent.Font.loadAsync({
      'Roboto_medium': require('native-base/Fonts/Roboto_medium.ttf')
    });
    this.setState({fontsAreLoaded: true});
  }

  render() {
    if (this.state.fontsAreLoaded)
      return <App/>;
    else
      return <Exponent.Components.AppLoading/>;
  }
}```

The blog post was misleading since it implied that native-base would work without any special tweaks beyond doing npm install native-base, this was not the case. Summarizing required steps were:

- Edit exp.json to tweak asset extensions
- Install exponent/vector-icons
- Load roboto font using exponent's load-async.

https://github.com/exponent/nativebase-example


Is there a way for strict null checking when working with OneToOne and ManyToOne references?

Hi,
I noticed that documentation suggest to define one-2-one and many-2-one references this way:
@Entity()
export class User {
  @PrimaryKey()
  id!: number;

  @Property()
  username: string;

  @OneToOne({ mappedBy: 'credentials' })
  credentials: Credentials;

  constructor(props: Omit<User, 'id'>) {
    super();
    this.username = props.username;
    this.credentials = props.credentials;
  }

but when i load user with entity manager property credential can be Reference. It causes problems with typing and strict null checking.
user.credentails.some_property can just be null in runtime, but it can't be null from perspective of my domain model and type system.  This problem can be solved if credentials property will not be Credentials type directly, but some proxy generalized with Credentials type. For example,
@Entity()
export class User {
  @PrimaryKey()
  id!: number;

  @Property()
  username: string;

  @OneToOne({ mappedBy: 'credentials' })
  credentials: Reference<Credentials>;

  constructor(props: Omit<User, 'id'>) {
    super();
    this.username = props.username;
    this.credentials = new Reference(props.credentials);
  }
}

So if i call user.credentials it returns Reference and to access credentials i can do something like:
await user.credentials.load()
If reference is already populated it will just skip queries. Is it possible already?

There is already IEntity.init() method that allows to load entity reference and IEntity.isInitialized() to tell you whether it is fully loaded already. Those methods are available on every discovered entity.
https://mikro-orm.io/entity-references/
Not sure what would be the difference if following your approach with Reference wrapper. You would still have always a reference type instead of the entity type, regardless its loading state, or not?
Currently IEntity.init() will always run queries and refresh the state, but we can easily add a parameter there to allow making queries only when needed.

There is already IEntity.init() method that allows to load entity reference and IEntity.isInitialized() to tell you whether it is fully loaded already.

The problem with this approach is loosing of advantages of Typescript type system. If i just remember to call IEntity.init() i will just only know about this problem with crush on production instead of compilation error that most developers wants.

Not sure what would be the difference if following your approach with Reference wrapper. You would still have always a reference type instead of the entity type, regardless its loading state, or not?

The difference in strictness of null checking. With my approach user.credentials have Reference type that doesn't expose Credential fields and you just can't write the code that will crash in runtime. The code will loos like:
const user = em.findOne(User, id);
const credentials = await user.credentials.load();
const passwordHash = credentials.passwordHash; // just can't be null from type system and runtime perspectives

With current approach i can just write the code
const user = em.findOne(User, id);
const passwordHash = user.credentials.passwordHash; // runtime crash here, but success compilation

I guess I understand now. So you would want a solution which will force you to always call the load() method that would either unwrap the entity reference to just entity, or do the necessary queries to load it first.
I am fine with having this as optional approach. Reference sounds like a good name for this. Would you be up to create a PR? If not, I can do this myself, probably as part of preparations for v3 release.

I guess I understand now. So you would want a solution which will force you to always call the load() method that would either unwrap the entity reference to just entity, or do the necessary queries to load it first.

I believe it is a good summary

I am fine with having this as optional approach. Reference sounds like a good name for this. Would you be up to create a PR? If not, I can do this myself, probably as part of preparations for v3 release.

For the moment i am considering switching from TypeORM to this project that looks promising. TypeORM looks unmaintainable, contains a lot of bugs, legacy, poor typings and based on patterns i don't like. I am not sure i am ready for PRs right now.  If you can provide some guidance i can try.

First step is to setup docker, bootstrap all deps via docker-compose, then create mysql and postgre databases and you should be fine to run the tests. There is a contributing guide here: https://github.com/mikro-orm/mikro-orm/blob/master/CONTRIBUTING.md
Feel free to try and feel free to reach out via slack to discuss any problems on the way. I would love to attract more collaborators, so if you find something you don't understand on this way, I will be more than happy to improve the contributing guide.
Regarding the implementation itself, this will probably require some advanced validation of entity type, as currently the type is sniffed via reflection (via ts-morph) if not provided manually. And in this case one will be probably required to specify the type manually (via entity or type decorator options). But not 100% sure, it might be possible to sniff the generic parameter via ts-morph as well (then this would have to be implemented in TypeScriptMetadataProvider.
I tagged the issue with help-wanted -- will remove that tag and assign myself when I have time for this so you know its being worked on. I would like to have it as part of v3 release, which is almost ready - I expect to finish with it by the end of this month (feature development is pretty much done, but need to write all the documentation and announcement article).

One last thing - if you try to work on this, please use the dev branch as a base, and point the PR to be merged there.

Initial implementation is available for testing via next tag (3.0.0-alpha.3).
There is Reference<T> entity wrapper and IdentifiedReference<T, PK> interface, which adds also PK property.
// define like this
@ManyToOne({ cascade: [Cascade.PERSIST, Cascade.REMOVE] })
publisher: IdentifiedReference<Publisher, 'id'>; // second type parameter will be optional, it specifies the PK field name (can be also `'id' | '_id'` for mongo)

// assign via factory method that returns the `IdentifiedReference` interface
book.publisher = Reference.create(new Publisher(...));

// you can access PK and some helper methods directly
console.log(book.publisher.id);
console.log(book.publisher.isInitialized());
console.log(book.publisher.toJSON());

// you can also use unwrap to access the entity directly
console.log(book.publisher.unwrap()); // Publisher entity

// or use the load method to initialize if needed
console.log(await book.publisher.load()); // returns always intialized Publisher entity

In 3.0.0-alpha.4 there is also wrapped flag in em.getReference():
const ref = orm.em.getReference<Author, 'id' | '_id'>(Author, 1, true);

// or via repository
const repo = orm.em.getRepository(Author);
const ref1 = repo.getReference<'id' | '_id'>(1, true);
console.log(ref1.id); // ok
console.log(ref1._id); // ok

// one can also omit the PK type parameter, but then unwrap() or load() is needed to access the PK
const ref2 = repo.getReference(1, true);
console.log(ref2.id); // type error
console.log(ref2.unwrap().id); // ok
console.log((await ref2.unwrap()).id); // ok

Closing as resolved, feel free to reopen if you have any problem with that.


Error while searching for files with a given file extension (macOS)

$ countfiles -fe txt /

Recursively searching for .txt files in /.

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/bin/countfiles", line 11, in <module>
    load_entry_point('countfiles', 'console_scripts', 'countfiles')()
  File "/Users/username/Dropbox/Count-files/countfiles/__main__.py", line 102, in main_flow
    include_hidden=include_hidden)
  File "/Users/username/Dropbox/Count-files/countfiles/utils/word_counter.py", line 84, in get_files_by_extension
    in Path(os.path.expanduser(location)).rglob(f"*.{extension}")
  File "/Users/username/Dropbox/Count-files/countfiles/utils/word_counter.py", line 83, in <listcomp>
    files = sorted([f for f
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 1094, in rglob
    for p in selector.select_from(self):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 543, in _select_from
    for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 533, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 533, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 531, in _iterate_directories
    if entry.is_dir() and not entry.is_symlink():
OSError: [Errno 9] Bad file descriptor: '/dev/fd/4'

This error does not occur when using the counting feature (countfiles /), only when searching for file extensions.
Since /dev/ is composed of device files, I would suggest to simply skip it on Linux and macOS systems. Any additional thoughts?

I had the idea to exclude some folders from the search. The program is initially designed to count files in the developer's projects. And not for processing of system files of OS.
Though on Windows it does not give out errors (python -m countfiles -fe txt /).
Our program can be considered safe, because it does not change files. But it's better not to touch the system folders.

So... It seems that pathlib.Path.rglob() does not cope well with the kind of stuff that we can find in /dev. But os.walk() seems to work fine, without any errors. We could explicitly exclude some folders, and probably it's a good idea to do so, but I am afraid that these kind of special files (or links to them) could eventually show up in other places. Maybe it will be easier to use os.walk() after all.
I have just made a few changes and it seems to work. But we have now some additional redundant code that needs to be refactored (like functions that do basically the same, with slightly different options). We can schedule that refactoring for v.1.5, though.
@NataliaBondarenko, can you please take a look and see if it still works properly on Windows?

We can use def non_recursive_search and def recursive_search in  def get_files_by_extension. And then filter the results(paths) by extension with def get_file_extension.
os.walk() and os.scandir() works correctly.

We can use def non_recursive_search and def recursive_search in def get_files_by_extension. And then filter the results(paths) by extension with def get_file_extension.

@NataliaBondarenko, I was thinking the other way around (getting rid of recursive_search() and non_recursive_search(), and using get_files_with_extension() instead. This function can replace  both recursive_search() and non_recursive_search(), and even get_files_without_extension(), by adding an if clause to check for the presence of an extension. The resulting function could be renamed to something simpler, like get_files(). I am trying this approach right now. It will probably imply further changes in word_counter.py and __main__.py.

Try it, we need to test different options.
Just update the comments to the function. it's not entirely clear in def get_files():
elif not extension:, extension: str <- this is a required parameter.

Yes, it’s required, but not extension returns False if the content of extension is Falsy, like for instance an empty string.
The docstring may need to be updated to show the purpose of that parameter with empty string. An empty string means “all files, with or without extension”. A string with . means “files without extension”.
I had done that but maybe it was lost with some undo at some point...

Good. Did you add a new option, all the files? But now the function does not return a list of objects <class 'pathlib.Path'>, but a list of strings for os.walk() and list with os.DirEntry objects  for os.scandir()

Yeah, need to correct that.

Just a quick notice to let you know that I submitted an issue for this on the Python Bug Tracker. Seems to be something specific to Mac, probably a bug in the operating system...
https://bugs.python.org/issue33635

Just a quick notice to let you know that the Python team seems to have finally solved this bug in the just released Python 3.7.1.
https://bugs.python.org/issue33635
This project has already contributed to a better Python world :-)


Invalid File Upload in IE 11 Invalid Argument

When i attach an invalid file it doesn't allow n show empty but after i click update button it generates error in console invalid argument ... waiting for reply :)

Post your code

Fixed at 12.1.0