Add Status Effects

Defining my own status effects is pretty simple. I should just add the positive and negative ones from the game. The difficult part will probably be adding/removing them when the condition is cleared. Ideally, you could use the token interface to apply the effect and adjust stats accordingly.

The easiest way to do this would be sacrificing the status icons that are on the player sheet. I'm not sure if I want to do this. Having both reflect each other would be a huge pain. Since it has to apply an active effect to an actor and a token, toggle on both, set a status boolean on the actor, and it probably should check for immunity.


Add -O3 flag to remove remark on compile with icx (intel compiler) on Windows

Remark on compile with icx (CC=icx BIN=libterminal.dll ./build.sh -g)
icx: remark: Note that use of '-g' without any optimization-level option will turn off most compiler optimizations similar to use of '-O0'; use '-Rno-debug-disables-optimization' to disable this remark [-Rdebug-disables-optimization]
Flags to realy optimize:
CFLAGS="$CFLAGS -O3 -fPIC -Ilib/lite-xl/resources/include"

Sure; but this is because you're specifying -g without any optimization-levels. Because this is a user specified option; there's nothing we really need to change on our end, it's just something to note if you're using icx.


Demo doesn't show selection border

It looks like the demo is broken. It hasn't been published in almost a month

Looks like this has been addressed


Add BroadcastChannel

@toddburnside wdyt?

This looks great! So we just need to create Serializers for our messages? Thanks @armanbilge!

So we just need to create Serializers for our messages?

Yes, exactly. I think the Explore messages are already serializable, it is just identity/casting. The big picture goal is to offer a combinator API to build Serializers for arbitrary data classes e.g. similar to skunk codecs.


zip files from CICFlowMeter cannot be opened on Windows OS

zip files from
https://www.unb.ca/cic/research/applications.html#CICFlowMeter
cannot be opened on Windows OS

Can you highlight the instance from where this link is picked from the book? There are links in the book that redirects to the CIC datasets but not the one which is mentioned above.

I can not find them neither
may you share working links from book?


If the permanent filter within dataTableContext is set to required, on render it does not apply the filter it acts as an optional filter than the required one

If you set permanent filter to required, when the filter is executed it acts as an optional filter instead of required
Steps to reproduce the behavior:

Go to 'https://function-adminportal-test.shesha.dev/shesha/forms-designer?id=0964f73a-a16d-404a-86e6-04da8f59131d'
Click on 'dataTableContext'
Click on 'permanent filter'
Apply a filter and set it to required
Save and preview
Actual Result: All data gets displayed without applying the filter

TableViewSelector works correctly. When the filter is required, it applies the filter immediately
Expected behavior
When a filter is set to required, on render it should apply the filter as configured

Hi @IvanIlyichev , let me review this item once this item: #1300  is fixed before investing time on it

Hi @hlayisanimazuze. It's fixed, you can check


Missing llvm_ext.o in Linux release

When I try to compile crystalline I get this error:
Error target crystalline failed to compile:
/home/linuxbrew/.linuxbrew/bin/ld: cannot find /var/lib/snapd/snap/crystal/1441/share/crystal/src/llvm/ext/llvm_ext.o: No such file or directory

Issue 1. Snap package missing llvm_ext.o.
Issue 2: The latest release package for Linux (crystal-1.6.2-1-linux-x86_64.tar.gz) is also missing llvm_ext.o. But crystal-1.6.2-1-darwin-universal.tar.gz includes it.
For example, the Arch Linux crystal package includes llvm_ext.o.
At the same time, the Ubuntu package (which is maintained by Crystal team) is missing it too:
dpkg -S crystal | grep llvm_ext
crystal: /usr/share/crystal/src/llvm/lib_llvm_ext.cr
crystal: /usr/share/crystal/src/llvm/ext/llvm_ext.cc

Is it a bug at all? Should the release packages include  llvm_ext.o? Or did I miss something?
Thanks!

llvm_ext.o depends on the installed LLVM version. Hence it's not supposed to be packaged (unless the package would also install a specific LLVM version). There's a good chance it won't work with the version you have installed anyways.
The darwin package including it is probably just an omission. In the linux package we're explicitly excluding it: https://github.com/crystal-lang/distribution-scripts/blob/d8c44c0a2c977bfb90363923047f2c64cef3bf8e/docker/alpine.Dockerfile#L36 (related: #10880).

I have, the similar issue, just compile it by myself. and all works.
cd ~/Downloads/crystal-1.6.2-1/share/crystal/src/llvm/ext
gcc -c llvm_ext.cc `llvm-config --cppflags` -O3

Thanks, @kostya ! All works.

My case with Arch Linux and llvm13:

Compile llvm_ext.o from Crystal sources
cd ~/src/crystal/src/llvm/ext
gcc -c llvm_ext.cc `/usr/lib/llvm13/bin/llvm-config --cppflags` -O3

Build crystalline
git clone https://github.com/elbywan/crystalline
cd crystalline
CRYSTAL_PATH=~/src/crystal/src:lib LLVM_CONFIG=/usr/lib/llvm13/bin/llvm-config shards build

I suppose we could at least provide a makefile to build llvm_ext.o easier.
However, I'm also wondering if we can get rid of llvm_ext entirely at some point. I think by now many of the functions have an equivalent in libllvm.

Isn't that make deps?

$ make deps
c++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc<EMAIL_ADDRESS>-std=c++14 -stdlib=libc++   -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

Yes, when you have the source repository checked out. But the makefile is not distributed with the stdlib source.

I'm encountering a similar error on Ubunto 22.04 LTS:
Building: crystalline
Error target crystalline failed to compile:
/usr/bin/ld: cannot find /usr/share/crystal/src/llvm/ext/llvm_ext.o: No such file or directory
/usr/bin/ld: cannot find -lstdc (this usually means you need to install the development package for libstdc)++: No such file or directory
collect2: error: ld returned 1 exit status
I have to admit, I'm new to Ubuntu; I've only used Gentoo prior to this, so the idea of not having source code or debugging symbols installed for everything is a bit confusing to me. 😀 How would I resolve this? Do I need to install something extra for Crystal (since the file it's looking for is under the Crystal folder) or for LLVM?

Haha, so far I've only seen people coming from Ubuntu and other binary package distributions being overwhelmed from building everything themselves in Gentoo 😆
Actually, this fits perfectly because you just have to build the file. The sources should all be there in that ext directory. Only the Makefile is (probably) missing. It should be at /usr/share/crystal/Makefile.
It's this recipe though: https://github.com/crystal-lang/crystal/blob/bb96c5398e001f41b31431eea283a1a28198a84b/Makefile#L225C2-L225C2
So you can either copy the Makefile from the source repository and run make llvm_ext or run the command directly.
g++ -c -o llvm_ext.o llvm_ext.cc $(llvm-config --cxxflags)

I think that's a question for the crystalline project.

True, apologies, I wasn't paying attention and thought this issue was on the Crystalline git.


Add python3-insightface-pip

Please add the following dependency to the rosdep database.
Package name:
InsightFace
Package Upstream Source:
insightface
Purpose of using this:
InsightFace is an integrated Python library for 2D&3D face analysis. InsightFace efficiently implements a rich variety of state of the art algorithms of face recognition, face detection and face alignment, which optimized for both training and deployment. Research institute and industrial organization can get benefits from InsightFace library.
Links to Distribution Packages

Debian: (https://pypi.org/project/insightface/)
Ubuntu: (https://pypi.org/project/insightface/)

To confirm, you can only use this in a source workspace. It cannot be a dependency of a released package.


Listeners triggered on two different services with same Characteristic UUID

I am using a custom service to control a light. the service contains one characteristic for the light state, I have multiple lights and therefore am using multiple instances of services with the same service UUID and characteristic UUID. When I send data from the peripheral device through the state characteristic of one service it activates the listener on both services.

In these months I worked a lot on a scenario like yours. I choose to save the characteristic in a Map, for each device and I'm able to get different data from each device, without an effect like yours. I'm using notify-feature and listen to each stream on at least 4 different devices.

In these months I worked a lot on a scenario like yours. I choose to save the characteristic in a Map, for each device and I'm able to get different data from each device, without an effect like yours. I'm using notify-feature and listen to each stream on at least 4 different devices.

would you be ab

In these months I worked a lot on a scenario like yours. I choose to save the characteristic in a Map, for each device and I'm able to get different data from each device, without an effect like yours. I'm using notify-feature and listen to each stream on at least 4 different devices.

I have not tried connecting on multiple devices so I do not know if this issue persists if I were to connect to multiple. my issue is that I have multiple services with a characteristic with the same UUID on the same device.

just tried your solution to save in a map and am still getting the same result. @pauldemarco any idea what causes this issue?

I have not tried connecting on multiple devices so I do not know if this issue persists if I were to connect to multiple. my issue is that I have multiple services with a characteristic with the same UUID on the same device.

Whoa, that's sound new for me. I google a few query and I found that is a common policy for many sensors. At this link
https://stackoverflow.com/questions/27413272/what-gatt-services-use-multiple-characteristics-with-the-same-uuid
you can find a docs about BLE standard.

again, this is not my issue. the issue I am having is that I have multiple of the same service with the same characteristic. not one service with multiple characteristics with the same UUID.
my case:

Light Service
Characteristic
Light Service
Characteristic

the link:

Light Service
Characteristic
Characteristic

in my case when a value is changed on one characteristic the listener in both services is activated

I have this same problem.  I have a device with two characteristics with the same UUID.  Writing to either characteristic only writes to the first service.
Note that I have native code for both Android and iOS that works perfectly for this, so the device is not the problem.  In my native code, writing to each characteristic writes to a separate service.


Bug fix: marshal and unmarshal messages to be stored in Redis

Description
Bug fix: marshal and unmarshal messages to be stored in Redis
(new list[BaseMessage] can't be stored directly)
Tested against real Redis
Type of change

[ ] Refactor
[ ] New feature
[x] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 96.93%. Comparing base (f5f6370) to head (b22f4cb).
Report is 20 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #597      +/-   ##
==========================================
- Coverage   96.99%   96.93%   -0.06%
==========================================
  Files          49       49
  Lines        1597     1567      -30
==========================================
- Hits         1549     1519      -30
  Misses         48       48

Files
Coverage Δ

ols/src/cache/redis_cache.py
100.00% <100.00%> (ø)

... and 4 files with indirect coverage changes

Tested against real Redis

/lgtm

/approve


Bug - Update apps: Pressing Upgrade all packages results in an attempt to install already updated apps

For example, let's say when WingetUI has finished loading that it detected something like 5 apps that have updates available, so the user presses Upgrade all packages - but one of them fails to update, so the user attempts to fix the last one easily by pressing Upgrade all packages again.
That will result in the app not only attempting to update the app that's left - but all the apps that were detected during that session.
See the video below.
https://user-images.githubusercontent.com/67732686/185784126-2906dcfa-1c1e-47f6-b497-bcde0d755b28.mp4

Could you please show the error shown in the error boxes?

https://user-images.githubusercontent.com/67732686/187735453-32170d4f-b3a7-4077-a0e8-685e7ab9460e.mp4

Done!
Additionally, the button won't update the unknown versions if they were not selected to be shown


[5.3] Repopulate remember box

We already use old() to repopulate the email field, so it makes sense to also re-populate the remember box if it was ticked.

Maybe send to 5.4?

It is only a stub for a generated view - so wouldnt affect anything?


Robustness boost

Attempt to enhance software robustness through introducing the following two features:

automatic garbage collection in base env
support recent ray API change

Pull Request Test Coverage Report for Build 2330

29 of 50 (58.0%)  changed or added relevant lines in 11 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 86.396%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

flow/envs/base_env.py
9
10
90.0%

examples/rllib/cooperative_merge.py
2
4
50.0%

examples/rllib/figure_eight.py
2
4
50.0%

examples/rllib/green_wave.py
2
4
50.0%

examples/rllib/multiagent_exps/multiagent_figure_eight.py
2
4
50.0%

examples/rllib/multiagent_exps/multiagent_stabilizing_the_ring.py
2
4
50.0%

examples/rllib/stabilizing_highway.py
2
4
50.0%

examples/rllib/stabilizing_the_ring.py
2
4
50.0%

examples/rllib/velocity_bottleneck.py
2
4
50.0%

flow/visualize/visualizer_rllib.py
2
4
50.0%

Files with Coverage Reduction
New Missed Lines
%

tests/fast_tests/test_two_loops_one_merging.py
2
95.83%

Totals

Change from base Build 2309:
-0.2%

Covered Lines:
7100

Relevant Lines:
8218

💛  - Coveralls

Thx!


Roadmap

This is a loose roadmap of our plans for major changes to Gymnasium:
December:

[x] Experimental new wrappers
[x] Experimental functional API
[x] Python 3.11 support

February:

[ ] Official Conda packaging
[ ] Add Experimental vector API
[ ] Add full testing for experimental wrappers
[ ] Add Experimental vector wrappers
[ ] Add initial support for Minari
[ ] Release v0.28.0

March / April:

[ ] Fix all bugs and update documentation for experimental features
[ ] Add functional versions of all gymnasium environments
[ ] Make initial release of Brax environments
[ ] Extensive envpool integration for creating vectorized environments
[ ] Release v0.29.0 as an intermediate version with experimental functional, new wrappers and vectors to root

May:

[ ] Release v0.30.0 with old wrapper and vector removed

June:

[ ] Move Box2D and MuJoCo environments to separate repos for reproducibility
[ ] 1.0 release

How can new contributors get involved?

Could you join our discord server https://discord.gg/bnJ6kubTg6 and put a message in the new contributes channel with any particular topics that you would like to work on

I'd be interested to have a Gymnasium.jl Julia wrapper, happy to help in any way if that's in scope! :slightly_smiling_face:

We're planning to worry about multi-language support in the future after some of the current items are ironed

I didn't want to open an extra issue for that, but regarding the full type hint support in the future, it should be considered to switch to numpys typing module.

@juhannc Thanks for the suggestion, we are already in the process of adding this in https://github.com/Farama-Foundation/Gymnasium/pull/37
If you have experience with using numpy typing, I would be open to you review the PR and any suggestions for improving it

Not to clutter this thread, but @kir0ul I needed a julia wrapper for my research as well. Just made an unofficial minimal wrapper around the Gymnasium python interface here: https://github.com/mkschleg/Gymnasium.jl. Pretty under-tested and no guarantees, but a starting point. It isn't (and probably won't be depending on the makers of gymnasium's plans) registered.
The makers of gymnasium for python can fork this in their own repo for official registration if they want.

Hi! Will you add support for Gym Retro? :)

Hi, I believe that the retro is being maintained - https://github.com/MatPoliquin/stable-retro

Hi, I believe that the retro is being maintained - https://github.com/MatPoliquin/stable-retro

Thank you! :)
It looks a personal project, is it monitored from some team related with OpenAI or Farama Foundation?

Closing, since 1.0 is released


#272 Fix module resolution bug in lib and fix offline check in node …

…environments
PR Checklist:

[x] Link to related issues:
[ ] Add changelog entry linking to issue
[ ] Add tests (if needed)
[ ] (If new feature) added in description / readme

Similar to #270. I think I should merge 270, as that also includes the react libs.

Close in favor of PR #271


restart: always not applied to services in compose format

Repro:

rancherctl s enable https://gist.githubusercontent.com/imikushin/fec33701294b77641022/raw/d6c488f0fb1cb21d60e9d8b64c5a1e4abff767fe/nginx.yml
Reboot.
Wait until nginx container is started. docker ps -a shows the container is running.
Simulate container main process failure:
ps -ef | grep 'nginx: master' | awk '{print $2}' | xargs -I{} kill {}
docker ps -a shows nginx container is NOT running.

If I just use docker CLI to run a container with --restart=always and kill its main process from outside, the container gets restarted as expected.

The compose parser is quiet incomplete at the moment.  Will fix.  I honestly thought this one worked, but obviously not.

I tried to fix this myself today, but got lost in config datastructures :)
On Fri, Apr 24, 2015, 18:56 Darren Shepherd<EMAIL_ADDRESS>wrote:

The compose parser is quiet incomplete at the moment. Will fix. I honestly
thought this one worked, but obviously not.
—
Reply to this email directly or view it on GitHub
https://github.com/rancherio/os/issues/230#issuecomment-95941034.

What!?  There's only four different data structures called all called config.  How could that be confusing :)
The code already has some cruft from the rapid iterations.  I can point you to what needs to been change in a bit.
Darren

On Apr 24, 2015, at 7:02 AM, Ivan Mikushin<EMAIL_ADDRESS>wrote:
I tried to fix this myself today, but got lost in config datastructures :)
On Fri, Apr 24, 2015, 18:56 Darren Shepherd<EMAIL_ADDRESS>wrote:

The compose parser is quiet incomplete at the moment. Will fix. I honestly
thought this one worked, but obviously not.
—
Reply to this email directly or view it on GitHub
https://github.com/rancherio/os/issues/230#issuecomment-95941034.

—
Reply to this email directly or view it on GitHub.

I didn't even see that this was enabled per #221. At least I didn't see it in the factory.go file. I'll close my issue in favor of this one.

The same for core system services: adding Restart: "always" in their config code doesn't propagate to their containers' HostConfig.RestartPolicy property, so they don't get restarted.

This looks fixed in v0.3.1-rc1, but @imikushin could you test this out for me? Just want a second tester.

Confirm fixed.


bmt dependency issue

@LucaCappelletti94 could you have a look at this?
Setting environment variables from .travis.yml
$ export CODACY_PROJECT_TOKEN=[secure]
$ export CC_TEST_REPORTER_ID=[secure]
0.01s$ source ~/virtualenv/python3.6/bin/activate
$ python --version
Python 3.6.3
$ pip --version
pip 9.0.1 from /home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages (python 3.6)
before_install.1
3.27s$ pip install --upgrade pytest
before_install.2
4.57s$ pip install --upgrade mypy
before_install.3
3.63s$ pip install --upgrade pylint
0.84s$ pip install bmt
Collecting bmt
  Downloading https://files.pythonhosted.org/packages/dd/d0/915b5604b4ddefc5832cc1ab0f42157842c21b9f18ba6060f5d9a4f8c829/bmt-0.1.1-py3-none-any.whl
bmt requires Python '>=3.7' but the running Python is 3.6.3
The command "pip install bmt" failed and exited with 1 during .

Yes I'm currently investigating.

@LucaCappelletti94 thanks - let me know if you want to Zoom and chat today about this or other CI things

It seems like Pypi was briefly offline a few hours ago. Can you try to re-run the build?

Close as this particular issue has been solved.


Gradle sync error with gradle 3.0 and butterKinfe plugin 8.8.1 in library

first set classpath in root project gradle
 dependencies {
        classpath 'com.android.tools.build:gradle:3.0.0'
        classpath "com.jakewharton:butterknife-gradle-plugin:8.8.1"
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
}

apply plugin in library module
apply plugin: 'com.jakewharton.butterknife'

dependencies {
     // butter knife
    implementation "com.jakewharton:butterknife:8.8.1"
    annotationProcessor "com.jakewharton:butterknife-compiler:8.8.1"
}

then I got gradle sync error:

Error:Unable to find method 'com.android.build.gradle.api.BaseVariant.getOutputs()Ljava/util/List;'.
Possible causes for this unexpected error include:Gradle's dependency cache may be corrupt (this sometimes occurs after a network connection timeout.)
Re-download dependencies and sync project (requires network)The state of a Gradle build process (daemon) may be corrupt. Stopping all Gradle daemons may solve this problem.
Stop Gradle build processes (requires restart)Your project may be using a third-party plugin which is not compatible with the other plugins in the project or the version of Gradle requested by the project.In the case of corrupt Gradle processes, you can also try closing the IDE and then killing all Java processes.

You'll have to use 9.0.0-SNAPSHOT for now.

This is still not working.

when I user the 8.8.1 version,I have the same problem.

when I use the 10.2.3 version ,I have the same problem

when I use the 10.2.3 version ,I have the same problem


Cannot set appDirectory and outputDirectory to external path out of project root

I wanna build a windows-installer on my Mac, when I set the appDirectory and outputDirectory to the path out of project root, it prepends automatically the root path of project which is not what I want. So could you tell me how to make it right? Thanks!

I guess this is related to #26

@Castafiore please, confirm is #22 fixes it. Already merged into master.

@Castafiore can you try upgrading to 2.0.5?

@Castafiore please, check it with 2.0.5, otherwise I'll close the issue.

Feel free to reopen if you still have this problem.


Axis scale improvements

This PR improves the "dual" axis scale feature, makes the xminorticks/yminorticks Axes.format "toggle" always use the default minor locator for the minor scale, cleans up _ScaleBase, changes the default symlog subs and linthresh, and fixes the default LogitScale formatters and locators.

This also considerably cleans up the dualx/dualy internals. Now the "transform" interpretation is moved to FuncScale, and dualx/dualy only accept functions, function-pairs or axis scale classes (i.e. no more on-the-fly generation).


Doesn't work on folder that has windows-written files in it

I'm using this on truenas Scale (if that matters), trying to rebalance a directory (that I changed checksuming on) and get the following error:
 cp -adxp foo.tif foo.tmp
cp: preserving permissions for ‘foo.tmp’: Operation not permitted

-a and -p don't seem to work, my guess possibly because of windows permissions?
-dx work but this creates new timestamps.  The weird thing is, even with the "operation not permitted" message, the file is created with the same timestamp.
-rwxrwxr-x  1 evan root 122878676 Jul 16  2017  foo.tmp
-rwxrwxr-x  1 evan root 122878676 Jul 16  2017  foo.tif.balance
-rwxrwxr-x  1 evan root 122878676 Jul 16  2017  foo.tif

the closest i can get to fixing this is
-dxR --preserve=ownership,timestamps

it strips out the mode, but seems to work the same
it also seems that with the -adxp, the file does get created, you just get a warning

Do you think this should somehow be integrated into the script? Or do we need a better solution?

I am not sure what you mean by "windows written files", do you have some kind of VM thats using a ZFS dataset as a volume? Would it be possible to create a replica file within this repository to add it to the test suite? Or would the git repository break this?
I have also lots of files on my NAS which are written through a Windows SSH connection to the ZFS filesystem and they work just fine.

Closing this because of inactivity.


Show errors when exit

Describe the bug
errors when exit
To Reproduce
Steps to reproduce the behavior:

go into a room
play movie by mpv
close mpv
See error

Expected behavior
do not see error when exit
Screenshots

Version and platform:

OS: Windows 11 64-bit
syncplay 1.7.1 beta installer
MPV

Additional context
Syncplay.log

I assume that by 1.7.1 beta 1 you mean 1.7.0 beta 1. I just tried the Windows build on Windows 10 and followed your steps and didn't see any error.
However, I've made a tweak that might fix it. Please try out the installer or portable Windows builds at https://github.com/Syncplay/syncplay/actions/runs/3802200716 and let me know if it solves the problem.

I reinstall this version.
Fixed.


Re-enable touch screen controls.

Version 1.0 will have a fixed game area size.

The coordinates given in the Layout Manager will be valid in both Tablet and non-tablet modes.
When in Tablet mode, non-touchscreen panels will be divided based on their given coordinates. The touchscreen controls will not be used in the calculation of the non-touchscreen controls.
When in non-tablet mode, the touchscreen controls will not be visible and the coordinates in the LayoutManager will cover the entire size of the size panels.


⚠ Bofrost.be has degraded performance

In 13ef3d7, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1650 ms

Resolved: Bofrost.be performance has improved in d58b4d8 after 45 minutes.


refactor(next): move vite to peer deps

#432  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @hi-ogawa and the rest of your teammates on  Graphite


Memory usage/requirements should be better documented

There should be two clearly separated pieces of info:

Memory for index running (this is already sh there)

Memory needed for index construction (e.g., for HG) – just a rough idea to know eg what machine to choose on a cluster - high mem vs standard, etc.

These are my values from the human genome, k=31, index construction:
experiment  run   real_s  real_s_py   user_s  sys_s  percent_cpu  max_ram_kb  fs_inputs  fs_outputs  exit_code  command
None        None  975.17  975.179138  955.58  15.56  99%          46845014    0          1396768     0          fmsi index -p masked_superstring.mones.fa


When we have a regression range, automatically set the affected value

When we have a regression range (hasregressionrange), we are able to detect which version is impacted, we should automatically set the status flags.
Idea from Pascal

We have been doing this for a while now with the auto_nag/scripts/regression_set_status_flags.py script.


Renamed ngTableParams to NgTableParams

When using the ngTable into our application we've encountered a jshint error because the ngTableParams is a method that is starting with a lower case instead of uppercase letter.
This issue can be sorted out by activating newcap in jshint so you can ignore it.
/* jshint newcap:true */
However, I've renamed the ngTableParams with NgTableParams so we could use jshint with default options and have no errors when placing ngTable inside our app.
This fix is for issue https://github.com/esvit/ng-table/issues/359

Closing as rename has already been applied


api test file updated

Codecov Report

Merging #88 into master will increase coverage by 1.3%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master      #88     +/-   ##
=========================================
+ Coverage   55.26%   56.56%   +1.3%
=========================================
  Files          80       80
  Lines         845      845
  Branches       70       70
=========================================
+ Hits          467      478     +11
+ Misses        338      328     -10
+ Partials       40       39      -1

Impacted Files
Coverage Δ

server/validation/login-trainer-val.js
100% <0%> (+23.07%)
:arrow_up:

server/controllers/login-trainer.js
100% <0%> (+57.14%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2447881...eccbeca. Read the comment docs.


Bookinfo with HTTPS external access example is not working

Bug description
When trying to follow this "Bookinfo with HTTPS access to a Google Books web service" example, it is unable to retrieve the book info from google via an https call.
https://istio.io/blog/2018/egress-https/#bookinfo-with-https-access-to-a-google-books-web-service
Expected behavior
Expected it to be able to retrieve the book info via an https call after applying the serviceentry per the example.
Steps to reproduce the bug

Install istio with my helm values below
Run through the example step by step

Version (include the output of istioctl version --remote and kubectl version)
$ istioctl version --remote
client version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Clean", GitTag:"1.1.4-10-g9b6d31b"}
citadel version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b-dirty", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.4-10-g9b6d31b"}
egressgateway version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Clean", GitTag:"1.1.4-10-g9b6d31b"}
galley version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b-dirty", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.4-10-g9b6d31b"}
ingressgateway version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Clean", GitTag:"1.1.4-10-g9b6d31b"}
ingressgateway-mqtt version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
nodeagent version: version.BuildInfo{Version:"", GitRevision:"", User:"", Host:"", GolangVersion:"", DockerHub:"", BuildStatus:"", GitTag:""}
pilot version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b-dirty", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.4-10-g9b6d31b"}
policy version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b-dirty", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.4-10-g9b6d31b"}
sidecar-injector version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b-dirty", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.4-10-g9b6d31b"}
telemetry version: version.BuildInfo{Version:"1.1.5", GitRevision:"9b6d31b74d1c0cc9358cc82d395b53f71393326b-dirty", User:"root", Host:"3e29fde4-6c3f-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Modified", GitTag:"1.1.4-10-g9b6d31b"}

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.7", GitCommit:"65ecaf0671341311ce6aea0edab46ee69f65d59e", GitTreeState:"clean", BuildDate:"2019-01-24T19:22:45Z", GoVersion:"go1.10.7", Compiler:"gc", Platform:"linux/amd64"}

How was Istio installed?
Installed via helm template (https://istio.io/docs/setup/kubernetes/install/helm/#option-1-install-with-helm-via-helm-template)
Values file:
---
# Top level istio values file has the following sections.
#
# global: This file is the authoritative and exhaustive source for the global section.
#
# chart sections: Every subdirectory inside the charts/ directory has a top level
#       configuration key in this file. This file overrides the values specified
#       by the charts/${chartname}/values.yaml.
#       Check the chart level values file for exhaustive list of configuration options.

#
# Gateways Configuration, refer to the charts/gateways/values.yaml
# for detailed configuration
#
gateways:
  enabled: true

  # istio-ingressgateway:
  #   resources:
  #     requests:
  #       cpu: 10m
  #       memory: 40Mi
  #     limits:
  #       cpu: 100m
  #       memory: 128Mi

  istio-egressgateway:
    enabled: true
    resources:
      requests:
        cpu: 10m
        memory: 40Mi
      limits:
        cpu: 100m
        memory: 128Mi

  istio-ingressgateway:
    enabled: true
    #
    # Secret Discovery Service (SDS) configuration for ingress gateway.
    #
    sds:
      # If true, ingress gateway fetches credentials from SDS server to handle TLS connections.
      enabled: true
      # SDS server that watches kubernetes secrets and provisions credentials to ingress gateway.
      # This server runs in the same pod as ingress gateway.
      image: node-agent-k8s
    labels:
      app: istio-ingressgateway
      istio: ingressgateway
    autoscaleEnabled: true
    autoscaleMin: 1
    autoscaleMax: 5
    # specify replicaCount when autoscaleEnabled: false
    # replicaCount: 1
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 2000m
        memory: 256Mi
    cpu:
      targetAverageUtilization: 80
    loadBalancerIP: ""
    loadBalancerSourceRanges: []
    externalIPs: []
    serviceAnnotations: {}
    podAnnotations: {}
    type: LoadBalancer #change to NodePort, ClusterIP or LoadBalancer if need be
    #externalTrafficPolicy: Local #change to Local to preserve source IP or Cluster for default behaviour or leave commented out
    ports:
      ## You can add custom gateway ports
      # Note that AWS ELB will by default perform health checks on the first port
      # on this list. Setting this to the health check port will ensure that health
      # checks always work. https://github.com/istio/istio/issues/12503
    # - port: 15020
    #   targetPort: 15020
    #   name: status-port
    - port: 80
      targetPort: 80
      name: http2
      # nodePort: 31380
    - port: 443
      name: https
      # nodePort: 31390
    # Example of a port to add. Remove if not needed
    # - port: 31400
    #   name: tcp
      # nodePort: 31400
    ### PORTS FOR UI/metrics #####
    ## Disable if not needed
    # - port: 15029
    #   targetPort: 15029
    #   name: https-kiali
    # - port: 15030
    #   targetPort: 15030
    #   name: https-prometheus
    # - port: 15031
    #   targetPort: 15031
    #   name: https-grafana
    # - port: 15032
    #   targetPort: 15032
    #   name: https-tracing
      # This is the port where sni routing happens
    # - port: 15443
    #   targetPort: 15443
    #   name: tls
    #### MESH EXPANSION PORTS  ########
    # Pilot and Citadel MTLS ports are enabled in gateway - but will only redirect
    # to pilot/citadel if global.meshExpansion settings are enabled.
    # Delete these ports if mesh expansion is not enabled, to avoid
    # exposing unnecessary ports on the web.
    # You can remove these ports if you are not using mesh expansion
    meshExpansionPorts:
    - port: 15011
      targetPort: 15011
      name: tcp-pilot-grpc-tls
    - port: 15004
      targetPort: 15004
      name: tcp-mixer-grpc-tls
    - port: 8060
      targetPort: 8060
      name: tcp-citadel-grpc-tls
    - port: 853
      targetPort: 853
      name: tcp-dns-tls
    ####### end MESH EXPANSION PORTS ######
    ##############
    secretVolumes:
    - name: ingressgateway-certs
      secretName: istio-ingressgateway-certs
      mountPath: /etc/istio/ingressgateway-certs
    - name: ingressgateway-ca-certs
      secretName: istio-ingressgateway-ca-certs
      mountPath: /etc/istio/ingressgateway-ca-certs
    ### Advanced options ############
    env:
      # A gateway with this mode ensures that pilot generates an additional
      # set of clusters for internal services but without Istio mTLS, to
      # enable cross cluster routing.
      ISTIO_META_ROUTER_MODE: "sni-dnat"
    nodeSelector: {}

    # Specify the pod anti-affinity that allows you to constrain which nodes
    # your pod is eligible to be scheduled based on labels on pods that are
    # already running on the node rather than based on labels on nodes.
    # There are currently two types of anti-affinity:
    #    "requiredDuringSchedulingIgnoredDuringExecution"
    #    "preferredDuringSchedulingIgnoredDuringExecution"
    # which denote “hard” vs. “soft” requirements, you can define your values
    # in "podAntiAffinityLabelSelector" and "podAntiAffinityTermLabelSelector"
    # correspondingly.
    # For example:
    # podAntiAffinityLabelSelector:
    # - key: security
    #   operator: In
    #   values: S1,S2
    #   topologyKey: "kubernetes.io/hostname"
    # This pod anti-affinity rule says that the pod requires not to be scheduled
    # onto a node if that node is already running a pod with label having key
    # “security” and value “S1”.
    podAntiAffinityLabelSelector: {}
    podAntiAffinityTermLabelSelector: {}

  istio-ingressgateway-mqtt:
    enabled: true
    #
    # Secret Discovery Service (SDS) configuration for ingress gateway.
    #
    sds:
      # If true, ingress gateway fetches credentials from SDS server to handle TLS connections.
      enabled: true
      # SDS server that watches kubernetes secrets and provisions credentials to ingress gateway.
      # This server runs in the same pod as ingress gateway.
      image: node-agent-k8s
    labels:
      app: istio-ingressgateway-mqtt
      istio: ingressgateway-mqtt
    autoscaleEnabled: true
    autoscaleMin: 1
    autoscaleMax: 5
    # specify replicaCount when autoscaleEnabled: false
    # replicaCount: 1
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 2000m
        memory: 256Mi
    cpu:
      targetAverageUtilization: 80
    loadBalancerIP: ""
    loadBalancerSourceRanges: []
    externalIPs: []
    serviceAnnotations: {}
    podAnnotations: {}
    type: LoadBalancer #change to NodePort, ClusterIP or LoadBalancer if need be
    #externalTrafficPolicy: Local #change to Local to preserve source IP or Cluster for default behaviour or leave commented out
    ports:
    - port: 443
      name: mqtt
    secretVolumes:
    - name: ingressgateway-certs
      secretName: istio-ingressgateway-certs
      mountPath: /etc/istio/ingressgateway-certs
    - name: ingressgateway-ca-certs
      secretName: istio-ingressgateway-ca-certs
      mountPath: /etc/istio/ingressgateway-ca-certs
    ### Advanced options ############
    env:
      # A gateway with this mode ensures that pilot generates an additional
      # set of clusters for internal services but without Istio mTLS, to
      # enable cross cluster routing.
      ISTIO_META_ROUTER_MODE: "sni-dnat"
    nodeSelector: {}

    podAntiAffinityLabelSelector: {}
    podAntiAffinityTermLabelSelector: {}

  # Second ingress gateway
  # istio-ingressgateway-2:
  #   enabled: true
  #   #
  #   # Secret Discovery Service (SDS) configuration for ingress gateway.
  #   #
  #   sds:
  #     # If true, ingress gateway fetches credentials from SDS server to handle TLS connections.
  #     enabled: true
  #     # SDS server that watches kubernetes secrets and provisions credentials to ingress gateway.
  #     # This server runs in the same pod as ingress gateway.
  #     image: node-agent-k8s
  #   labels:
  #     app: istio-ingressgateway
  #     istio: ingressgateway
  #   autoscaleEnabled: true
  #   autoscaleMin: 1
  #   autoscaleMax: 5
  #   # specify replicaCount when autoscaleEnabled: false
  #   # replicaCount: 1
  #   resources:
  #     requests:
  #       cpu: 100m
  #       memory: 128Mi
  #     limits:
  #       cpu: 2000m
  #       memory: 256Mi
  #   cpu:
  #     targetAverageUtilization: 80
  #   loadBalancerIP: ""
  #   loadBalancerSourceRanges: []
  #   externalIPs: []
  #   serviceAnnotations: {}
  #   podAnnotations: {}
  #   type: LoadBalancer #change to NodePort, ClusterIP or LoadBalancer if need be
  #   #externalTrafficPolicy: Local #change to Local to preserve source IP or Cluster for default behaviour or leave commented out
  #   ports:
  #     ## You can add custom gateway ports
  #     # Note that AWS ELB will by default perform health checks on the first port
  #     # on this list. Setting this to the health check port will ensure that health
  #     # checks always work. https://github.com/istio/istio/issues/12503
  #   - port: 15020
  #     targetPort: 15020
  #     name: status-port
  #   - port: 80
  #     targetPort: 80
  #     name: http2
  #     nodePort: 31381
  #   - port: 443
  #     name: https
  #     nodePort: 31391
  #   # Example of a port to add. Remove if not needed
  #   - port: 31400
  #     name: tcp
  #     nodePort: 31401
  #   ### PORTS FOR UI/metrics #####
  #   ## Disable if not needed
  #   - port: 15029
  #     targetPort: 15029
  #     name: https-kiali
  #   - port: 15030
  #     targetPort: 15030
  #     name: https-prometheus
  #   - port: 15031
  #     targetPort: 15031
  #     name: https-grafana
  #   - port: 15032
  #     targetPort: 15032
  #     name: https-tracing
  #     # This is the port where sni routing happens
  #   - port: 15443
  #     targetPort: 15443
  #     name: tls

#
# sidecar-injector webhook configuration, refer to the
# charts/sidecarInjectorWebhook/values.yaml for detailed configuration
#
sidecarInjectorWebhook:
  enabled: true

#
# galley configuration, refer to charts/galley/values.yaml
# for detailed configuration
#
galley:
  enabled: true

#
# mixer configuration
#
# @see charts/mixer/values.yaml, it takes precedence
mixer:
  enabled: true
  policy:
    # if policy is enabled the global.disablePolicyChecks has affect.
    enabled: true

  telemetry:
    enabled: true
#
# pilot configuration
#
# @see charts/pilot/values.yaml
pilot:
  enabled: true

#
# security configuration
#
security:
  enabled: true

#
# nodeagent configuration
#
nodeagent:
  enabled: true
  image: node-agent-k8s
  env:
    CA_PROVIDER: "Citadel"
    CA_ADDR: "istio-citadel:8060"
    VALID_TOKEN: true

#
# addon grafana configuration
#
grafana:
  enabled: false

#
# addon prometheus configuration
#
prometheus:
  enabled: true

#
# addon servicegraph configuration
#
servicegraph:
  enabled: false

#
# addon jaeger tracing configuration
#
tracing:
  enabled: false

#
# addon kiali tracing configuration
#
kiali:
  enabled: false

#
# Istio CNI plugin enabled
#   This must be enabled to use the CNI plugin in Istio.  The CNI plugin is installed separately.
#   If true, the privileged initContainer istio-init is not needed to perform the traffic redirect
#   settings for the istio-proxy.
#
istio_cni:
  enabled: false

# addon Istio CoreDNS configuration
#
istiocoredns:
  enabled: false

# Common settings used among istio subcharts.
global:
  # Default hub for Istio images.
  # Releases are published to docker hub under 'istio' project.
  # Daily builds from prow are on gcr.io, and nightly builds from circle on docker.io/istionightly
  hub: docker.io/istio

  # Default tag for Istio images.
  # tag: 1.1.3

  # monitoring port used by mixer, pilot, galley
  monitoringPort: 15014

  k8sIngress:
    enabled: false
    # Gateway used for k8s Ingress resources. By default it is
    # using 'istio:ingressgateway' that will be installed by setting
    # 'gateways.enabled' and 'gateways.istio-ingressgateway.enabled'
    # flags to true.
    gatewayName: ingressgateway
    # enableHttps will add port 443 on the ingress.
    # It REQUIRES that the certificates are installed  in the
    # expected secrets - enabling this option without certificates
    # will result in LDS rejection and the ingress will not work.
    enableHttps: false

  proxy:
    image: proxyv2

    # cluster domain. Default value is "cluster.local".
    clusterDomain: "cluster.local"

    # Resources for the sidecar.
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 2000m
        memory: 128Mi

    # Controls number of Proxy worker threads.
    # If set to 0 (default), then start worker thread for each CPU thread/core.
    concurrency: 2

    # Configures the access log for each sidecar.
    # Options:
    #   "" - disables access log
    #   "/dev/stdout" - enables access log
    accessLogFile: ""

    # Configure how and what fields are displayed in sidecar access log. Setting to
    # empty string will result in default log format
    accessLogFormat: ""

    # Configure the access log for sidecar to JSON or TEXT.
    accessLogEncoding: TEXT

    # Configure the DNS refresh rate for Envoy cluster of type STRICT_DNS
    # 5 seconds is the default refresh rate used by Envoy
    dnsRefreshRate: 5s

    #If set to true, istio-proxy container will have privileged securityContext
    privileged: false

    # If set, newly injected sidecars will have core dumps enabled.
    enableCoreDump: false

    # Default port for Pilot agent health checks. A value of 0 will disable health checking.
    statusPort: 15020

    # The initial delay for readiness probes in seconds.
    readinessInitialDelaySeconds: 1

    # The period between readiness probes.
    readinessPeriodSeconds: 2

    # The number of successive failed probes before indicating readiness failure.
    readinessFailureThreshold: 30

    # istio egress capture whitelist
    # https://istio.io/docs/tasks/traffic-management/egress.html#calling-external-services-directly
    # example: includeIPRanges: "<IP_ADDRESS>/16,<IP_ADDRESS>/16"
    # would only capture egress traffic on those two IP Ranges, all other outbound traffic would
    # be allowed by the sidecar
    includeIPRanges: "*"
    excludeIPRanges: ""

    # pod internal interfaces
    kubevirtInterfaces: ""

    # istio ingress capture whitelist
    # examples:
    #     Redirect no inbound traffic to Envoy:    --includeInboundPorts=""
    #     Redirect all inbound traffic to Envoy:   --includeInboundPorts="*"
    #     Redirect only selected ports:            --includeInboundPorts="80,8080"
    includeInboundPorts: "*"
    excludeInboundPorts: ""

    # This controls the 'policy' in the sidecar injector.
    autoInject: enabled

    # Sets the destination Statsd in envoy (the value of the "--statsdUdpAddress" proxy argument
    # would be <host>:<port>).
    # Disabled by default.
    # The istio-statsd-prom-bridge is deprecated and should not be used moving forward.
    envoyStatsd:
      # If enabled is set to true, host and port must also be provided. Istio no longer provides a statsd collector.
      enabled: false
      host: # example: statsd-svc.istio-system
      port: # example: 9125

    # Sets the Envoy Metrics Service address, used to push Envoy metrics to an external collector
    # via the Metrics Service gRPC API. This contains detailed stats information emitted directly
    # by Envoy and should not be confused with the the Istio telemetry. The Envoy stats are also
    # available to scrape via the Envoy admin port at either /stats or /stats/prometheus.
    #
    # See https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/metrics/v2/metrics_service.proto
    # for details about Envoy's Metrics Service API.
    #
    # Disabled by default.
    envoyMetricsService:
      enabled: false
      host: # example: metrics-service.istio-system
      port: # example: 15000

    # Specify which tracer to use. One of: lightstep, zipkin, datadog
    tracer: "zipkin"

  proxy_init:
    # Base name for the proxy_init container, used to configure iptables.
    image: proxy_init

  # imagePullPolicy is applied to istio control plane components.
  # local tests require IfNotPresent, to avoid uploading to dockerhub.
  # TODO: Switch to Always as default, and override in the local tests.
  imagePullPolicy: IfNotPresent

  # controlPlaneMtls enabled. Will result in delays starting the pods while secrets are
  # propagated, not recommended for tests.
  controlPlaneSecurityEnabled: false

  # disablePolicyChecks disables mixer policy checks.
  # if mixer.policy.enabled==true then disablePolicyChecks has affect.
  # Will set the value with same name in istio config map - pilot needs to be restarted to take effect.
  disablePolicyChecks: true

  # policyCheckFailOpen allows traffic in cases when the mixer policy service cannot be reached.
  # Default is false which means the traffic is denied when the client is unable to connect to Mixer.
  policyCheckFailOpen: false

  # EnableTracing sets the value with same name in istio config map, requires pilot restart to take effect.
  enableTracing: true

  # Configuration for each of the supported tracers
  tracer:
    # Configuration for envoy to send trace data to LightStep.
    # Disabled by default.
    # address: the <host>:<port> of the satellite pool
    # accessToken: required for sending data to the pool
    # secure: specifies whether data should be sent with TLS
    # cacertPath: the path to the file containing the cacert to use when verifying TLS. If secure is true, this is
    #   required. If a value is specified then a secret called "lightstep.cacert" must be created in the destination
    #   namespace with the key matching the base of the provided cacertPath and the value being the cacert itself.
    #
    lightstep:
      address: ""                # example: lightstep-satellite:443
      accessToken: ""            # example: abcdefg1234567
      secure: true               # example: true|false
      cacertPath: ""             # example: /etc/lightstep/cacert.pem
    zipkin:
      # Host:Port for reporting trace data in zipkin format. If not specified, will default to
      # zipkin service (port 9411) in the same namespace as the other istio components.
      address: ""
    datadog:
      # Host:Port for submitting traces to the Datadog agent.
      address: "$(HOST_IP):8126"

  # Default mtls policy. If true, mtls between services will be enabled by default.
  mtls:
    # Default setting for service-to-service mtls. Can be set explicitly using
    # destination rules or service annotations.
    enabled: true

  # ImagePullSecrets for all ServiceAccount, list of secrets in the same namespace
  # to use for pulling any images in pods that reference this ServiceAccount.
  # For components that don't use ServiceAccounts (i.e. grafana, servicegraph, tracing)
  # ImagePullSecrets will be added to the corresponding Deployment(StatefulSet) objects.
  # Must be set for any clustser configured with private docker registry.
  imagePullSecrets:
    # - private-registry-key

  # Specify pod scheduling arch(amd64, ppc64le, s390x) and weight as follows:
  #   0 - Never scheduled
  #   1 - Least preferred
  #   2 - No preference
  #   3 - Most preferred
  arch:
    amd64: 2
    s390x: 2
    ppc64le: 2

  # Whether to restrict the applications namespace the controller manages;
  # If not set, controller watches all namespaces
  oneNamespace: false

  # Default node selector to be applied to all deployments so that all pods can be
  # constrained to run a particular nodes. Each component can overwrite these default
  # values by adding its node selector block in the relevant section below and setting
  # the desired values.
  defaultNodeSelector: {}

  # Whether to perform server-side validation of configuration.
  configValidation: true

  # Custom DNS config for the pod to resolve names of services in other
  # clusters. Use this to add additional search domains, and other settings.
  # see
  # https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#dns-config
  # This does not apply to gateway pods as they typically need a different
  # set of DNS settings than the normal application pods (e.g., in
  # multicluster scenarios).
  # NOTE: If using templates, follow the pattern in the commented example below.
  #podDNSSearchNamespaces:
  #- global
  #- "[[ valueOrDefault .DeploymentMeta.Namespace \"default\" ]].global"

  # If set to true, the pilot and citadel mtls will be exposed on the
  # ingress gateway
  meshExpansion:
    enabled: false
    # If set to true, the pilot and citadel mtls and the plain text pilot ports
    # will be exposed on an internal gateway
    useILB: false

  multiCluster:
    # Set to true to connect two kubernetes clusters via their respective
    # ingressgateway services when pods in each cluster cannot directly
    # talk to one another. All clusters should be using Istio mTLS and must
    # have a shared root CA for this model to work.
    enabled: false

  # A minimal set of requested resources to applied to all deployments so that
  # Horizontal Pod Autoscaler will be able to function (if set).
  # Each component can overwrite these default values by adding its own resources
  # block in the relevant section below and setting the desired resources values.
  defaultResources:
    requests:
      cpu: 10m
    #   memory: 128Mi
    # limits:
    #   cpu: 100m
    #   memory: 128Mi

  # enable pod distruption budget for the control plane, which is used to
  # ensure Istio control plane components are gradually upgraded or recovered.
  defaultPodDisruptionBudget:
    enabled: true
    # The values aren't mutable due to a current PodDisruptionBudget limitation
    # minAvailable: 1

  # Kubernetes >=v1.11.0 will create two PriorityClass, including system-cluster-critical and
  # system-node-critical, it is better to configure this in order to make sure your Istio pods
  # will not be killed because of low priority class.
  # Refer to https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass
  # for more detail.
  priorityClassName: ""

  # Use the Mesh Control Protocol (MCP) for configuring Mixer and
  # Pilot. Requires galley (`--set galley.enabled=true`).
  useMCP: true

  # The trust domain corresponds to the trust root of a system
  # Refer to https://github.com/spiffe/spiffe/blob/master/standards/SPIFFE-ID.md#21-trust-domain
  # Indicate the domain used in SPIFFE identity URL
  # The default depends on the environment.
  #   kubernetes: cluster.local
  #   else:  default dns domain
  trustDomain: ""

  # Set the default behavior of the sidecar for handling outbound traffic from the application:
  # ALLOW_ANY - outbound traffic to unknown destinations will be allowed, in case there are no
  #   services or ServiceEntries for the destination port
  # REGISTRY_ONLY - restrict outbound traffic to services defined in the service registry as well
  #   as those defined through ServiceEntries
  # ALLOW_ANY is the default in 1.1.  This means each pod will be able to make outbound requests
  # to services outside of the mesh without any ServiceEntry.
  # REGISTRY_ONLY was the default in 1.0.  If this behavior is desired, set the value below to REGISTRY_ONLY.
  outboundTrafficPolicy:
    mode: ALLOW_ANY

  # The namespace where globally shared configurations should be present.
  # DestinationRules that apply to the entire mesh (e.g., enabling mTLS),
  # default Sidecar configs, etc. should be added to this namespace.
  # configRootNamespace: istio-config

  # set the default set of namespaces to which services, service entries, virtual services, destination
  # rules should be exported to. Currently only one value can be provided in this list. This value
  # should be one of the following two options:
  # * implies these objects are visible to all namespaces, enabling any sidecar to talk to any other sidecar.
  # . implies these objects are visible to only to sidecars in the same namespace, or if imported as a Sidecar.egress.host
  #defaultConfigVisibilitySettings:
  #- '*'

  sds:
    # SDS enabled. IF set to true, mTLS certificates for the sidecars will be
    # distributed through the SecretDiscoveryService instead of using K8S secrets to mount the certificates.
    enabled: true
    udsPath: ""
    udsPath: "unix:/var/run/sds/uds_path"
    useNormalJwt: true

  # Configure the mesh networks to be used by the Split Horizon EDS.
  #
  # The following example defines two networks with different endpoints association methods.
  # For `network1` all endpoints that their IP belongs to the provided CIDR range will be
  # mapped to network1. The gateway for this network example is specified by its public IP
  # address and port.
  # The second network, `network2`, in this example is defined differently with all endpoints
  # retrieved through the specified Multi-Cluster registry being mapped to network2. The
  # gateway is also defined differently with the name of the gateway service on the remote
  # cluster. The public IP for the gateway will be determined from that remote service (not
  # supported yet).
  #
  # meshNetworks:
  #   network1:
  #     endpoints:
  #     - fromCidr: "<IP_ADDRESS>/24"
  #     gateways:
  #     - address: <IP_ADDRESS>
  #       port: 80
  #   network2:
  #     endpoints:
  #     - fromRegistry: reg1
  #     gateways:
  #     - registryServiceName: istio-ingressgateway
  #       port: 443
  #
  meshNetworks: {}

  # Specifies the global locality load balancing settings.
  # Locality-weighted load balancing allows administrators to control the distribution of traffic to
  # endpoints based on the localities of where the traffic originates and where it will terminate.
  # Please set either failover or distribute configuration but not both.
  #
  # localityLbSetting:
  #   distribute:
  #   - from: "us-central1/*"
  #     to:
  #       "us-central1/*": 80
  #       "us-central2/*": 20
  #
  # localityLbSetting:
  #   failover:
  #   - from: us-east
  #     to: eu-west
  #   - from: us-west
  #     to: us-east
  localityLbSetting: {}

  # Specifies whether helm test is enabled or not.
  # This field is set to false by default, so 'helm template ...'
  # will ignore the helm test yaml files when generating the template
  enableHelmTest: false

Environment where bug was observed (cloud vendor, OS, etc)
AWS
Kube cluster brought up via Kops
Affected product area (please put an X in all that apply)
[ ] Configuration Infrastructure
[x ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[x ] User Experience
Additionally, please consider attaching a cluster state archive by attaching
the dump file to this issue.
istio-dump.tar.gz

@linsun @mbanikazemi I am reading over the link you just provided.  Everything i'm reading suggests that I need the serviceentry to allow it to go outbound but that doesn't seem to work (will keep on rereading it to see if I have not missed anything).
https://istio.io/blog/2018/egress-https/#enable-https-access-to-a-google-books-web-service
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: googleapis
spec:
  hosts:
  - www.googleapis.com
  ports:
  - number: 443
    name: https
    protocol: HTTPS
  location: MESH_EXTERNAL
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: googleapis
spec:
  hosts:
  - www.googleapis.com
  tls:
  - match:
    - port: 443
      sni_hosts:
      - www.googleapis.com
    route:
    - destination:
        host: www.googleapis.com
        port:
          number: 443
      weight: 100

I'm wondering if it is because of my istio setup (pasted the entire helm template install values above)?  Or how to debug this and get more information on why it is being blocked.
Im in another pod trying to cURL this url and I get:
bash-4.4# curl -v https://www.googleapis.com
* Rebuilt URL to: https://www.googleapis.com/
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to www.googleapis.com (<IP_ADDRESS>) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
* Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

Any tips on what info to try to look at or get to troubleshoot this?

Are all Istio pods up and running? what does kubectl get pod -n istio-system and kubectl get pod show?
Does the booking deployed within a single cluster work as expected?

@mbanikazemi
$ kubectl -n istio-system get pods -o wide
NAME                                         READY   STATUS             RESTARTS   AGE   IP              NODE                            NOMINATED NODE
grafana-668b548d7b-vms4c                     1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>
istio-citadel-d4977f4ff-wn5vx                1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>
istio-cleanup-secrets-1.1.5-gc56r            0/1     Completed          0          36m   <IP_ADDRESS>   ip-172-17-51-50.ec2.internal    <none>
istio-egressgateway-75697444bb-bmr5p         1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>
istio-galley-76478ddd-j2wkm                  1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>
istio-ingressgateway-6f8fcb6d7d-7p6xn        1/1     Running            0          14m   <IP_ADDRESS>    ip-172-17-50-105.ec2.internal   <none>
istio-ingressgateway-mqtt-7f59c4dcdb-56dbw   1/1     Running            0          14m   <IP_ADDRESS>   ip-172-17-51-50.ec2.internal    <none>
istio-init-crd-10-hn9cb                      0/1     Completed          0          1h    <IP_ADDRESS>    ip-172-17-51-50.ec2.internal    <none>
istio-init-crd-11-zc45s                      0/1     Completed          0          1h    <IP_ADDRESS>    ip-172-17-51-50.ec2.internal    <none>
istio-init-crd-certmanager-10-gml6x          0/1     Completed          0          1h    <IP_ADDRESS>    ip-172-17-51-116.ec2.internal   <none>
istio-init-crd-certmanager-11-kkfd2          0/1     Completed          0          1h    <IP_ADDRESS>    ip-172-17-51-50.ec2.internal    <none>
istio-pilot-5b5fdb6b4d-g46nc                 2/2     Running            0          36m   <IP_ADDRESS>    ip-172-17-50-253.ec2.internal   <none>
istio-policy-54dc466b6b-ndhf2                2/2     Running            1          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>
istio-security-post-install-1.1.5-njzlq      0/1     Completed          0          36m   <IP_ADDRESS>   ip-172-17-51-50.ec2.internal    <none>
istio-sidecar-injector-5cf654f4db-g7ccw      1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>
istio-tracing-74465dd996-xblbn               1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>
kiali-7f74567756-287qv                       1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-50-253.ec2.internal   <none>
prometheus-587cddcf4b-88vtt                  1/1     Running            0          36m   <IP_ADDRESS>    ip-172-17-51-151.ec2.internal   <none>

Everything looks like it is running fine.  I have an istio gateway/virtualservices routing inbound to the apps.
I am also able to do this to get traffic outbound with the HTTPS origination from the proxy instead of my app with this config.  This is following the book example.
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: managedkube
spec:
  hosts:
  - managedkube.com
  ports:
  - number: 80
    name: http
    protocol: HTTP
  - number: 443
    name: https
    protocol: HTTPS
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: rewrite-port-for-managedkube
spec:
  hosts:
  - managedkube.com
  http:
  - match:
    - port: 80
    route:
    - destination:
        host: managedkube.com
        port:
          number: 443

---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: originate-tls-for-managedkube
spec:
  host: managedkube.com
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
    portLevelSettings:
    - port:
        number: 443
      tls:
        mode: SIMPLE # initiates HTTPS when accessing managedkube.com
```

So it seems like the serviceentry/virtualservices are working?

@mbanikazemi can you help diagnose this?  Is this something we should add some automated testing around?
@sekka1 @vadimeisenbergibm  is on vacation fyi.   For blogs, we don't guarantee it will always work.

@sekka1 could you follow instruction here instead?  https://istio.io/docs/tasks/traffic-management/secure-ingress/.

If I add this, it works.  Pods can reach external now.
global:
  proxy:
    includeIPRanges: "<IP_ADDRESS>/16"

I was hoping to be able to selectively limit items and use the serviceentries

Isn't outboundTrafficPolicy.mode: ALLOW_ANY suppose to let any outbound connection to go through? I have this setup and my pods can't connect anywhere...

@sekka1 the error you posted above included:
 TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

is it possible an old version on client is being used? and if upgraded things may change? this is just a guess on my part so I don't want to make extra work for you but it may be worth following up along that line?

I've been having the same issue and finally resolved it today with help from this link..
https://discuss.istio.io/t/serviceentry-for-https-on-httpbin-org-resulting-in-connect-cr-srvr-hello-using-curl/2044/3
For me, I was deploying a dedicated ingress gateway for my application and as part of that I added a port called 'http-port443'.
This in tern was creating a service (as part of the ingress deployment) and I then hit the same problem described in the above link.
My fix was to rename the port to 'https-port443' and now egress is working correctly.
This and your workaround leads me to think that the problem is with how the sidecar forwards https traffic to the egress but I don't know enough about this to dig deeper and confirm if this is a bug or not.
Hope this helps

@chr15murray - I'm able to reach the world just fine from inside the cluster, I just can't make google.mydomain.com proxy. It looks like your problem is different, but I may be missing something... Thanks for your response. :)

Hi @mabushey
It may well be different but we share a symptom which is how I came across this and why I mentioned it. Specifically your curl test generated curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
Looking at your config you have a gateway called istio-ingressgateway-mqtt with a port 443 named 'mqtt'. If your problem is the same as me then renaming this to 'https-mqtt' or maybe tcp-mqtt could resolve your issue.
maybe worth a try if you can.

1.1.7 here - cannot do apt-get update and etc. outgoing traffic blocked despite ALLOW_ANY policy enabled. Also documentation says:
In versions prior to Istio 1.1.4, ALLOW_ANY only worked on ports with no HTTP services or service entries defined within the mesh. External hosts using the same port as any internal HTTP service fell back to a blocking-by-default behavior. Because some ports, such as port 80, have HTTP services inside Istio by default, prior to Istio 1.1.3 you couldn’t call external services on any of those ports either.

but it seemsIstio still blocks traffic as before. Cannot solve this with  ServiceEntry, only includeIPRanges works so far.

@eigokor Same behavior here, I can't get anything to communicate externally unless ServiceEntry or IncludeIPRanges are created. Using "ALLOW_ANY" changes absolutely nothing... Same behavior with/without that.
The tipping was when Istio broke our RabbitMQ deployment. I'm removing istio in the meantime, I will try again in the future.

We are tracking this in https://github.com/istio/istio/issues/16458. Thanks for all the reports to help us pin down this issue.


Updating the document to current information for custom installation …

…for AADC
Optional settings are currently unselected (all boxes unchecked) by default, so users would have to "select" instead of "clear" the appropriate boxes in this step.

@yukieryu : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@billmath
Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


Wrongly formatted Self-attested claims JSON example

Dear Team,
In the section called "Input type: Self-attested claims", the JSON file is wrongly formatted. It has too many commas.
This is how it should be to make it work:
{   "attestations": {     "selfIssued": {       "mapping": {         "alias": {           "claim": "name"         }       }     }   },   "validityInterval": 2592001,   "vc": {     "type": [       "ProofOfNinjaNinja"     ]   } }

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f47338c7-5503-fab8-11fd-f3260dc568bc
Version Independent ID: 2304a681-074e-755a-05f8-884aaa76518c
Content: How to customize your Azure Active Directory Verifiable Credentials (preview)
Content Source: articles/active-directory/verifiable-credentials/credential-design.md
Service: active-directory
Sub-service: verifiable-credentials
GitHub Login: @barclayn
Microsoft Alias: barclayn

#please-close


Installation from Maven failing on Databricks

Installation using the coordinates "com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc1" is failing on Databricks clusters.  I'm using DBR 6.1 with Spark 2.4.4 and Scala 2.11.  I'm also using the clusters UI to install the library.
The full stack trace:
Library resolution failed. Cause: java.lang.RuntimeException: unresolved dependency: com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc1: not found 	at com.databricks.libraries.server.MavenInstaller.doDownloadMavenPackages(MavenLibraryResolver.scala:438) 	at com.databricks.libraries.server.MavenInstaller.$anonfun$downloadMavenPackages$2(MavenLibraryResolver.scala:381) 	at com.databricks.backend.common.util.FileUtils$.withTemporaryDirectory(FileUtils.scala:431) 	at com.databricks.libraries.server.MavenInstaller.$anonfun$downloadMavenPackages$1(MavenLibraryResolver.scala:380) 	at com.databricks.logging.UsageLogging.$anonfun$recordOperation$4(UsageLogging.scala:417) 	at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:239) 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) 	at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:234) 	at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:231) 	at com.databricks.libraries.server.MavenInstaller.withAttributionContext(MavenLibraryResolver.scala:57) 	at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:276) 	at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:269) 	at com.databricks.libraries.server.MavenInstaller.withAttributionTags(MavenLibraryResolver.scala:57) 	at com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:398) 	at com.databricks.logging.UsageLogging.recordOperation$(UsageLogging.scala:337) 	at com.databricks.libraries.server.MavenInstaller.recordOperation(MavenLibraryResolver.scala:57) 	at com.databricks.libraries.server.MavenInstaller.downloadMavenPackages(MavenLibraryResolver.scala:379) 	at com.databricks.libraries.server.MavenInstaller.downloadMavenPackagesWithRetry(MavenLibraryResolver.scala:137) 	at com.databricks.libraries.server.MavenInstaller.resolveMavenPackages(MavenLibraryResolver.scala:113) 	at com.databricks.libraries.server.MavenLibraryResolver.resolve(MavenLibraryResolver.scala:44) 	at com.databricks.libraries.server.ManagedLibraryManager$GenericManagedLibraryResolver.resolve(ManagedLibraryManager.scala:263) 	at com.databricks.libraries.server.ManagedLibraryManagerImpl.$anonfun$resolvePrimitives$1(ManagedLibraryManagerImpl.scala:193) 	at com.databricks.libraries.server.ManagedLibraryManagerImpl.$anonfun$resolvePrimitives$1$adapted(ManagedLibraryManagerImpl.scala:188) 	at scala.collection.Iterator.foreach(Iterator.scala:941) 	at scala.collection.Iterator.foreach$(Iterator.scala:941) 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) 	at scala.collection.IterableLike.foreach(IterableLike.scala:74) 	at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 	at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 	at com.databricks.libraries.server.ManagedLibraryManagerImpl.resolvePrimitives(ManagedLibraryManagerImpl.scala:188) 	at com.databricks.libraries.server.ManagedLibraryManagerImpl$ClusterStatus.installLibs(ManagedLibraryManagerImpl.scala:772) 	at com.databricks.libraries.server.ManagedLibraryManagerImpl$InstallLibTask$1.run(ManagedLibraryManagerImpl.scala:473) 	at com.databricks.threading.NamedExecutor$$anon$1.$anonfun$run$1(NamedExecutor.scala:317) 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) 	at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:239) 	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) 	at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:234) 	at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:231) 	at com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:256) 	at com.databricks.threading.NamedExecutor$$anon$1.run(NamedExecutor.scala:317) 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 	at java.lang.Thread.run(Thread.java:748)

@conorbmurphy I believe there is already an issue for this here:
https://github.com/Azure/mmlspark/issues/715
also adding @mhamilton723

Yes, looks like the same issue.  Thanks!

Adding "https://mmlspark.azureedge.net/maven" to the repository field fixed the issue for me.

@imatiach-msft @mhamilton723
For the next release, would it be possible to use the same repository used for version 0.18.x? In some very restrictive environments adding an additional repo is a problem (I still could not test v1.0.0-rc1)

@mhamilton723 / @imatiach-msft  can we close this?

Is there a fix for this? I'm having the same problem and @Hugheym 's fix didn't work for me.
Thanks


New clinical analyte assays template: bottom up from EHR

@cstoeckert  and I have prepared a new assays template, designed to be executed with ROBOT, and using the same column layout as the established OBI assays template.
Construction of this template was based on inspection of order, specimen and result names in an Electronic Health Record database, as opposed to interpretation of links to any external coding system.
I will be bringing this template up for discussion on the Monday, November 16th OBI call. OBI developers and friends are invited to examine the template in advance of that meeting.

Can I share this with the CMO folks? They responded to me and are
interested in following what OBI is doing.
On Tue, Nov 10, 2020 at 12:50 PM Mark A. Miller<EMAIL_ADDRESS>wrote:

@cstoeckert https://github.com/cstoeckert and I have prepared a new
assays template
https://docs.google.com/spreadsheets/d/1muJJSitpHF0nqWOK1gcvd7tXSOaNZw9JNCe21eoSIfU/edit?usp=sharing,
designed to be executed with ROBOT, and using the same column layout as the
established OBI assays template.
Construction of this template was based on inspection of order, specimen
and result names in an Electronic Health Record database, as opposed to
interpretation of links to any external coding system.
I will be bringing this template up for discussion on the Monday, November
16th OBI call. OBI developers and friends are invited to examine the
template in advance of that meeting.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/obi-ontology/obi/issues/1253, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKDY44WWN66AZYPTIA2JNTSPGRRBANCNFSM4TREQ3ZA
.

--
Randi Vita, M.D.
Lead Ontology and Quality Manager
Immune Epitope Database and Analysis Project
La Jolla Institute for Allergy & Immunology
9420 Athena Circle
La Jolla, Ca 92037
<EMAIL_ADDRESS>www.immuneepitope.org
858-752-6912

All feedback is welcome!

Hello Mark and Chris,
A few comments/suggestions/questions regarding some of the terms in the spreadsheet. I hope these don't overlap too much with what we discussed during the call.

For the terms that are more simple such as rows 30-183, perhaps follow the format of current assay terms - for example: instead of 'assay for acetone' use 'acetone assay'.
I apologize if you mentioned this during the call and I missed it, but I'm confused by the terms with 'molecular entity' in their names. For example: row 66, 'assay for calcium molecular entity' - why wouldn't this just be 'assay for calcium' (or if you decide to go with the suggestion in (1) above 'calcium assay'). Does it have something to do with distinguishing elemental forms of the substance?
Row 184 'calcium molecular entity in venous blood by basic metabolic panel assay' has a different format than the other terms in this group in rows 185-191.
As discussed during the call, I think it might be more consistent with existing term formats if the names were arranged so that the panel (and other orders like that) were made to be modifiers of the root assay. For example, in the group mentioned above in (3) could you use a name along the lines of 'basic metabolic panel venous blood carbon dioxide assay', as it makes the panel information a modifier of carbon dioxide assay.
Question related to (4), why is it important to keep track of what kind of order was used for the test? Is it because the assays done in the different panels/orders are different from each other?
The names in rows 192-198 seem a bit redundant with themselves. For example, could 'bodily fluid albumin assay for albumin in bodily fluid' be renamed as just 'bodily fluid albumin assay'? I'm guessing these are names that were generated by the naming script and that would be candidates for manual curation? Similar situation for rows 271-290
I suggest instead of making all of these new terms direct children of 'analyte assay', that perhaps you could group related terms involving the same analyte under a grouping parent. For example (picking just two of the many calcium assay terms in your list): row 184 'calcium molecular entity in venous blood by basic metabolic panel assay' and row 201 'bone panel assay for calcium molecular entity in venous blood'. Perhaps make these (and all the other calcium assay terms in your list) children of a new term 'calcium assay', which itself would be a child of 'analyte assay'. This way, people could easily find all of the calcium assay terms as well as have a term to use when they don't know how the assay was ordered or if the assay was done outside a clinical setting.
For the protein related terms, there are a couple of existing assay terms involving proteins ('protein localization assay', 'protein-protein interaction detection assay'). I was thinking that maybe using 'protein abundance' in the new term names instead of just 'protein' might help to distinguish these. Also, the other protein terms are not under 'analyte assay'. Is a protein considered an analyte? I tried looking around in OBI and didn't see anything that said one way or the other.
There is an existing term 'measuring glucose concentration in blood serum assay' OBI:0000418  that overlaps with some of your proposed glucose terms.
There is an existing term 'clinical chemistry assay' OBI:0000520 that could be a parent to many of your terms. Although I haven't thought about the implications of that. But either way, consider what to do about relating your terms to that term.

These are some examples of things that jumped out at me when I skimmed through the list.
I hope you find this helpful. Thanks for reading,
Michelle

Thank you very much, @mgiglio99!
@cstoeckert and I are on board with most of your label improvements numbered 1 - 6. There's several improvement patterns in there, so we'll need to think through how much of them can be implemented with software rules, since we're trying to do this in bulk for as many research-relevant lab results as possible.
My sense is that we will also act on many of your hierarchy suggestions numbered 7 - 10. We just haven't discussed that as much between the two of us yet.
Regarding your comment 2
We're not trying to add any emphasis or clarification to any parts of the labels, just assemble the assay labels from the order text, the label of the evaluant term and the label of the analyte term. In the case of calcium lab results, sometimes the orders do specify "ionized calcium", but even if it just says "calcium" we're using CHEBI:22985, whose label is "calcium molecular entity." We'll look into removing "molecular entity" from the labels, but we want to do it in a way that's generalization to other patters and acceptable to clinicians.
Regarding your comment 5
The orders frequently contain differentia that aren't present in the other two data types we're processing: specimen and "result item". We have seen some of these differentia in our review of other systems that use lab results to build research cohorts. For example, direct bilirubin measurements vs. indirect measurements. We're thinking about turning these differentia into objective, detection technique, or instrument axioms in the future. But for now we'll probably just capture in the labels... unless they're redundant with other parts of the label!

Some more (overdue!) responses to @m3giglio99
Point 3: the format of that label is different because @cstoeckert manually experimented with some different patterns while we were having a Zoom call (November 16th?) and we forgot to revert it. Let me know if you want me to explain how to view the revision history in Google Sheets.
Point 8: At least some proteins modeled by the Protein Ontology, like 'alkaline phosphatase' PR:000023540 are asserted to subclasses of PR's protein term, PR:000000001, which the Protein Ontology asserts to be a subclass of ChEBI's 'molecular entity', CHEBI:23367
Note that ChEBI has its own 'protein' term, CHEBI:36080, for what it's worth.

@cstoeckert and I have updated this "lab results" based assays template based on recent OBI calls.

minor updates to class labels
assays that measure the abundance of cells are now considered to be subclasses of 'cytometry assay'
assays have been modelled, using equivalent class logic, based on any analyte or target entity that appear at least twice in the template. under OWL reasoning, 'basic metabolic panel venous blood carbon dioxide assay' is classified as a 'carbon dioxide assay', along with 16 other assays.
equivalent-class parents are not created when a parent already exists on OBI, but that's based on a single pattern: assay's specified output is a value specification that specifies the values of some quality. The template has some specific kinds of temperature and volume assays, but new temperature and volume superclasses are not include, as assays matching the specified pattern already exist in OBI. Unfortunately, 'mass value specification' follows a different pattern.

@cstoeckert and I have updated this "lab results" based assays template based on recent OBI calls.

minor updates to class labels
assays that measure the abundance of cells are now considered to be subclasses of 'cytometry assay'
assays have been modelled, using equivalent class logic, based on any analyte or target entity that appear at least twice in the template. under OWL reasoning, 'basic metabolic panel venous blood carbon dioxide assay' is classified as a 'carbon dioxide assay', along with 16 other assays.
equivalent-class parents are not created when a parent already exists on OBI, but that's based on a single pattern: assay's specified output is a value specification that specifies the values of some quality. The template has some specific kinds of temperature and volume assays, but new temperature and volume superclasses are not include, as assays matching the specified pattern already exist in OBI. Unfortunately, 'mass value specification' follows a different pattern.

Hi,
Sorry for the delay. Just a few things I wanted to ask you about the parent-child relationships. It may be that I'm just confused, but I wanted to make sure.

Above you said that you were grouping some of the new terms together under new parents and specifically mentioned 'basic metabolic panel venous blood carbon dioxide assay' to be a child of the new parent 'carbon dioxide assay'. But when I look in the spreadsheet, that term has 'analyte assay' in the parent column, not 'carbon dioxide assay'. Also, the definition of 'basic metabolic panel venous blood carbon dioxide assay' should start with "A carbon dioxide assay that measures..." rather than "An analyte assay that measures....".  The same is true for the other 16 carbon dioxide terms.
The above seems to be generally true for all of the terms with CLASS_TYPE equivalent - which I infer are the new parent terms. None of these terms are ever used in the parent column.
I'm wondering if there is inconsistency in parentage for some of the terms. For example, 'venous blood 25-hydroxyvitamin D2 assay' is listed as a child of 'analyte assay', while 'venous blood lactic acid assay' is listed as a child of 'assay'. Should they both be children of 'analyte assay'? This is just one of several similar examples.

Thanks,
Michelle

Hi,
Sorry for the delay. Just a few things I wanted to ask you about the parent-child relationships. It may be that I'm just confused, but I wanted to make sure.

Above you said that you were grouping some of the new terms together under new parents and specifically mentioned 'basic metabolic panel venous blood carbon dioxide assay' to be a child of the new parent 'carbon dioxide assay'. But when I look in the spreadsheet, that term has 'analyte assay' in the parent column, not 'carbon dioxide assay'. Also, the definition of 'basic metabolic panel venous blood carbon dioxide assay' should start with "A carbon dioxide assay that measures..." rather than "An analyte assay that measures....".  The same is true for the other 16 carbon dioxide terms.
The above seems to be generally true for all of the terms with CLASS_TYPE equivalent - which I infer are the new parent terms. None of these terms are ever used in the parent column.
I'm wondering if there is inconsistency in parentage for some of the terms. For example, 'venous blood 25-hydroxyvitamin D2 assay' is listed as a child of 'analyte assay', while 'venous blood lactic acid assay' is listed as a child of 'assay'. Should they both be children of 'analyte assay'? This is just one of several similar examples.

Thanks,
Michelle

Thanks @mgiglio99
You first two points touch on a design decision we made, specifically to assert fairly general parent classes for our assays, and to impose any additional hierarchy though reasoning. As you observed, 'basic metabolic panel venous blood carbon dioxide assay' (putatively OBI:2100093) is asserted to be a subclass of 'analyte assay' in the template. However, after executing the template, reasoning with Hermit, and re-saving, it is only asserted to be a subclass of 'carbon dioxide assay' (OBI:2100001), along with various anonymous class expressions. This approach allows us to create additional parent classes in the future (maybe a venous blood assay class?) and discover which more specific assays are subsumed. I have attached a working copy of the assays, reasoned in the presence of OBI.
Having taken the hierarchical approach described above, we followed standard OBO principles for building the definitions: an X that ..., where X is the directly asserted parent class.

Your third point touches on the question of what's eligible to bear an analyte role in an analyte assay. The logical definition for 'analyte assay' doesn't make any reference to analyte role, but the textual definition of 'analyte role' does make reference to 'analyte assay', and one of the logical definitions for 'analyte role' is 'role of' some ('molecular entity' or atom)
We have asserted that the thing being measured in a lactic acid assay (OBI:2100042) is racemic lactic acid. For background, the ChEBI page for that compound includes text from Wikipedia describing the various biological contexts in which one would find either of the enantiomers. I haven't felt  like we can take the authoritative position that a lactic acid assay, as performed in health care settings, can or does only measure one enantiomer of lactic acid.
Having said that, ChEBI doesn't not consider racemic mixtures to be molecular entities, so racemic lactic acid can't bear an analyte role. By extension, we are not asserting an assay for racemic lactic acid to be an analyte assay.

Thanks @mgiglio99
You first two points touch on a design decision we made, specifically to assert fairly general parent classes for our assays, and to impose any additional hierarchy though reasoning. As you observed, 'basic metabolic panel venous blood carbon dioxide assay' (putatively OBI:2100093) is asserted to be a subclass of 'analyte assay' in the template. However, after executing the template, reasoning with Hermit, and re-saving, it is only asserted to be a subclass of 'carbon dioxide assay' (OBI:2100001), along with various anonymous class expressions. This approach allows us to create additional parent classes in the future (maybe a venous blood assay class?) and discover which more specific assays are subsumed. I have attached a working copy of the assays, reasoned in the presence of OBI.
Having taken the hierarchical approach described above, we followed standard OBO principles for building the definitions: an X that ..., where X is the directly asserted parent class.

Your third point touches on the question of what's eligible to bear an analyte role in an analyte assay. The logical definition for 'analyte assay' doesn't make any reference to analyte role, but the textual definition of 'analyte role' does make reference to 'analyte assay', and one of the logical definitions for 'analyte role' is 'role of' some ('molecular entity' or atom)
We have asserted that the thing being measured in a lactic acid assay (OBI:2100042) is racemic lactic acid. For background, the ChEBI page for that compound includes text from Wikipedia describing the various biological contexts in which one would find either of the enantiomers. I haven't felt  like we can take the authoritative position that a lactic acid assay, as performed in health care settings, can or does only measure one enantiomer of lactic acid.
Having said that, ChEBI doesn't not consider racemic mixtures to be molecular entities, so racemic lactic acid can't bear an analyte role. By extension, we are not asserting an assay for racemic lactic acid to be an analyte assay.

Hi
I wondered if there was a reasoning thing going on for points one and two. I don't understand how you encoded that though - I'll have to learn that at some point so I don't make this mistake again.  :)
Thanks

Hi
I wondered if there was a reasoning thing going on for points one and two. I don't understand how you encoded that though - I'll have to learn that at some point so I don't make this mistake again.  :)
Thanks

@turbomam Is there a new Google Sheet for this?

@jamesaoverton I don't think there's a new sheet. Do you see some disparity? The script that built the assays reads from a local delimited text file. I can post that if necessary.

When I tried to access the Google Sheet linked in your issue description, Google told me that it had been moved to Trash and would soon not be available (or something).

Ok, I'll deal with that in a few hours.
On Tue, Feb 9, 2021, 12:03 James A. Overton<EMAIL_ADDRESS>wrote:

When I tried to access the Google Sheet linked in your issue description,
Google told me that it had been moved to Trash and would soon not be
available (or something).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/obi-ontology/obi/issues/1253#issuecomment-776089555,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGAEX6JEA47PX6L4CDMERW3S6FTGJANCNFSM4TREQ3ZA
.

I took the Google Sheet out of the trash and saw that it used a different OBI ID range (starting at 3000000) from the range that was used in the pull request (starting ~ 2000000). I'll link the correct sheet in an hour or two.

I had the Google Sheet with the correct OBI IDs in my Google Drive, so I changed the link in the top of this issue. Does that look OK to you @jamesaoverton ? Thanks for catching the broken link.

Great, thanks!

Oops, just made it world-suggestable


May I inquire about the output indexes of the DPP, whether it starts from 0 or 1?

I have just start to use this library. And I am confused about this.

Hi @2019zhou,
With Python indices start from 0.
The kernel you have used might be associated to a very low inclusion probability P[0 \in X] = K_00
In this case, 0 won't appears that often in your samples.


Deleting documents using JSON-style matcher won't work

This works:
db.doggies.destroy({id : 2}, function(err,res){
//stinky smelled horrible
});
This doesn't:
db.doggies.destroy({"body ->> 'name'" : "Stinky"}, function(err,res){
//deletes records where name is "Stinky"
});
Tested on PostgreSQL 9.4.4

As I recall, the destroy() method is not implemented currently in a manner allowing criteria against documents. The current implementation only works against column-based criteria.
I'm thinking this is a candidate for an enhancement.
@robconery thoughts?

That changed a little while back when I added support for JSON traversal in the find-style args. This unit test passes:

    it('deletes by matching json', function (done) {
      db.docs.destroy({'body->>title': 'Starsky and Hutch'}, function(err, deleted) {
        assert.equal(deleted.length, 1);

        db.docs.find({id: deleted[0].id}, function (err, found) {
          assert.equal(err, null);
          assert.equal(found.length, 0);
          done();
        });
      });
    });

Note the omission of the single-quotes around the JSON field name ("body->>title" vs "body->>'title'"); Massive adds those itself after splitting on the possible operators. Vincent, that's probably what's messing your call up -- give it a shot without those.

Looks like the docs are incorrect -- sorry about that! I'll see if I can make it a little less sensitive about whitespace at the very least though.

Yes, that did work!

#171 allows pretty much any legal combination of whitespace and quoting so the docs shouldn't need to be updated. I don't know when Rob's planning to release the next version though.


OLD Luma not compatible with NEW boot9strap

Currently, the guide uses old Luma in conjuction with new B9S. That gives a loader error in Luma. I would reccomend to switching to a Luma nightly to avoid updating B9S.

New B9S has been marked as a pre-release not the latest for now. People should not be updating until the Luma version supporting it comes out.