Removes runStatus check, fixes some logic

This fixes an issue where the OpenAPI would not return the undocumented runStatus flag, which caused the plugin to query the OpenAPI continuously. Unfortunately, the moving flag is incorrectly missing from the API, so we have no way to detect if we're moving and thus the plugin doesn't update HomeKit correctly: it assumes that the blind tilt has stopped moving.

This works around a crasher in the OpenAPI where sending down;0 would return an error response, resulting in the plugin continuously querying its position and thus getting rate limited for a day. Workaround is to send closeDown for down;0 instead. Also sends closeUp and fullyOpen for the relevant cases.

Fixes an issue with use_tilt_for_direction not working correctly

@donavanbecker
Due to bugs in the OpenAPI, the plugin could hit its rate limit: when the tilt (and curtain!) are moving the plugin keeps querying the OpenAPI to update HomeKit. When something goes wrong, the querying is never stopped tho. It might be a good idea to add an internal rate-limiter somewhere in the plugin. E.g. after querying more than 60s for changes, we stop querying anyways and assume something went wrong. What do you think?

@AndreasVerhoeven I like the idea!!
Do you want me to merge this?
And then if you bring another PR with that enhancement, we can merge that PR.
No wonder I was hitting my rate limit.......

Also thanks for all of these enhancements.

I've never used discord, so no :) Would it help with development?

It could. It is designed for users and developers.
Main reason I asked is because then we could communicate off GitHub, about about changes..

It could. It is designed for users and developers.
Main reason I asked is because then we could communicate off GitHub, about about changes..

Just signed up and joined switchbot-channel !


failed to add React Native 0.5x.x

[Edited] The issue was from the app name which started with a digit. Apparently an issue!
Hi,
I got a problem when trying to pick a custom RN version. It used to work with this precise version.
However when I tried again for a new app I got this error:
$ ignite new 60plus --react-native-version 0.54.2
🔥 igniting app 60plus
✔ using the Infinite Red boilerplate v2 (code name 'Andross')
✖ failed to add React Native 0.54.2
if I try with the latest I got a different error which I believe it's normal:
💩  react native version 0.55.4 not found on NPM -- 0.42.0 recommended
since you might not add it.
But then I tried with 0.42.0 and I got:
✖ failed to add React Native 0.42.0
and without anything:
✖ failed to add React Native 0.50.4
Nothing special on
ignite doctor results:
System
platform           darwin
arch               x64
cpu                4 cores                     Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz
directory          /Users/vali/MyWorks/react
JavaScript
node               9.4.0        /Users/vali/.nvm/versions/node/v9.4.0/bin/node
npm                5.6.0        /Users/vali/.nvm/versions/node/v9.4.0/bin/npm
yarn               1.3.2        /usr/local/bin/yarn
React Native
react-native-cli   2.0.1
Ignite
ignite             2.0.0        /Users/vali/.nvm/versions/node/v9.4.0/bin/ignite
Android
java               1.8.0_111    /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/bin/java
android home       -            /Users/vali/Library/Android/sdk
iOS
xcode              9.3

Hey, thanks for the report. I just tried ignite new example --react-native-version=0.54.2 and it was successful. Maybe you were previously using the shorthand flag (which doesn't require the equals sign)?


Setting Environment Variable Works on Linux But Fails on Windows

Does this issue occur when all extensions are disabled?: Yes (Beside those of Python, Also on clean VS Code with only Python extensions).

VS Code Version: 1.65.2
OS Version: Windows 10.

Steps to Reproduce:

Open a Jupyter file.
Run the following code:

os.environ['KAGGLE_USERNAME'] = "kaggleuser"
os.environ['KAGGLE_KEY'] = "342618somekey4328179"

userName    = 'dmcgow'
dataSetName = 'birds-200'

!kaggle datasets download -d $userName/$dataSetName

This works on Linux yet fails on Windows.
Working on the cmd in Windows with:
set KAGGLE_USERNAME=kaggleuser
set KAGGLE_KEY=342618somekey4328179

kaggle datasets download -d dmcgow/birds-200

Works.
So it fails reading the API authentication data from environment variables.

Thanks for filing this issue, can you confirm the same code works in Juptyer Notebooks or Jupyter Lab (i.e. try this outside VS Code)

OK,
Just tried it again on version 1.73.1 and it works.
Maybe fixed on the os package or the IPython kernel.


[PROTOCOL] Per-file statistics documentation in protocol is ambiguous

Bug
Describe the problem
Per-file Statistics in protocol specification is a bit ambiguous and could provide more detail regarding the availability and format of per-file statistics for columns of specific data types. For instance, columns of array data type do not specify whether min and max statistics should be provided (and whether they are seems to be at the discretion of the engine being used). For instance, using DBR 15.2 with Spark 3.5.0 and Scala 2.12, the max and min statistics for arrays (nested or otherwise) are not provided, only the nullCount. Further, for nested arrays, only the nullCount of the nested array itself, not any of its fields, is given using the same settings as above. The _delta_log was generated by creating a Spark DataFrame and then df.write.format("delta").mode("append").save(storagePathway).
Moreover, the format of min and max statistics for type DateTime seems also to be up to the discretion of the engine. For example, whether HH:MM:SS are included in the min and max statistics or whether the min and max statistics are truncated to their "short" form of YYYY-MM-DD. Using DBR 15.2 with Spark 3.5.0 and Scala 2.12, the statistics were truncated to YYYY-MM-DD in the _delta_log.
The only example included in this section of the protocol specification seems to show how the min and max values in stats would be formatted, no further details regarding how these stats should be formatted for their respective data types.
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

[ ] Yes. I can contribute a fix for this bug independently.
[ ] Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
[X] No. I cannot contribute a bug fix at this time.

Thanks for reporting this. Some of the info is in the Delta docs and in code around the configs that influence the stats collections. There is some info missing. We will be adding them soon to spec.

@vkorukanti I would be interested in covering this. If that's ok, please assign the ticket to me. Thanks!


Winpanda App: Fix Mishandling Of The AppEnvironmentExtra Option For NSSM

fix passing command line arguments to external commands executed as
sub-processes
refactor the external command launch method of the NSSM-based windows
service manager type
refactor service management (API) methods of the NSSM-based windows
service manager type
add support for the "AppEnvironment" NSSM configuration option
improve logging and error reporting

JIRA: DCOS_OSS-5769 - Winpanda App: Mishandling Of The AppEnvironmentExtra
Option For NSSM

@mesosphere-mergebot changelog-not-required Changes to unreleased functionality

@mesosphere-mergebot request-review @jongiddy

@mesosphere-mergebot request-review @DominikDary


android: Remove need for strings.xml

Currently the only reason we need strings.xml is due to the usage of @string/app_name in AndroidManifest.xml

CI Vulkan-ExtensionLayer build queued with queue ID 33063.

CI Vulkan-ExtensionLayer build queued with queue ID 33073.

Jenkins CI will fail until I update VulkanTests

CI Vulkan-ExtensionLayer build # 699 running.

CI Vulkan-ExtensionLayer build # 699 failed.

CI Vulkan-ExtensionLayer build queued with queue ID 33874.

CI Vulkan-ExtensionLayer build # 701 running.

CI Vulkan-ExtensionLayer build # 701 passed.


Broken pipe on send_packet

Using lakeside on Home Assistant and, after updating firmware for bulbs T1012, there are several broken pipe errors on bulb update. For example:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 196, in async_update_ha_state
    yield from self.async_device_update()
  File "/usr/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 319, in async_device_update
    yield from self.hass.async_add_job(self.update)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.6/site-packages/homeassistant/components/light/eufy.py", line 63, in update
    self._bulb.update()
  File "/usr/lib/python3.6/site-packages/lakeside/__init__.py", line 166, in update
    response = self.get_status()
  File "/usr/lib/python3.6/site-packages/lakeside/__init__.py", line 112, in get_status
    packet.sequence = self.get_sequence()
  File "/usr/lib/python3.6/site-packages/lakeside/__init__.py", line 108, in get_sequence
    return device.get_sequence(self)
  File "/usr/lib/python3.6/site-packages/lakeside/__init__.py", line 94, in get_sequence
    response = self.send_packet(packet, True)
  File "/usr/lib/python3.6/site-packages/lakeside/__init__.py", line 105, in send_packet
    return device.send_packet(self, packet, response)
  File "/usr/lib/python3.6/site-packages/lakeside/__init__.py", line 69, in send_packet
    self.s.send(encrypted_packet)
BrokenPipeError: [Errno 32] Broken pipe

Something was broken with firmware 3.6 on Eufy?

Experimenting with lakeside on console, the errors are the same
>>> import lakeside
>>> devices = lakeside.get_devices("xxxxx", "yyyyyy")
>>> bulb = lakeside.bulb(devices[0]["address"], devices[0]["code"], devices[0]["type"])
>>> bulb
<lakeside.bulb object at 0x103f6ae48>
>>> bulb.get_status()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 112, in get_status
    packet.sequence = self.get_sequence()
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 108, in get_sequence
    return device.get_sequence(self)
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 94, in get_sequence
    response = self.send_packet(packet, True)
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 105, in send_packet
    return device.send_packet(self, packet, response)
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 69, in send_packet
    self.s.send(encrypted_packet)
>>> bulb.set_state(power=True, brightness=50, temperature=100)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 161, in set_state
    packet.sequence = self.get_sequence()
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 108, in get_sequence
    return device.get_sequence(self)
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 94, in get_sequence
    response = self.send_packet(packet, True)
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 105, in send_packet
    return device.send_packet(self, packet, response)
  File "/Users/dclobato/Downloads/x/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 69, in send_packet
    self.s.send(encrypted_packet)
AttributeError: 'bulb' object has no attribute 's'

@mjg59 does Eufy have any sort of development documentation? :)
I am searching for this kind of information for some time (for trying to fix this error by myself) and can't find anything...

What I have noticed, while playing with lakeside on console, is that some commands work fine just after connecting to the bulb. If there is no activity (commands sent to the bulb) for sometime, then we start getting errors. Issuing another bulb.conect() does the trick...
Maybe the new firmware introduced some form of timeout for innactive connection?
>>> bulb = lakeside.bulb(devices[7]["address"], devices[7]["code"], devices[7]["type"])
>>> bulb.connect()
>>> bulb.address
'<IP_ADDRESS>'
>>> bulb.get_status()
sequence: 19090895
code: ""
bulbinfo {
  type: 2
  packet {
    unknown1: -1
    bulbstate {
      command: 6
      values {
        brightness: 58
        temperature: 0
      }
      power: 1
      unknown1: 0
      values2 {
        brightness: 58
        temperature: 58
      }
      unknown2: -48
    }
  }
}

>>> bulb.set_state(power=True, brightness=50, temperature=100)
>>> bulb.set_state(power=True, brightness=58, temperature=0)
>>> bulb.set_state(power=True, brightness=58, temperature=100)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 161, in set_state
    packet.sequence = self.get_sequence()
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 108, in get_sequence
    return device.get_sequence(self)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 94, in get_sequence
    response = self.send_packet(packet, True)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 105, in send_packet
    return device.send_packet(self, packet, response)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 76, in send_packet
    length = struct.unpack("<H", decrypted_packet[0:2])[0]
struct.error: unpack requires a buffer of 2 bytes
>>> bulb.connect()
>>> bulb.set_state(power=True, brightness=50, temperature=0)
>>> bulb.set_state(power=True, brightness=100, temperature=0)
>>> bulb.set_state(power=True, brightness=50, temperature=0)
>>> bulb.set_state(power=True, brightness=50, temperature=10)
>>> bulb.set_state(power=True, brightness=50, temperature=60)
>>> bulb.set_state(power=True, brightness=50, temperature=100)
>>> bulb.set_state(power=True, brightness=50, temperature=0)
>>> bulb.set_state(power=True, brightness=50, temperature=100)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 161, in set_state
    packet.sequence = self.get_sequence()
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 108, in get_sequence
    return device.get_sequence(self)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 94, in get_sequence
    response = self.send_packet(packet, True)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 105, in send_packet
    return device.send_packet(self, packet, response)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 76, in send_packet
    length = struct.unpack("<H", decrypted_packet[0:2])[0]
struct.error: unpack requires a buffer of 2 bytes
>>> bulb.connect()
>>> bulb.set_state(power=True, brightness=50, temperature=100)
>>> bulb.set_state(power=True, brightness=50, temperature=0)
>>> bulb.set_state(power=True, brightness=50, temperature=100)
>>> bulb.set_state(power=True, brightness=50, temperature=100)
>>> bulb.set_state(power=True, brightness=50, temperature=0)
>>> bulb.set_state(power=True, brightness=50, temperature=100)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 161, in set_state
    packet.sequence = self.get_sequence()
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 108, in get_sequence
    return device.get_sequence(self)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 94, in get_sequence
    response = self.send_packet(packet, True)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 105, in send_packet
    return device.send_packet(self, packet, response)
  File "/Users/dclobato/Downloads/lakeside/lib/python3.7/site-packages/lakeside/__init__.py", line 76, in send_packet
    length = struct.unpack("<H", decrypted_packet[0:2])[0]
struct.error: unpack requires a buffer of 2 bytes
>>>

I have updated lakeside manually on my home assistant installation... The "broken pipe" and "unpack" errors are less frequent, but they are still there, along with a new "connection reset by peer" error
Almost all errors are on self.s.send(encrypted_packet) (lakeside/init.py", line 70) or self.s.connect((self.address, 55556)) (lakeside/init.py", line 57)

Can you try with the code from the keepalive branch? I'm afraid I don't have access to any Eufy hardware at the moment, so I can't verify it.

Can you try with the code from the keepalive branch?

I am running this code for 12h now. I see no disconnect errors.  I'll keep it running for one or two more days
The only errors that I see are some "no route to host" when, for some reason, the bulbs are disconnected from wifi and HA try to set/get bulb status; but I think this is, some how, an expected behaviour, because the device is disconnected and there is no way to tell HA to stop/resume polling it
Mon Aug 13 2018 09:11:30 GMT-0300 (Brasilia Standard Time)

Update for light.sbmurilo2 fails
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 82, in send_packet
    self.s.send(encrypted_packet)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 199, in async_update_ha_state
    yield from self.async_device_update()
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 322, in async_device_update
    yield from self.hass.async_add_job(self.update)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/light/eufy.py", line 62, in update
    self._bulb.update()
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 192, in update
    response = self.get_status()
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 138, in get_status
    packet.sequence = self.get_sequence()
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 134, in get_sequence
    return device.get_sequence(self)
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 115, in get_sequence
    response = self.send_packet(packet, True)
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 131, in send_packet
    return device.send_packet(self, packet, response)
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 84, in send_packet
    self.connect()
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 128, in connect
    return device.connect(self)
  File "/srv/homeassistant/lib/python3.6/site-packages/lakeside/__init__.py", line 65, in connect
    self.s.connect((self.address, 55556))
OSError: [Errno 113] No route to host

I believe that can be interesting to add some log/debug messages to the code to help pinpoint these errors...

Fixed in https://github.com/google/python-lakeside/commit/b37d46c8686b22517ca28c7091c3af5c6588021d - I think Lakeside is now working as intended, although HA could potentially handle some failures more usefully.


refactor: make output exactly the same as cutechess

improve cutechess compatibility

There are two more differences remaining:
just naming:
< Finished game 1 (sf_1 vs sf_2): 1/2-1/2 {Draw by insufficient mating material}
---
> Finished game 1 (sf_1 vs sf_2): 1/2-1/2 {Draw by insufficient material}

The other one is:
< Finished game 2 (sf_2 vs sf_1): 1/2-1/2 {Draw by 3-fold repetition}
---
> Finished game 2 (sf_2 vs sf_1): 1/2-1/2 {Draw by adjudication}

Here it looks like in cutechess the actual game status is checked first, while the adjudication comes afterwards. (The game does end in a real 3-fold).


Can't specify --max-old-space-size argument to mocha

I have a test suite that's running out of memory and I want to increase the heap size, but it looks like this plugin doesn't allow any way to do that. Node takes a --max-old-space-size arg to set the max heap size in MB, and Mocha accepts the arg as well and passes it through as a Node arg, so generally the right approach is to invoke Mocha with something like --max-old-space-size=4096. Note that the = is necessary here; Node complains otherwise. Unfortunately, I don't think there's any way to specify --max-old-space-size=4096 in the options object when invoking this plugin.
Here's one attempt:
{
  maxOldSpaceSize: 4096
}

Unfortunately, this results in --max-old-space-size 4096, with a space, so it doesn't work.
Here's another (hackier) attempt:
{
  'maxOldSpaceSize=4096': {}
}

This results in --max-old-space-size-4096, which also doesn't work.

Fixed in v3.3.1. Thanks!

🎉 Thank you!


define the interface required for JuiceFS here.

This PR clarify the API of object storage used by JuiceFS here, rather than use the wider interface used by Juicesync.
So we can add implementation here much easier.
Close #21

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Support upgrading wins via wins up

Adds support for running .\wins up[grade], which runs a Powershell script within wins that restarts the service with a different wins server binary.
This is an alternative to the current mechanism for upgrades that wins provides wherein it watches for a path and utilizes checksums to see whether the service should be modified / restarted and can also be used as an alternative method to enable wins on a Windows host (e.g. you can just run .\wins up instead of .\wins srv app run --register; Start-Service -Name rancher-wins).
The primary use case of this feature is to support a new Helm chart that will allow users to maintain wins configs / wins versions across entire clusters using just a Helm chart for configuration. This Helm chart uses wins (cli) to pass wins (upgrade) to wins (server) in order to update wins (server) on the host on demand. More succinctly it runs:
# Transfer the generated config to the directory
Transfer-File -Src "c:\scripts\config" -Dst "c:\host\etc\rancher\wins\config"
# Force a restart of wins server, which will naturally use the newly generated config
wins.exe cli prc run --path=$pathToWinsBinary --args="up --wins-args=`'--register --config=$pathToWinsBinaryDir\config`'"

Tested within v2 monitoring testcases in Monitoring+v2+Windows+Support document. Closing.


feature request semver tags for docker images

the app uses a internal semver "1.0.5"
can you apply the same process to github and docker images.

hey @xstar97 , most definitely. this is actually the plan going forward as api changes in the upcoming version releases would requite multiple versions to allow backwards compatibility with older versions of jellystat.


HL7 IG covering FHIR ADT

We should understand and document how PIXm solution compares and contrasts to DaVinci Alerts IG.
http://hl7.org/fhir/us/davinci-alerts/STU1/index.html

architecture: one main difference is that the DaVinci Alerts IG is defining FHIR messages where as PIXm is proposing a FHIR RESTFul interaction
usecase: ADT vs Patient Identifier Cross-referencing

@JohnMoehrke can we close this or does this need further discussion


String and Xstring literal with multiple lines returns incompatible result

Hi, I am experimenting to apply parser-prism to Opal.
During the experiment, I noticed some strings are parsed differently than parser gem.
Given this script:
"foo
bar"

%x{
  date
}

Parse result by whitequark/parser:
(begin
  (dstr
    (str "foo\n")
    (str "bar"))
  (xstr
    (str "\n")
    (str "  date\n")))

Parse result by parser-prism
(begin
  (str "foo\nbar")
  (xstr
    (str "\n  date\n")))

Ahh interesting - I didn't know the parser gem split that up. Interestingly that hasn't come up in any of my fixtures so far. I'll have to add it. Thanks for the report!

@takaram I've moved this issue over to prism itself (https://github.com/ruby/prism/issues/2480), so I'm going to close this one.


decreased urine output

Decreased urine output is currently an exact synonym for oliguria in MP but a parent to oliguria and anuria in HPO. I think the MP should change to mimic HPO since oliguia is decreased but still some urine output and anuria is absence of urination. Both would fall under decreased urine output.

Added new term:
decreased urine output MP:0031358
DEF: a reduction in the volume of urine produced and excreted or the absence of urine production and excretion
Child terms include: oliguria MP:0003620 and anuria MP:0003624


genus level average bar plot

Hi @sbslee ,
Thanks for your codes, much easier to make bar plots,
Please can you also suggest, how to make an average of all samples from each group and make a bar plot at the genus level from qzv file?
Thanks

Thanks for the idea, I will work on this.

Yes, please that will be great.  Thanks

@khemlalnirmalkar,
As you can see below, I added the group option to the taxa_abundance_bar_plot method, which will create a bar for each group instead of each sample. This is implemented in dokdo-1.9.0. Please install Dokdo again in order to use this feature. If you have any questions regarding the update, please let me know. I also updated Dokdo API to reflect this change.

dokdo.taxa_abundance_bar_plot(qzv_file,
                              level=6,
                              count=8,
                              group='body-site',
                              figsize=(10, 7),
                              legend_short=True,
                              artist_kwargs=dict(show_legend=True))
plt.tight_layout()

Thank you so much, for the quick action,
I could replicate the plot.

@khemlalnirmalkar,
First off, you can easily change the font size of group labels right now:
qzv_file = '/Users/sbslee/Desktop/dokdo/data/moving-pictures-tutorial/taxa-bar-plots.qzv'
dokdo.taxa_abundance_bar_plot(qzv_file,
                              level=6,
                              count=8,
                              group='body-site',
                              figsize=(10, 7),
                              legend_short=True,
                              artist_kwargs=dict(show_legend=True, xticklabels_fontsize=20))
plt.tight_layout()

For more details, please see the Title, Axis, Legend, Font Size section in Dokdo API.
As for the order of group names, I agree it'd be nice to be able to manually provide an order (or subset the groups). That's why I created a development branch 1.10.0-dev which implements the group_order option to the taxa_abundance_bar_plot method:
qzv_file = '/Users/sbslee/Desktop/dokdo/data/moving-pictures-tutorial/taxa-bar-plots.qzv'
dokdo.taxa_abundance_bar_plot(qzv_file,
                              level=6,
                              count=8,
                              group='body-site',
                              group_order=['right palm', 'left palm', 'gut'],
                              figsize=(10, 7),
                              legend_short=True,
                              artist_kwargs=dict(show_legend=True, xticklabels_fontsize=20))
plt.tight_layout()

In order to use this new feature, you have to install the development version of Dokdo:
$ git clone https://github.com/sbslee/dokdo
$ cd dokdo
$ git checkout 1.10.0-dev
$ pip uninstall dokdo
$ pip install .

Let me know if you have any further questions/requests.

awesome, thank you so much!! i will use development version.


Suggestion: Add a License

Hi there! A lot of people use this project and I wanted to see if you were open to adding a software license on this project? It would set clear boundaries for this project to be used or extended. A lot of projects on GitHub choose a license to use and here is a doc on GitHub that I've found helpful in choosing a license: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository
Thank you for considering!

Thanks! Added CC0.


restarting on s7 crashe on bus when you try jumping out

Same thing for Season 8

Bro restarting doesn't even work for me 😂


Don't require vm tags

What this PR does / why we need it:
Removes our absolute reliance on VM tags in the upgrade and scale commands.
In the common case of a single agent pool, the upgrade and scale commands can identify VMs to be updated even if the user has deleted the tags we attached. (After upgrade or scale, these tags are restored as well.) This is preferable to the current behavior where VMs that lack tags are skipped over in upgrade, which is confusing to users.
This preserves the current tag-based strategy, only falling back to other methods such as contacting the Kubernetes API or testing the VM name if that approach fails.
Special notes for your reviewer:
If applicable:

[ ] documentation
[x] unit tests
[x] tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

Don't require VM tags when upgrading or scaling most clusters

/lgtm


glb loading issue vto

Hi, What is wrong in here?? why isn't it working?
const threeGlasses = new THREE.Object3D();
const loader = new THREE.GLTFLoader();
   loader.load( 'models3D/glasses.glb', function (glassesFramesGeometry) {
		glassesFramesGeometry.computeVertexNormals();
		const glassesFramesMesh = new THREE.Mesh(glassesFramesGeometry);
		threeGlasses.add(glassesFramesMesh.scene)
   });

  const occluderMesh = THREE.JeelizHelper.create_threejsOccluder(spec.occluderURL);

  return {
    glasses: threeGlasses,
    occluder: occluderMesh
  };

Please read the guidelines before submitting an issue. This is not a THREE.js help forum.

Hi..! I have used stl file format to load different frames. But I wanted to load textures too. Hence i chose glb format. But computevertexnormal function is not working with gltf loader. Could you guys please guide how should i move ahead.


fix typo at docs

Description

Fix typo texarea => textarea
Related Issues

Docs

Form fields
- input

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


global is not defined

During execution of angular application I get the following error
webpack-internal:///../../../../node-libs-browser/node_modules/buffer/index.js:43 Uncaught ReferenceError: global is not defined
at eval (webpack-internal:///../../../../node-libs-browser/node_modules/buffer/index.js:43)
at Object.../../../../node-libs-browser/node_modules/buffer/index.js (vendor.bundle.js:607)
at webpack_require (inline.bundle.js:55)
at eval (webpack-internal:///../../../../safe-buffer/index.js:2)
at Object.../../../../safe-buffer/index.js (vendor.bundle.js:1423)
at webpack_require (inline.bundle.js:55)
at eval (webpack-internal:///../../../../bitcoinjs-lib/src/script.js:1)
at Object.../../../../bitcoinjs-lib/src/script.js (vendor.bundle.js:228)
at webpack_require (inline.bundle.js:55)
at eval (webpack-internal:///../../../../bitcoinjs-lib/src/index.js:1)
buffer and safe-buffer packages are added as dependencies for bitcoinjs-lib.
This only occurs when using angular-cli 1.5.0-rc.6 (it works fine when using angular-cli 1.4.9)

Actually this is caused by angular cli 1.5.0-rc.3
https://github.com/angular/angular-cli/releases/tag/v1.5.0-rc.3
in which node global is tuned off. It works fine if I manually turn it on again.


Online Players

After a normal player goes online, after he logs out the character it still remains as online in the table.

I can confirm that bots are left marked "logged in" in the database even though you the player have logged out.  I also am having issues where I want to invite a bot to my part but it says they are already in one.  Doing a ``.group disband " doesn't solve the issue.  Logging out and back in as that bot character show they are not in a party.

I saw a chat on Discord for Cmangos where a playerbots user reported this same issue of being unable to invite bots due to "already in a group" -- so I think that means it must be a problem with the playerbots module.

I was able to fix this by doing a GM .die command on the character.  Then I made sure my own party was disbanded.  Then when he resurrected I invited him and it worked.  I have no idea how or why this worked but this is what the guy in the Mangos discord said to do and it worked for me.


Document threading support a bit more

This adds a few known limitations around threading to the README and suggests the users to look into GitHub issues to learn more.
Addresses https://github.com/rust-lang/miri/issues/1388#issuecomment-939317828

Thanks. :)
@bors r+


Chirpstack network server configuration examples for US915 are incorrect (missing channels 64,65)

As commented in the title:
The example in chirpstack-network-server.us915.0.toml contains the following:
  [network_server.network_settings]   enabled_uplink_channels=[0, 1, 2, 3, 4, 5, 6, 7]
For sub-band 1, this should be instead:
  [network_server.network_settings]   enabled_uplink_channels=[0, 1, 2, 3, 4, 5, 6, 7, 64]
The file chirpstack-network-server.us915.1.toml has a similar misconfiguration - it is missing channel 65 for a proper sub-band 2 configuration.

@avillacis would you like to create a PR for this?


Init structure for imports

Which issue(s) are closed by this pull request?
Closes #74
Changes proposed in this pull request:
Init structure for consistent imports of respective subpackages and submodules:

Data/container classes are top level imports (i.e. pyfar.Signal, pyfar.Coordinates)
Sub-packages are imported into the main package and handled equivalent to sub-modules.
Sub-packages import functions from their respective sub-modules for convenience and ease of use.
Sub-modules in sub-packages with rare/no direct user interaction are either imported as sub-sub-modules or kept private.

As a result, imports such as from pyfar import Signal in modules/sub-modules need to be avoided as they might lead to cyclic import issues.

Fabian, whats up there? your suggestions are often removing and adding the same line ;D

Fabian, whats up there? your suggestions are often removing and adding the same line ;D

Fabian, whats up there? your suggestions are often removing and adding the same line ;D

It was the quickest way to suggest adding a new (second) line - we have to import pyfar.signals, which was introduced after you did the work in this Pull :)


error when trying to market execute

i can make pending orders fine,
but when selecting marked execution i get this error

It seems like this has nothing to do with plugin or expert advisor. I did a google search with the error code returned by the MT terminal and got this trouble shooting guide: https://medium.com/@4xpip/mt5-err-trade-send-failed-error-4756-431f43a39077
Please let me know if this works and I would like know your MT terminal version. Is it MT4 or MT5?

Hi, im using MT5,
But i get the error with the same settings (lot, SL,TP) as with the limit/stop orders, and they work fine, so i dont really know what to change 😅


chore(api): update api dependencies

Update API dependencies.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@7371433). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main   #10810   +/-   ##
=======================================
  Coverage        ?   47.37%
=======================================
  Files           ?      669
  Lines           ?    33066
  Branches        ?     6673
=======================================
  Hits            ?    15665
  Misses          ?    15723
  Partials        ?     1678

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

Closing this in favor of https://github.com/aws-amplify/amplify-cli/pull/10821

Merged in https://github.com/aws-amplify/amplify-cli/commit/446d02bb15f271843bd2a915b6cd1b16e6f93604


[WIP] Add profile guided optimization to wheels

This commit adds scripts to build profile data for profile guided
optimization to the retworkx repo and then updates the release jobs to
use the script and the subsequently generated profile data into the
release binaries in the built wheels. The scripts are based on the
official rust docs for using PGO. [1]
The profile data is built using qiskit-terra's unit tests, a dedicated
100 x 20 quantum volume transpile run, transpiling a randomized
benchmarking experiment, and running qiskit-aqua's abelian grouper.
[1] https://doc.rust-lang.org/rustc/profile-guided-optimization.html

Before we go to merge this we should need to test that the cibuildwheel jobs work with PGO enabled. The other thing missing here is benchmarking to see if this is worth the effort or not. In local testing I haven't really seen any performance impact when running a QV transpile (the only thing I've tested so far). We'll need to do some real benchmarking to determine what the tradeoffs are with PGO and whether it's worth it.

Pull Request Test Coverage Report for Build 366793627

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 94.535%

Totals

Change from base Build 364555944:
0.0%

Covered Lines:
2889

Relevant Lines:
3056

💛  - Coveralls

In local testing, I'm not really seeing any improvement for binaries built with PGO. I'm not sure if this is going to be worth the extra effort. I'll keep playing with this eventually to see if it's just an instance of not having enough (or good enough) profile data or something else that can be tuned.

I've been doing some testing and when building with PGO enabled it's about 2-5% faster than without PGO enabled. For release builds I think the one time cost in extra build time and complexity are worth it, even if the gains are modest. I'm gonna leave this as WIP because I need to push a DNM commit to the branch to run cibuildwheel and verify that the PGO builds work in the release jobs.

I've been doing some testing and when building with PGO enabled it's about 2-5% faster than without PGO enabled. For release builds I think the one time cost in extra build time and complexity are worth it, even if the gains are modest. I'm gonna leave this as WIP because I need to push a DNM commit to the branch to run cibuildwheel and verify that the PGO builds work in the release jobs.

I'm going to close this, I've only been able to get it to build on macOS (using the clang tools), the complexity of the build and ensuring all the llvm tools are present and working is more effort than this is worth right now. I might reopen it in the future, but for a small 2-5% performance increase it doesn't seem worth the effort if it's only on a handful of release binaries.

I'm going to close this, I've only been able to get it to build on macOS (using the clang tools), the complexity of the build and ensuring all the llvm tools are present and working is more effort than this is worth right now. I might reopen it in the future, but for a small 2-5% performance increase it doesn't seem worth the effort if it's only on a handful of release binaries.


Table row height: 2

If you set the table row height to 2 -> nothing will happen?

I've just checked this out and it seems like 16px is the minimum height for the table. I'll add this information to the tooltip, should a lower number still be enterable or should I change it to 16px whenever someone tries to enter a lower value?

simply set it to 16 if it's lower...


Disk:  Setting fslabel on multiple partitions labels all partitions the same

When I use the Disk command to create multiple partitions on the same disk, all having their own fslabel setting then it seems to change the label to be the same on all partitions on each iteration.
The code used to set the disk partitions is as follows :
	WaitForDisk Disk2
    {
         DiskId = 2
         RetryIntervalSec = 60
         RetryCount = 60
    }
    Disk HVolume
    {
         DiskId = 2
         DriveLetter = "H"
		 FSLabel = "SQL"
		 Size = 9.77GB
		 FSFormat = "NTFS"
		 PartitionStyle = 'GPT'
         DependsOn = '[WaitForDisk]Disk2'
    }
	Disk RVolume
    {
         DiskId = 2
         DriveLetter = 'R'
		 FSLabel = "BACKUP"
		 Size = 117.19GB
		 FSFormat = 'NTFS'
		 PartitionStyle = 'GPT'
         DependsOn = '[Disk]HVolume'
    }
	Disk TVolume
    {
         DiskId = 2
         DriveLetter = "T"
		 FSLabel = "TEMPDB"
		 Size = 97.66GB
		 FSFormat = "NTFS"
		 PartitionStyle = 'GPT'
         DependsOn = '[WaitForDisk]Disk2'
    }
	Disk LVolume
    {
         DiskId = 2
         DriveLetter = "L"
		 FSLabel = "LOGS"
         Size = 73.24GB
		 FSFormat = "NTFS"
		 PartitionStyle = 'GPT'
         DependsOn = '[WaitForDisk]Disk2'
    }
	Disk FVolume
    {
         DiskId = 2
         DriveLetter = "F"
		 FSLabel = "DATA"
		 Size = 204.25GB
		 FSFormat = "NTFS"
		 PartitionStyle = 'GPT'
         DependsOn = '[WaitForDisk]Disk2'
    }
	Disk SVolume
	{
		 DiskId = 2
         DriveLetter = "S"
		 #Size = 204.25GB
		 FSLabel = "Software"
		 FSFormat = "NTFS"
		 PartitionStyle = 'GPT'
         DependsOn = '[WaitForDisk]Disk2'
	}

The target operating system is Windows 2016 running in Azure, using a standard 2016 installation from Microsoft.
The version is StorageDsc <IP_ADDRESS>

Strange, this kind of worked on some new machines...the labels were correct but I cannot get it to work on old machines.  The only difference is I upgraded from <IP_ADDRESS> to <IP_ADDRESS> and the new machines are wiped clean with the latest modules, the old machines are just upgraded and no matter what they won't get correct labels.
However, there is a new odd behaviour...I've created 2xvolumes of 550GB and 2xvolumes of 75GB each called DATA & DATA2, and LOGS & LOGS2.  Now I see that it creates the first of the 2x550GB volumes correctly, 550GB/F letter/DATA volume label but when that's finished it renames this volume to DATA2 and never creates the 2nd 550GB volume.
This is similar to what happens with the volume labels, when it gets to the last volume it renames all the volumes the same label as the last volume is created.  It seemingly doesn't seperate the individual volumes properly, but rather goes back to edit volumes created.

Hi @ndoody - thanks for raising this. Are you possibly able to post the verbose logs from running the DSC config here (minus any identifiable info)?

Where do I get these logs from?  This is deployed through the DSC agent installed from within the azure cloud.  It has the usual operational logs for DSC, and you can trick it into running a new "update-dscconfiguration" by recompiling the config in the cloud and I can take info from "Get-DSCConfigurationStatus" this way.

This was easily repeatable on a new disk and configuration.  This is the end product 👎

This is the script part run
WaitForDisk Disk3
{
DiskId = 3
RetryIntervalSec = 60
RetryCount = 60
}
Disk GVolume
{
DiskId = 3
DriveLetter = 'G'
Size = 64GB
FSFormat = 'NTFS'
FSLabel = "G"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}
Disk HVolume
{
DiskId = 3
DriveLetter = 'H'
Size = 64GB
FSFormat = 'NTFS'
FSLabel = "H"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}
Disk IVolume
{
DiskId = 3
DriveLetter = 'I'
Size = 32GB
FSFormat = 'NTFS'
FSLabel = "I"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}
Disk JVolume
{
DiskId = 3
DriveLetter = 'J'
Size = 32GB
FSFormat = 'NTFS'
FSLabel = "J"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}
This is the "resourcesindesiredstate" output 👎
ConfigurationName     : prodServers
DependsOn             :
ModuleName            : StorageDsc
ModuleVersion         : <IP_ADDRESS>
PsDscRunAsCredential  :
ResourceId            : [WaitForDisk]Disk3
SourceInfo            : ::1136::3::WaitForDisk
DurationInSeconds     : 0.047
Error                 :
FinalState            :
InDesiredState        : True
InitialState          :
InstanceName          : Disk3
RebootRequested       : False
ResourceName          : WaitForDisk
StartDate             : 2/21/2019 7:38:21 PM
PSComputerName        :
CimClass              : root/Microsoft/Windows/DesiredStateConfiguration:MSFT_ResourceInDesiredState
CimInstanceProperties : {ConfigurationName, DependsOn, ModuleName, ModuleVersion...}

CimSystemProperties   : Microsoft.Management.Infrastructure.CimSystemProperties
ConfigurationName     : prodServers
DependsOn             : {[WaitForDisk]Disk3}
ModuleName            : StorageDsc
ModuleVersion         : <IP_ADDRESS>
PsDscRunAsCredential  :
ResourceId            : [Disk]HVolume
SourceInfo            : ::1152::3::Disk
DurationInSeconds     : 0.687
Error                 :
FinalState            :
InDesiredState        : True
InitialState          :
InstanceName          : HVolume
RebootRequested       : False
ResourceName          : Disk
StartDate             : 2/21/2019 7:38:30 PM
PSComputerName        :
CimClass              : root/Microsoft/Windows/DesiredStateConfiguration:MSFT_ResourceInDesiredState
CimInstanceProperties : {ConfigurationName, DependsOn, ModuleName, ModuleVersion...}

CimSystemProperties   : Microsoft.Management.Infrastructure.CimSystemProperties
ConfigurationName     : prodServers
DependsOn             : {[WaitForDisk]Disk3}
ModuleName            : StorageDsc
ModuleVersion         : <IP_ADDRESS>
PsDscRunAsCredential  :
ResourceId            : [Disk]JVolume
SourceInfo            : ::1172::3::Disk
DurationInSeconds     : 0.391
Error                 :
FinalState            :
InDesiredState        : True
InitialState          :
InstanceName          : JVolume
RebootRequested       : False
ResourceName          : Disk
StartDate             : 2/21/2019 7:38:36 PM
PSComputerName        :
CimClass              : root/Microsoft/Windows/DesiredStateConfiguration:MSFT_ResourceInDesiredState
CimInstanceProperties : {ConfigurationName, DependsOn, ModuleName, ModuleVersion...}
CimSystemProperties   : Microsoft.Management.Infrastructure.CimSystemProperties

Thanks @ndoody - I'm having a go at replicating this now. Hopefully I have an idea where this might occur.

Hi @ndoody - facepalm from me - I've realized what is happening here. The issue is that each the Disk resource uses the Size to determine if it should create a new volume on an existing Disk or update an existing volume. In your case, because each volume is 32GB the resource is updating the same Volume with a new label and Drive letter.
This is a bit of a limitation of the Disk resource (a whole reason I've planned to rework the resources with https://github.com/PowerShell/StorageDsc/issues/91). So what you should do is change the set the size of each volume to a different number (even just by just 1 byte). E.g.
WaitForDisk Disk3
{
DiskId = 3
RetryIntervalSec = 60
RetryCount = 60
}
Disk GVolume
{
DiskId = 3
DriveLetter = 'G'
Size = 64.0000GB
FSFormat = 'NTFS'
FSLabel = "G"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}
Disk HVolume
{
DiskId = 3
DriveLetter = 'H'
Size = 64.0001GB
FSFormat = 'NTFS'
FSLabel = "H"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}
Disk IVolume
{
DiskId = 3
DriveLetter = 'I'
Size = 32.0000GB
FSFormat = 'NTFS'
FSLabel = "I"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}
Disk JVolume
{
DiskId = 3
DriveLetter = 'J'
Size = 32.0001GB
FSFormat = 'NTFS'
FSLabel = "J"
PartitionStyle = 'GPT'
DependsOn = '[WaitForDisk]Disk3'
}

Creating new more granular DiskEx, Partition and Volume resources is still my longer term plan as this would allow more precise control over the disk layout.
Does that make sense?

Thinking about this further, I think we need some better examples and documentation of this behavior until we have a good way of resolving it.

Hi,
I'm encountering a similar if not the same issue.
I am trying to format an Azure P30 managed disk into 10 partitions of 50GB each.
What I am finding is the resource detects there is a partition with the same size already and just modifying the first partition to have the drive letter and label of the next partitions.
I am currently changing the size by 0.0001GB by each parition as a dirty hack. Which appears to work.
I say appear because when I inspect the capacity of the partitions after formatting each one displays the same capacity in bytes which can only be true if the number is being rounded down. I don't know what implications this has for subsequent enforcement of state.

DiskAccessPath exhibits the same behavior when creating mount points of the same size on the same disk.

This is an unfortunate side effect of how this resource was designed.
My intention was to create a new set of resources that would allow more granular control of partitions/volumes: https://github.com/dsccommunity/StorageDsc/issues/91 . However, it is a reasonably large job that I haven't found time to do.
However, thinking about this a bit, I'm wondering if it would be possible to for us to add an optional PartitionNumber parameter that will allow you to implement more control over the process and not rely on the size of a partition to determine if it already exists. More investigation would be required to validate this, but in theory it could work without requiring the new resources (which I don't have time to create at the moment).
There is some risk however with this approach: It will enable configurations that are technically invalid that can never enter state
(or worse exhibit configuration flapping) if the the disk is already partitioned and the drive letters are already assigned to the wrong partitions. This is because technically the combination of PartitionNumber and DiskId should be unique across all resources, but if PartitionNumber is optional then we can't make it a Key property. But this might be worth the risk as long as the behavior is documented and appropriate examples are created.
Avoiding a breaking change here is important as well. Once I've completed this PR (https://github.com/dsccommunity/StorageDsc/pull/242) I can take a look further.

Could this be related?
I have this very simple PowerShell DSC Disk config. Just trying to set a label.
Disk CVolume
{
DiskId = ‘0’
DriveLetter = ‘C’
FSLabel = ‘System’
}
However when the config is executed, DSC is trying to resize the disk for some reason. I am not telling it to touch the disk size. See below the WARNING line in bold.
VERBOSE: [server]: LCM: [ Start Resource ] [[Disk]CVolume]
VERBOSE: [server]: LCM: [ Start Test ] [[Disk]CVolume]
VERBOSE: [server]: [[Disk]CVolume] Test-TargetResource: Testing disk with Number ‘0’ status for drive letter ‘C’.
VERBOSE: [server]: [[Disk]CVolume] Test-TargetResource: Checking if disk with Number ‘0’ is initialized.
VERBOSE: [server]: [[Disk]CVolume] Perform operation ‘Query CimInstances’ with following parameters, ”queryExpression’ = SELECT BlockSize from Win32_Volume WHERE DriveLetter = ‘C:’,’queryDialect’ = WQL,’namespaceName’ = root\cimv2′.
VERBOSE: [server]: [[Disk]CVolume] Operation ‘Query CimInstances’ complete.
VERBOSE: [server]: [[Disk]CVolume] Test-TargetResource: Volume assigned to drive C label ” does not match expected label ‘System’.
VERBOSE: [server]: LCM: [ End Test ] [[Disk]CVolume] in 4.3440 seconds.
VERBOSE: [server]: LCM: [ Start Set ] [[Disk]CVolume]
VERBOSE: [server]: [[Disk]CVolume] Set-TargetResource: Setting disk with Number ‘0’ status for drive letter ‘C’.
VERBOSE: [server]: [[Disk]CVolume] Set-TargetResource: Checking disk with Number ‘0’ partition style.
VERBOSE: [server]: [[Disk]CVolume] Set-TargetResource: Disk with Number ‘0’ is already initialized with ‘GPT’.
VERBOSE: [server]: [[Disk]CVolume] Set-TargetResource: Partition ‘4’ is already assigned as drive C.
WARNING: [server]: [[Disk]CVolume] Set-TargetResource: Skipping resize of C from<PHONE_NUMBER>96 to<PHONE_NUMBER>76. AllowDestructive is not set to $true.
VERBOSE: [server]: LCM: [ End Set ] [[Disk]CVolume] in 3.0470 seconds.
VERBOSE: [server]: LCM: [ End Resource ] [[Disk]CVolume]

Hi @jojiklmts - I'm not certain, but this might be because of trying to set the size of C: Drive. I'd assume that this is your %SYSTEMROOT% drive so it would not be a good idea to configure it using this resource. This is most likely because your boot disk usually has one or more additional partitions on it (why C: drive is partition 4).
What is the reason for applying a config to C: drive?

I wish to use DSC as one stop shop for whole config management. It would be better if DSC was not trying to resize the partition if it was not explicitly told to do so. Just trying to contribute if I come across ignorant.

Hi @jojiklmts - you are absolutely right to use DSC as a one stop shop for whole config management, and it is appropriate to do so.
I think though the issue you're seeing is actually the same cause as this issue. This was is caused by a bug in Get-PartitionSupportedSize (see this user voice). The Disk resource uses Get-PartitionSupportedSize to determine if the partition you're configuring is at the maximum supported size (e.g. it uses up all remaining storage on the disk). However, it misreports this value in some situations - which is leading to the DSC resource thinking that the partition needs to be resized.
Get-PartitionSupportedSize seems to report incorrect values with used on disks with more complex partition configuration - for example, when used on a partition on the boot drive. This is because Windows adds a number of supporting partitions to the disk structure:

As you can see (this is from my machine):

So, my recommendation would be to avoid using DSC Disk resource against C: Drive and/or your boot disk. Besides, if DSC did determine that something needed to be changed (for example the partition needed to be resized) then it would be a destructive action and performing that on C: drive would result in your OS being wiped.

Thanks for your quick response. I implemented workaround. Would it anyway make sense to rewrite DSC Disk resource, not to be using Get-PartitionSupportedSize, unless the programmer wants to do something specifically with disk size?

The challenge there is that the resource is designed so that if you don't specify a size for the partition it assumes you want to make the partition/volume consume all remaining space on the disk.
We could potentially implement a flag to force ignoring the size of the partition - if the partition already exists - that would resolve the problem I think? Could you raise that as a separate issue so I can track it outside of this issue (don't want it to get lost).


-fix-mode option is missing for commands which use -fix-clashes

# drive pull --help
...
  -files
    	pull only files
  -fix-clashes
    	fix clashes by renaming or trashing files
  -force
    	forces a pull even if no changes present
...
# drive push --help
...
  -files
    	push only files
  -fix-clashes
    	fix clashes by renaming or trashing files
  -force
    	forces a push even if no changes present
...

Hello @jean-christophe-manciot, thanks for the report.
Good catch. Yap yap, we need to fix this. This is a trivial change that can be added by refactoring
the fix mode selection to a method that can then be invoked everywhere once.

@jean-christophe-manciot please get the latest from master, fixed by https://github.com/odeke-em/drive/pull/851.
You can now do:
$ drive push -fix-clashes -fix-mode trash a1 notes
$ drive pull -fix-clashes -fix-mode rename 2016 patents

Thank you.


how to use trongrid-js in reactnative

After installing the package on the React Native project, it gives an error during the project execution

Could you please provide text of error? Are you running on Expo (bare or managed)?
If you was able to solve this issue, could you please provide detailed explanation on how to fix that? :)


Face detection

It would be awesome to add face detection to the crop function. This might be a bit of a pipe dream. There is this PHP package. Also, could be good to look at this JavaScript project.
Function would work like this:
kayaks.jpg?w=500&h=500&crop=face

Yes, something like www.imgix.com and www.cloudinary.com support would be fantastic.

And interestingly I don't think it would take TOO much to implement, but I'm not sure it makes sense to build face detection into Glide itself. This is better served as a separate package. I sort of keep hoping that a package will pop up...there is already a number of them in Javascript. (ie. http://facedetection.jaysalvat.com/).
If I were to make this package I would build it in such as way that it simply returned the coordinates of the faces, and didn't actually do the cropping itself. This would allow it to be used with any sort of image manipulation library...like Glide.

This was also discussed in Intervention/image: https://github.com/Intervention/image/issues/75
I also looked at it but couldn't produce good results + it was very slow.

Yeah, someone who knows a thing or two about face detection needs to take the bull by the horns and make a good PHP library for this. I bet it would get used a LOT.

We maybe need to convince @jaysalvat with beer to create a PHP version of his JavaScript library. :)

The problem is performance, or more specifically that certain tasks need to be written in C, delegated to other libraries - like image magic - or parallelized. All these things are annoying, and tbh solved better by other libraries/tools like thumbor.

Very fair point @josegonzalez, but if this can be done with JavaScript, surely it can also be done in PHP, no?

Javascript gets web workers, which happen off the ui thread, can be parallelized and thus happen much faster than in PHP :(

Hello all!
There are some PHP face detection projects.
https://github.com/mauricesvay/php-facedetection is one on them.
:)

thumbor's focal-point detection algorithm for facials and features is implemented through OpenCV (http://opencv.org). They also implement what they call "Lazy Detection" which is nothing more then a image processing queue based on redis.

Closing this issue for now. I love the idea of face detection in Glide, but it's too big of a feature for me to handle at this point. Further, I think it's probably better better handled by a lower level library, as @josegonzalez suggested.

For Javascript, there is: https://github.com/jwagner/smartcrop.js
So if anybody feels the urge to pick this up, I suggest to make a port ;)

Awesome @barryvdh, I hadn't seen this one before. Thanks for sharing.


autoFixOnSave alternates between correct and wrong indentation

I have quite strange error. I set eslint.autoFixOnSave to true. When I save a file once it formats my code successfully:

When I save it again it messes up the indentation.

When I save a 3rd time it fixes the indentation again as in screenshot 1 and so on.
How can this happen?
This is my .eslintrc.json:
{
  "extends": "prettier",
  "parserOptions": {
    "sourceType": "module",
    "ecmaVersion": 2017
  },
  "env": {
    "es6": true,
    "node": true
  },
  "plugins": [
    "prettier"
  ],
  "rules": {
    "prettier/prettier": [
      "error",
      {
        "semi": false,
        "printWidth": 120
      }
    ]
  }
}

Edit: I recorded a video with the strange behaviour:

@dj-hedgehog do you have any other extension installed (especially one of the other prettiers out there)?

If not can you provide me with a GitHub repository that demos the behavior with steps on how to reproduce.

I had the same issue with "JavaScript Standard Style" and "editor.formatOnSave": true, in workspace settings. This resulted in the formatters competing to format the document ( I suppose), which might give you this behavior.
Setting "editor.formatOnSave": false, solved it for me.

@arenddeboer You were actually right but it was a little bit hidden. In my global settings I did not have editor.formatOnSave but someone from my team commited the .vscode/settings.json file which contained editor.formatOnSave = true. Since I removed it everything has been working butter smooth.


Spoofing by patching the target app requiring GCM, instead of microG itself?

Hi, microG works perfectly on my previous phone with LineageOS and haystack patch. However I recently changed to a new xiaomi mi6 without GMS preinstalled, and I do not intend to root or use a 3rd-party ROM.
 This piece of code would be very similar across all apps, and it is exactly this code asking for the signature of microG and check it against the correct one(which must be a static value inside the code). If we can decompile the and patch the target app (for instance, search/replace the static signature with microg's sig, or something more straightforward), repackage and install it, it should work with microg. The result apk will have different signature, but it can be managed.
I have over 100 app on my phone, but the ones that really need with GCM are less than 10. If we can automate this repackage process(termux have apktool so this process could be done on android), we can enjoy GCM without root/spoofing/bloatware, on any phones.
I would like to try this idea first with the search/replace trick, but I don't know google's original signature, so I can not search/replace. Could anyone give me the google signature that need to be replaced?

Hi are you using China ROM or Global ROM?

You can also apply the signature spoofing patch for the ROM from the recovery (it may work without root).

Could anyone give me the google signature that need to be replaced?

I think you will find the original signature here https://github.com/thermatk/FakeGApps/blob/master/app/src/main/java/com/thermatk/android/xf/fakegapps/FakeSignatures.java

@tryptophan0 Thanks for the tip, I will try it.
@WillsonChiang I know there are ways to get GMS even on China ROM, but I do not intend to do that, it's too battery unfriendly.
@ale5000-git Thanks for your suggestion, but I don't want even unlock my bootloader. Instead I want to take a cleanest experimental approach in terms of rom-modification, and see how it goes. Wish me good luck!

@ccaapton: You can also unlock, patch and relock (I have never tried but I suppose it works).
I don't know how to search/replace the signature but you can see the faked signature also from the sources of microG here.

@ccaapton Yes I have installed GMS on my China ROM and everything works perfectly except for GCM services, besides it doesn't seem to kill battery a lot.

@WillsonChiang: My microG unofficial installer support installation also on China ROMs and should fix all problems automatically.

@ccaapton: Good start but in this way if a virus create a fake GCM then the app will happily use it, I think it would be better to check microG signature.

@ccaapton  Hello is there a way to do the same for waze? Thank you


get_all_tickers() returns inactive markets

prices = client.get_all_tickers()
will return tickers with BREAK status.
I thought that this endpoint should give me only active markets price, as it was like year ago.
I tested on BTTBTC pair, this pair is inactive since 2019-09-30 05:45
But still returned its price from past {'symbol': 'BTTBTC', 'price': '0.00000005'}.
My Q how I can get all last prices for only active markets now?

This issue is still present, anyone manage to find a workaround?


Filter prunning for ssd

Is filter prunning supported for SSD based object detection models ?

@sushruthn96, it should be possible to do so - try adapting the available classification filter pruning configs to the SSD training configs (basically just copy and paste "compression" sections between the NNCF .json configs).
@mkaglins might correct me if I'm wrong.

@sushruthn96 @vshampor Yes, filter pruning for SSD models is supported by default with the adapted config.
But unfortunately, export to ONNX for SSD models is not supported yet.


Install script should not exit PowerShell

When running the install script again while Scoop is already installed, it prints out "scoop is already installed. run 'scoop update' to get the latest version." and then exits PowerShell.
I see https://github.com/lukesampson/scoop/blob/master/bin/install.ps1#L16 intends to prevent that, but that does not seem to work.

Thanks for spotting this. I think it's fixed now.

Yes, it is. Thanks on behalf of less confused future users. :-)


Integration not working anymore

Hello,
It looks like i have an issue with the integration, it was working great for a long time now but since few days (i don't know exactly when) it's not working anymore. I tried to reconnect but the plugin can't load properly
2023-12-27 20:09:51.169 WARNING (SyncWorker_2) [homeassistant.loader] We found a custom integration purrsong which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2023-12-27 20:09:51.185 WARNING (SyncWorker_2) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2023-12-27 20:10:53.811 WARNING (MainThread) [homeassistant.config_entries] Config entry 'PurrSong' for purrsong integration not ready yet; Retrying in background 2023-12-27 20:11:52.925 DEBUG (MainThread) [custom_components.purrsong] Finished fetching purrsong data in 2.922 seconds (success: False) 2023-12-27 20:11:52.925 WARNING (MainThread) [homeassistant.config_entries] Config entry 'PurrSong' for purrsong integration not ready yet; Retrying in background 2023-12-27 20:11:59.745 DEBUG (MainThread) [custom_components.purrsong] Finished fetching purrsong data in 1.407 seconds (success: False) 2023-12-27 20:12:11.319 DEBUG (MainThread) [custom_components.purrsong] Finished fetching purrsong data in 1.447 seconds (success: False) 2023-12-27 20:12:34.319 DEBUG (MainThread) [custom_components.purrsong] Finished fetching purrsong data in 2.788 seconds (success: False) 2023-12-27 20:12:41.694 DEBUG (MainThread) [custom_components.purrsong] Finished fetching purrsong data in 1.375 seconds (success: False) 2023-12-27 20:12:41.694 WARNING (MainThread) [homeassistant.config_entries] Config entry 'PurrSong' for purrsong integration not ready yet; Retrying in background 2023-12-27 20:12:48.216 DEBUG (MainThread) [custom_components.purrsong] Finished fetching purrsong data in 1.383 seconds (success: False)

Probably yet another API change. I'll have to look into it - am experiencing it as well.

@Drakalinou
Thanks for bringing this to my attention. They did indeed make changes to their API. They killed off the PetMain operation which we were using to fetch the cats list. I have made changes to the backend code to use the new query, which should get you back up and running again. Please update to 0.1.11!

That was quick ! Everything is working fine now, thank you so much ! :)

Hello,
I think it's happening again :(


Multiple references should be enclosed by a single parentheses

From: https://github.com/NASA-IMPACT/nasa-apt/issues/781

When multiple references are cited at the same time, there should only be one parenthesis around all of the references. See the first example image below. The first red box is (Armston et al., 2013) ; (NI-Meister et al., 2010). It should look like this:
(Armston et al., 2013; Ni-Meister et al., 2010).

@kamicut, let's investigate this in sprint 2.

@bwbaker1, in the photo you provided in #781, it looks like there are in fact two different references because the semicolon is the same size as the paragraph text (larger than the references). Did the user add this semicolon manually?

@wrynearson

The font size looks correct now. I tried force refreshing and adding a new document yesterday afternoon, but it wasn't updated on staging yet. Not sure why it took so long, but it looks good now.

Regarding #781, the user did manually enter the semicolon. However, the parentheses are still incorrect even if there isn't a semicolon. Here is what it looks like:

To technically be correct, it should look like this:
This is a test (BradBaker, n.d.; Baker & Doe, 2021; Doe, 2019).

I just realized that now the references are showing the authors full names. It should just be the last name. It's also not changing three or more authors to et al. anymore.

Thanks for the update @bwbaker1.
We'll look into issues 2 and 3.
For issue 3 (the last name part), we're parsing authors by the using the entered format LastName1, FirstName1 and LastName2, FirstName2 and .... If the names are entered like this, the parsing works correctly, because we're searching for , and and to do the parsing. If users don't enter names like this, we can't differentiate which name is which because we just have one text field.
I just tested this on a document, and it works as expected (both in the view and PDF modes). Here's what the reference looks like:

And the result:

However, I see that we're not instructing users correctly (we say to enter first last and first last. We should update this.

@wrynearson okay I wasn't adding a comma between lastname and firstname. I don't remember that being necessary, but it's been awhile. Yeah, we just need to update the instructions with a comma.
Thanks for checking on this!

No problem! Maybe I didn't communicate it properly. I have a PR up: https://github.com/NASA-IMPACT/nasa-apt-frontend/pull/535.
Let me know if this is good.

The problem with names are that there's an infinite number of formats, standards, etc. This will definitely not fit all name formats, but hopefully will meet the majority of use cases.

@wrynearson I've found that citation-js can merge the citations of a single bibtex together. You can see the example at the bottom here: https://runkit.com/kamicut/citation-agu-example.
We would have to change how the reference manager keeps track of citations in the inserted text: the user would have to insert multiple references in one go instead of two separate references so that we know that they should be merged. We can modify the insert reference form to allow for multiple references.

Thanks @kamicut! How complex would you estimate the change to be?

@bwbaker1 are you referring to specifically the issue of multiple references within single parentheses, or the broader reference management discussion?

@wrynearson I'm referring to multiple references within single parentheses. For example, let's say someone has this:
(Doe et al. 2023; Duncan 2019; Jones 2010) and later on what's to add "Abbott 2011." I think they will get frustrated if they have to re-add all four references.

I don't think this would affect existing ATBD's but we'd ensure/test on staging before pushing any changes to production.
Do you have any insight into how often users are coming back and adding additional references into the same citations, or adding new citations?
Currently, we don't have a way to change citations that are already added. Users need to delete the citation, and then re-add a citation if they want to change the reference.
We could probably come up with a frontend solution to allow for users to add/remove multi-reference citations, but I'm worried that we're running up against our Sep 30 deadline for feature development.

I'm investigating the internal representation of references within APT. Currently, references are imported into APT from BibTex (or from the form) into a non-standard format. Some fields are mapped from Bibtex into APT fields.
Here is the structure that maps fields from Bibtex to APT fields.
const propsToMap = [
      // from -> to
      ['address', 'publication_place'],
      ['author', 'authors'],
      ['doi', 'doi'],
      ['edition', 'edition'],
      ['isbn', 'isbn'],
      ['note', 'other_reference_details'],
      // Journal and series get mapped to the same APT property. If both exist
      // "series" prevails.
      ['journal', 'series'],
      ['journaltitle', 'series'],
      ['booktitle', 'series'],
      ['series', 'series'],

      ['report_number', 'report_number'],
      ['pages', 'pages'],
      ['publisher', 'publisher'],
      ['title', 'title'],
      ['url', 'online_resource'],
      ['volume', 'volume'],
      ['year', 'year'],
      ['date', 'year']
    ];

For reference, here are the standard Bibtex fields: https://www.bibtex.com/format/fields/
Ideally, we should not have a mapping and should use citation-js to manage the fields for us. However, I'm wondering if there were specific requirements for non-standard fields and a from<->to mapping from Bibtex before we use citation-js as the internal representation.

@bwbaker1, I'd like to investigate this issue again. Do you know why our internal representation in APT is not Bibtex as mentioned in https://github.com/NASA-IMPACT/nasa-apt/issues/784#issuecomment-1695641227? If we don't need this intermediate representation, we can use Bibtex as the internal representation for references which could help solve this issue.

@kamicut I'm not sure why the internal representation is not bibtex. I'm fine with changing this if it solves the issue.

This is a priority FY24.3.

@bwbaker1 before we push this to production, I just want to reiterate that all references would lose their parentheses, and that users would have to add parentheses around all references (regardless of whether there are multiple references as described in this ticket, or not).

Could you confirm that this is the desired behavior on production?

@wrynearson Yes, this is the desired behavior. I've informed the most active science teams about this and they just prefer that things are formatted correctly.
Thanks for double checking!


Segmentation fault error

Hi,
I am using a Ubuntu 14.04 PC to compile the project.
I am getting the following error after executing "./autogen.sh":
Updating build configuration files for rnnoise, please wait....
configure.ac:8: installing 'scripts/compile'
configure.ac:13: installing 'scripts/install-sh'
configure.ac:13: installing 'scripts/missing'
Makefile.am:9: error: Libtool library used but 'LIBTOOL' is undefined
Makefile.am:9:   The usual way to define 'LIBTOOL' is to add 'LT_INIT'
Makefile.am:9:   to 'configure.ac' and run 'aclocal' and 'autoconf' again.
Makefile.am:9:   If 'LT_INIT' is in 'configure.ac', make sure
Makefile.am:9:   its definition is in aclocal's search path.
Makefile.am: installing 'scripts/depcomp'
autoreconf: automake failed with exit status: 1

How did you solve the seg fault?

@zaptrem In which stage you are getting the segmentation fault? Please elaborate the issue.

@zaptrem In which stage you are getting the segmentation fault? Please elaborate the issue.
I solve this issue by install libtool

sudo apt-get install autoconf
sudo apt-get install libtool


Feature Request: Parameterize the DB Used by Jellystat

I plan to use Jellystat in an environment where I have many different docker images that reference the same postgres server, but on my other services I have them using a separate database with a separate user. I would like my postgres server to have a 'jellystat' user which only has permissions to access the 'jellystat' database. This should be able to be accomplished by creating a variable in the docker-compose.yml file for the database and interpolating that value where necessary.

Looking at this again, it may be an issue only with the way the Unraid image for this is setup. I'll comment again and close the issue if I look into it and find out it can be resolved from that end

Following variables are already available
POSTGRES_USER
POSTGRES_PASSWORD
POSTGRES_IP
POSTGRES_PORT

Database name is always jfstat, is it that that you want configurable ?

See https://hub.docker.com/r/cyfershepard/jellystat for information about variables

Thanks, I didn't realize that the database would be named jstat or that the user was already configurable. I'll close this issue out.


Load balancer fallbacks to another cluster

When using EDS to update the endpoint configuration dynamically, DNS resolution is not allowed for hostname in EDS response, as the comment says:

//   The form of host address depends on the given cluster type. For STATIC or EDS,
//   it is expected to be a direct IP address (or something resolvable by the
//   specified :ref:resolver <envoy_api_field_core.SocketAddress.resolver_name>
//   in the Address). For LOGICAL or STRICT DNS, it is expected to be hostname,
//   and will be resolved via DNS.

and the DNS is now allowed for customer resolver

// The name of the custom resolver. This must have been registered with Envoy. If
// this is empty, a context dependent default applies. If the address is a concrete
// IP address, no resolution will occur. If address is a hostname this
// should be set for resolution other than DNS. Specifying a custom resolver with
// STRICT_DNS or LOGICAL_DNS will generate an error at runtime.

When adding some external services to the load balancing endpoints via EDS. It would be impractical to add the IP of the endpoints considering service VMs can and will go up and down, and the service's endpoint IP addresses will change frequently. Supporting hostname in EDS response seems to be a reasonable solution.
Not sure if not supporting DNS resolution for hostname is by design or the Envoy restriction. Please let me know and I can help working on this feature if Envoy needs it.
In istio, we use priority filed to implement failover logic. When the endpoints in higher priority are down, the load balancer will select the endpoints with lower priority. It assumes that all the endpoints have the same settings (e.g. TLS context). But sometimes it may be different. For example, for external fallback service, mTLS is not required,  but inside the service mesh, mTLS is required. If the external service endpoints and internal service endpoints are added into one cluster. The traffic to external endpoint will be broken.
Downstream Envoy setting:
clusters:
  - name: proxy
    type: strict_dns
    lb_policy: round_robin
    load_assignment:
      cluster_name: proxy
      endpoints:
      - lb_endpoints:
        priority: 1
        - endpoint:
            address:
              socket_address:
                address: proxy
                port_value: 80
      - lb_endpoints:
        priority: 0
        - endpoint:
            address:
              socket_address:
                address: proxy
                port_value: 443
    tls_context: {}

Upstream listener setting:
  listeners:
  - address:
      socket_address:
        address: <IP_ADDRESS>
        port_value: 80
      ...
  - address:
      socket_address:
        address: <IP_ADDRESS>
        port_value: 443
      ...
      tls_context:
        common_tls_context:
          tls_certificates:
          - certificate_chain:
              filename: /etc/cert.pem
            private_key:
              filename: /etc/key.pem
          validation_context: {}

When the proxy:443 is down, the traffic to proxy:80 will be broken as well because proxy:80 doesn't support mTLS.
Thanks @PiotrSikora for the solution. Allow load balancer to fallback to another cluster would solve the problem. For above case, split the cluster configuration into two clusters and load balancer can select another cluster when one cluster is down and use the setting for the cluster.
cc @duderino @htuch @PiotrSikora @mattklein123

I'm pretty sure that's by design - if you want to use DNS, then stick to STRICT_DNS or LOGICAL_DNS clusters.
Going back to the real problem at hand (i.e. fallback to external service if all local endpoints are down), then I believe that the real solution is fallback to another cluster, and not hacking EDS. Notably, the fallback service might have very different settings (TLS certificates, timeouts, to name a few) than the local endpoints, so we need to have the ability to configure those.

+1 to what @PiotrSikora said. If you want DNS I would deliver a DNS cluster over CDS.

I misunderstand the requirement just now. I updated the description.

This came up recently and I thought we already had an issue on this but I can't quickly find it.
I don't think there is going to be any easy way to do this, especially if you want support across both TCP and HTTP. For HTTP we could likely do this via some type of retry plugin and allow the router to target a different cluster on retry. For TCP I suppose we could do something similar also.
It might also be possible to somehow build an aggregate cluster type that could contain multiple clusters and failover between them? @snowp any thoughts here?

Are you referring this issue? https://github.com/envoyproxy/envoy/issues/7135#issue-451246689
I opened that issue for introducing opportunistic encryption. Due to security issue, @PiotrSikora suggested to close that one.
Between retry for HTTP/TCP and aggregate cluster, the latter seems to be more interesting and promising to me.

I think aggregate cluster + aggregate LB might be the easiest way to accomplish this, passing information in the LB context that can influence the decision of which cluster/lb to select. That way it would be fairly easy to reuse for multiple proxy filters (router, tcp_proxy, etc.)

Here's a Wild-and-Crazy-Idea; configure a listener on the Envoy process itself as a fallback in EDS, and have this listener TCP proxy to a DNS discovered cluster. This comes at some operational and CPU cost, but avoids having to build a massively complicated aggregated cluster type and LB, while providing the flexibility to do complicated fallback for the rare cases it is needed.

So, let's say the loopback idea ^^ is not going to fly because of the CPU cost (and needing to overprovision to support this for fallback). How many others folks in the Envoy community would benefit from a tiered LB approach? It would be good to get some additional motivation.

We would have benefitted from a tired LB approach a while back during our rollout to fall back to a not health checked VIP cluster after attempting to hit the actively health checked service mesh cluster. The health checking bit makes it difficult to express as retries within the cluster, since the fallback endpoints would also get health checked.
I'm not sure if we'd use it right now as we're mostly migrated at this point cc @mpuncel

This is something we are running into with Consul. We need different Upstream TLS context for some failover endpoints. Being able to failover instead of another cluster would work just as well allowing us to configuring the TLS context for that cluster separately.

Will tiered LB handle the use case that @mkeeler mentions? Maybe the simplest thing to do would be able to have Cluster failover rather than LB tiering.

@htuch That sounds perfect. We are trying to map the concept of failover into LB endpoints with priorities but being able to just specify an alternative cluster to use for failover would be a much closer match to the concept we are trying to implement.

The problem with "cluster failover" is if we keep the clusters separate we will need to build config/code logic in all of the users to handle failover to the other cluster. This is doable but seems sub-optimal. Another option would be to have some kind of failover list within a cluster itself, and some ability to grab hosts from the failover cluster, but at that point it seems cleaner to me to just build the aggregate cluster type.

For Consul, the root of the problem really is that the UpstreamTLS context has to be the same for all endpoints in the cluster.
Failing over to another cluster is just one possible solution (that happens to conceptually map really nicely to what we are trying to accomplish)
If there was some subset of endpoints used for failover, that would work to so long as we could override the TLS context.

If there was some subset of endpoints used for failover, that would work to so long as we could override the TLS context.

I think this is feasible, especially with the subset LB. I think it would be nice to clarify the actual end user cases for everyone following this thread and we can go from there?

@mattklein123 I think there are multiple requirements here:

Being able to failover to other endpoints (and potentially override settings like TLS context).
Being able to failover to a completely different service discovery mechanisms (e.g. the endpoints usually come from EDS but the failover endpoints come from DNS).

(2) is a requirement coming from Istio based on real-world use cases (not sure how much I can share about specifics). (2) is also seemingly harder to do with just a failover endpoint subset.

@htuch I think Istio's case covers both of them. Background is that our customer tries to failover from internal service to external service.

The internal service and external service have different TLS context. The internal service enables mTLS but the external service doesn't enable;
The external service endpoint also comes from DNS and internal service endpoints are provided by EDS. It would be impractical to add the IP of the endpoints considering service VMs can and will go up and down, and the service's endpoint IP addresses will change frequently.

From everything I've seen here I still think the aggregate cluster is going to be the cleanest way of doing this and will give us a lot of flexibility. IMO it wouldn't be that bad to implement either.

/cc @incfly
Even though I know the https://github.com/envoyproxy/envoy/issues/7135 is closed, but thinking from another perspective as Istio, if we allow TLS settings to be override for a subset endpoints or a sub-cluster, the best effort mTLS adoption can be really simpler.
We no longer blindly retry TLS and fallback to plaintext if fails. The guided information from xDS server can guide the Envoy, endpoint subset A is able to serve mTLS, we do OE; if not, Envoy does not bother to do two attempts of TCP connection.

Working on the design doc of aggregate cluster

aggregate cluster design doc
cc @mattklein123 @htuch @snowp @duderino

Thanks @yxue, I've left a couple of comments on the doc.

Here is a much simpler and more extensible proposal rather than adding yet another type of a cluster.
Extend the filter api in the cluster to be able to decide / mutate the settings that need to be used for a given endpoint. Settings include TLS settings, http2 settings, etc. This filter callback needs to be invoked before making the connection. This is all envoy would have to do. I would not do anything else, else its additional baggage in Envoy core for no good reason.
We (in Istio) can then write a cluster level filter that takes advantage of this functionality. We will keep the filter in a resuable module so that the Consul folks can also use it. Please dont add this aggregate cluster which is confusing AF, and further complicates the load balancing behavior of a cluster.

And in terms of being able to support DNS endpoints in a cluster that otherwise has static IPs, we can do that detection in Istio and ship that entire cluster as a STRICT_DNS cluster. What envoy would need to do is to support locality LB for strict DNS, and not just EDS [which is a useful thing in its own accord].

Aside from the DNS issue, (sorry I wasn't aware of this issue before), overriding TLS context (perhaps with merge context) per endpoint is raised in the past, and I think that is the right way to go vs defining different clusters and aggregate them. This is more lightweight solution in the scenario that each endpoint have different certificate with different SAN, which I believe is the Consul issue and the reason people is also asking about wildcard match of SAN.

overriding TLS context (perhaps with merge context) per endpoint is raised in the past, and I think that is the right way to go

Any pointer on that one?

@incfly I can't find the exact issue, but I think we discussed this in the context of SDS, certification validation context and secure naming, perhaps it is in one of them. There was also request for overriding SNI for endpoint.

Folks, I think the TLS discussion is orthogonal. Can we get some clarity on whether there is an actual need to clusters with completely different discovery types at different priorities (EDS vs. DNS)? This was what motivated aggregate clusters originally I believe.

Walmart's use case does include clusters with different discovery types. EDS for hosts inside mesh and DNS for hosts outside the mesh. The requirement is that failing over from internal to external when internal hosts are unhealthy.

@htuch right so that comes down to my question in the doc, aside from TLS, can't that address by control plane rather than adding a new type of clusters? The proposal sounds overkill with the insane math implied to address the specific issue.

The proposal sounds overkill with the insane math implied to address the specific issue.

@lizan, I haven't update the doc yet. The load balancer will reuse current load balancing algorithm.

@lizan I also suggested control plane early on, I think @duderino might be able to clarify why that wasn't considered the appropriate solution here.

per the original issue by @uk000, they needed to load balance traffic to k8s pod ips plus dns endpoints, wherein traffic to k8s pods should use mtls while the one to dns should use plain text. So @htuch the tls discussion is very much related to this. A cluster with different discovery types and different tls blocks. And I am saying that the control plane can handle the different discovery type thing by simply using strict dns (if locality lb is implemented for it), while the selective tls can be accomplished by the cluster level filters.

@rshriram afaik locality lb has worked with STRICT_DNS for months (ever since load_assignment was added to the cluster), as most of this code is shared between all the cluster types. Are you seeing otherwise?

Thanks for checking on that.. I can confirm the same from Pilot code as well (we have been using it for months). So, the entire issue boils down to doing selective TLS within the cluster. I then do not understand the rationale for doing this aggregate cluster thing..

@htuch the drawbacks from making the control plane do the resolution, as I recall, in order of convincingness:

dns resolvers may return different answers depending on where the dns client is.  If the control plane dns client isn't near the envoy, it may get an answer that is inappropriate or even unroutable for the envoy
general concerns about adding latency by sending the dns resolution through the control plane to envoy.
I've lost track of whether the xDS updates would be incremental, but if not that'd be a ton of work.

@rshriram if we used the filter api to do this, we would have to know in the filter when all the endpoints in a cluster are dead / overloaded / in general bad and then provide an alternate dest addr or, better, fqdn.   Less essential but still desired is to change the TLS details on the alternate connection (potentially failing over from mTLS to TLS or even to the clear which ... well ...).
I am sympathetic to the objection that aggregate cluster is a lot of complexity for a simple request (I just want to failover to a fqdn darnit).  I wonder though...  If this use case is handled with some ad hoc mechanism, and other similar use cases are handled by different ad hoc mechanisms, would we end up with more complexity (both internal and the API) and weird interactions that test cases fail to cover?

I was not suggesting doing DNS resolution from the control plane. I was suggesting using the STRICT_DNS clusters where dns resolution is done by Envoy. Second, all the failover logic is already in place in form of locality/priority lb. We dont have do to anything other than deciding whether or not we should use TLS for that endpoint when establishing a connection to an endpoint in the cluster.

@rshriram others were suggesting doing DNS resolution from the control plane so I was addressing that.
@flatmap13 and @uk000 are the originators of the fqdn fallback request.  Pulling them in to get rid of the lossy filter effects.

Just summarizing the discussion so far for my own understanding :) From end result viewpoint, if istio control plane configures an envoy cluster (type STRICT_DNS) that has both internal k8s and external fqdn endpoints with appropriate priorities and localities, so that envoy routes to the internal k8s endpoints with higher preference and then falls back to the external fqdn, and the external fqdn gets resolved via DNS (with caching etc), that solves the routing aspect of the problem. Then comes the need for the same cluster to support different TLS contexts (mTLS vs TLS vs plain) for different endpoints. @rshriram 's proposal of delegating the TLS context decision to a filter has the benefit of not adding additional complexity until really needed. OTOH, @duderino 's thought has merit that support for hybrid clusters can provide flexibility for any complex future needs as well.

From my perspective, given that the aggregate cluster is an extension and not being built into the core, I don't really have a strong opinion on which way you all go on this topic. The aggregate cluster type with an extremely simple policy to choose between the clusters seems a reasonable and generic solution but if there is a simpler solution to your problem that works also.

Just FYI, I filed https://github.com/envoyproxy/envoy/issues/8016 separately to track the proposal of customize per endpoint TLS configuration, delegating that decision to a cluster filter extension.

@rshriram I'm not sure this boils down to selective TLS / per endpoint TLS configuration within a cluster.   I think @flatmap13 and @uk000 want to prefer local k8s endpoints (mTLS) and if all of those are down failover to a remote fqdn (TLS or even clear).
Can that be addressed using selective TLS / per endpoint TLS config alone?

I think @flatmap13 and @uk000 want to prefer local k8s endpoints (mTLS) and if all of those are down failover to a remote fqdn (TLS or even clear).

This is already addressed using the existing locality based failover.

This is already addressed using the existing locality based failover.

This does not address the mtls issue though, right?
If we do this with a STRICT_DNS cluster we loose all other load balancing functionality, right? Because we would return [foo.svc.cluster.local, foo.external.com], which would resolve to just the IP of the kubernetes svc rather than all of the endpoints. Maybe we instead send all of the endpoints instead of the service name, but I suspect that would have substantial performance implications

This is already addressed using the existing locality based failover.

This does not address the mtls issue though, right?

It does not, but if that's the only limitation it could be addressed by Jianfei's simpler proposal at  #8016

If we do this with a STRICT_DNS cluster we loose all other load balancing functionality, right? Because we would return [foo.svc.cluster.local, foo.external.com], which would resolve to just the IP of the kubernetes svc rather than all of the endpoints. Maybe we instead send all of the endpoints instead of the service name, but I suspect that would have substantial performance implications

I think it was this limitation that took us down the aggregate cluster path.  Thanks John

I think it would be useful to separate the TLS issue to avoid distraction.
The main case is really mixing EDS for a set of endpoints, and using DNS as fallback. EDS is important because it allows real load balancing - using load/health information and properly distributing  the load. I don't think we can afford to trade that off and use only DNS.
I'm not familiar enough with Envoy to comment on how and if it can be done - but I think
any solution should allow proper load balancing to be supported.
It would be great if DNS fallback would also be used if EDS server  is unavailable, which I
assume is equivalent with all EDS endpoints missing.
Regarding MTLS - I think majority of users will use MTLS for mesh expansion and multicluster,
simply because multi-network support in Istio depends on it.

@rshriram wondering if you and @costinm could have a 1:1 to see if you can get to the bottom of the debate here. What @costinm wrote is where I was in terms of understanding before we expanded on this design discussion.

Not stale.

gRPC now has a use-case where we want to override the LB policy when falling back to a lower-priority set of endpoints.  It looks like the aggregate cluster design would address that use-case as well.  Perhaps it makes sense to kill two birds with one stone rather than having to come up with a different solution for each one?

cc @snowp who went through the aggregate cluster review and can comment on Mark's request above. FWIW I don't have any objection to the aggregate cluster proposal as long as we feel that it will be generally useful.

Yeah the aggregate cluster would be able to express that pretty nicely. I think its very flexible solution that probably would be useful in other situations as well, so as long as we have active users of it I would also be fine with including it in Envoy.

+1. While we now have a point solution for TLS context handling during fallback, and some ideas of how to deal with differences in service discovery (mixed EDS and DNS), this is another situation which needs to be handled over fallback. I think 3 examples is sufficient to convince that there probably will be  more in the future and we don't have strong confidence that we have covered all the bases.

Closing this as we landed aggregate cluster. We can open new issues for specific requests.