Adhere to logging best practices... after figuring them out

Our use of logging must

not emit garbage "No handlers could be found" messages to any output, ever
be suitably configurable by applications

. It probably should

be easy enough for us to maintain without asking "what's with these basicConfig calls?"
adhere to Python community-wide best practices (what even are those?)

. Some other open questions are

should we use one logger object, several logger objects in a tree relationship, or several unrelated logger objects?

I'm leaning "just one, with the name 'grpc'", but it would be nice to see that inclination endorsed by some kind of guidance from someone in a position to know.
We currently use one-per-source file, which is roughly one-per-module. Turns out this isn't necessarily the best way to use logger objects. Who knew?

should we expose the logger object we use as a library-scope constant or should we just refer to it by name in our documentation?

This documentation addressing library maintainers is authoritative, but it just isn't sufficiently detailed to be helpful. I've filed this documentation enhancement request to have it improved.

The 1.16.0 release, which includes the new basicConfig calls, clobbers my logging configurations in the microservices I build.

We have the same problem: 1.6.0 release broke logging in our Python app (no output at all now when INFO level logging was visible before). It seems that logging configuration (e.g. using logging.basicConfig()) should be done on the top app level and avoided on a library level.
For example, from the ‘Configuring Logging for a Library’ section of the Python Logging HOWTO:

It is strongly advised that you do not add any handlers other than NullHandler to your library’s loggers. This is because the configuration of handlers is the prerogative of the application developer who uses your library. The application developer knows their target audience and what handlers are most appropriate for their application: if you add handlers ‘under the hood’, you might well interfere with their ability to carry out unit tests and deliver logs which suit their requirements.

@levi-rs, @kuzmiigo: with what versions of the Python interpreter are you executing your code and seeing these problems?

I have the same issue, running grpcio version 1.16.0 on Python 3.6.6 on Ubuntu.
Minimal example:
import logging
import grpc

def main():

    logger = logging.getLogger('myapp')
    hdlr = logging.FileHandler('myapp.log')
    logger.addHandler(hdlr)
    logger.setLevel(logging.INFO)

    logger.info('Hello')

if __name__ == '__main__':
    main()

With grpc 1.15.0 my log message ends up in the log file only, but using 1.16.0 it also gets printed to stderr.

I used Python 3.6.6 on macOS 10.14.

$ docker run --entrypoint python <my docker image>:local -V
Python 3.6.6

Here is a simple example step by step (Python 3.6.6 on macOS 10.14).
Version 1. Let’s enable logging for INFO level:
import logging

log_fmt = '%(asctime)s %(levelname)s %(message)s'
logging.basicConfig(level=logging.INFO, format=log_fmt)

logger = logging.getLogger(__name__)
logger.info('TEST info')
logger.error('TEST error')

Result (as expected):
2018-10-26 21:21:11,255 INFO TEST info
2018-10-26 21:21:11,255 ERROR TEST error

Version 2. Now, let’s import grpc (1.16.0):
import grpc
import logging

log_fmt = '%(asctime)s %(levelname)s %(message)s'
logging.basicConfig(level=logging.INFO, format=log_fmt)

logger = logging.getLogger(__name__)
logger.info('TEST info')
logger.error('TEST error')

Result:
ERROR:__main__:TEST error

As you can see, the format has changed and INFO messages do not appear.
Version 3. Let’s import grpc after we have called logging.basicConfig():
import logging

log_fmt = '%(asctime)s %(levelname)s %(message)s'
logging.basicConfig(level=logging.INFO, format=log_fmt)

import grpc

logger = logging.getLogger(__name__)
logger.info('TEST info')
logger.error('TEST error')

Result:
2018-10-26 21:27:32,829 INFO TEST info
2018-10-26 21:27:32,829 ERROR TEST error

This result is the same as version 1 and is expected. So, it is possible to fix this problem by reordering imports, but it is a hack and it breaks PEP8 compliance.
The reason for this problem is that logging.basicConfig() can be called once only. More precisely, it has effect on the first invocation only. Some quotes from docs:

This function does nothing if the root logger already has handlers configured. (logging.basicConfig() docstring)
As it’s intended as a one-off simple configuration facility, only the first call will actually do anything: subsequent calls are effectively no-ops. (Logging HOWTO)

It seems like it is better to call logging.basicConfig() once in the main(). Libraries should not do that as they will essentially hijack logging configuration from applications that might use them.

This issue was partially addressed https://github.com/grpc/grpc/pull/17064. It seems to put logging back into a reasonable state, but doesn't address Nathan's fundamental question of how we should break logging up in the system.

I'm trying to get a feel for what the rest of the python ecosystem does with its logging, so I looked into the top few libraries on pypi:
urllib3
urllib3 maintains a logger for not quite every module, but for each one that does heavy lifting. The logger name is __name__, and no handlers are registered for any module-level loggers, including NullHandler. Their documentation spells out how to configure logging for the library.
They explicitly configure a library root-level logger called urllib3 to which they attach a NullHandler. This addresses the "no handlers could be found problem".
Their tests explicitly configure handlers, just like ours do.
scrapy
scrapy is more hands-on. It provides a configuration module for its logging and a whole document on how to handle logging with scrapy. It looks like log.py's whole reason for existence is making sure that a handler is attached to to the scrapy handler at startup.
I think the extra complexity here is because scrapy also offers a CLI, so there has to be someway to configure logging without resorting to writing python.

handler with
Based on all of the libraries I've looked at, I think our current approach is reasonable. The one change I would make is to explicitly configure a grpc logger and to only attach a NullHandler to it instead of putting the burden on the author of each new module to configure it there.
tl;dr: I think we should

Configure a logger in each module that cares about logging
Do not attach any handler to the module level loggers (including NullHandlers)
Explicity configure a grpc logger with a NullHandler attached

I've been playing with the 1.16.0 release for a bit, and the logging system annoyingly gets in my way.  I'm using logging.setLoggerClass() to set my default logger class to be one that outputs messages in a specific format, and this runs at the start of the application before all imports, but then every time one of the grpcio packages is imported, it adds a second handler onto the root logger, resulting in duplicate messages for each line.  I have to go and clean that out after all the imports (logging.root.handlers = [] does the trick) - but it would be nice if the grpc libraries didn't do this in the first place.

@alertedsnake We have fixed this problem. It'll be available soon in a patch to 1.16. (https://github.com/grpc/grpc/issues/17017)

@gnossen awesome, thanks!

Resolved by https://github.com/grpc/grpc/pull/17143.


#[belongs_to(Blah)] causes compile error

https://github.com/diesel-rs/diesel/pull/401 causes my build to fail.
Example project that fails: https://github.com/nerdrew/diesel_belongs_to_test
Failure:
% rustc --version
rustc 1.12.0-nightly (197be89f3 2016-08-15)
% cargo test
   Compiling libc v0.2.15
   Compiling byteorder v0.3.13
   Compiling winapi v0.2.8
   Compiling regex-syntax v0.3.4
   Compiling diesel_codegen_syntex v0.7.1
   Compiling pq-sys v0.2.3
   Compiling winapi-build v0.1.1
   Compiling utf8-ranges v0.1.3
   Compiling kernel32-sys v0.2.2
   Compiling diesel v0.7.0 (file:///./diesel)
   Compiling thread-id v2.0.0
   Compiling thread_local v0.2.6
   Compiling memchr v0.1.11
   Compiling aho-corasick v0.5.2
   Compiling regex v0.1.73
   Compiling dotenv v0.8.0
   Compiling dotenv_codegen v0.9.0
   Compiling dotenv_macros v0.9.0
   Compiling diesel_codegen v0.7.0 (file:////diesel_codegen)
   Compiling test_diesel v0.1.0 (file:///./test_diesel)
error: expected ident, found ,
  --> <diesel macros>:28:70
   |
28 | ( foreign_key_kind = $ foreign_key_kind , $ ( $ remaining_args ) * ) , fields
   |                                                                      ^

error: expected ident, found ,
  --> <diesel macros>:28:70
   |
28 | ( foreign_key_kind = $ foreign_key_kind , $ ( $ remaining_args ) * ) , fields
   |                                                                      ^

Build failed, waiting for other jobs to finish...
error: Could not compile `test_diesel`.

To learn more, run the command again with --verbose.

I'm pretty sure you need to use an older nightly -- try the one from
2016-07-07 (same as in Travis file).
In general, each diesel version will be tested against the nightly from the
date of release of the latest stable.
Andrew Ryan Lazarus<EMAIL_ADDRESS>schrieb am Mi. 17. Aug. 2016
um 08:54:

#401 https://github.com/diesel-rs/diesel/pull/401 causes my build to
fail.
Example project that fails:
https://github.com/nerdrew/diesel_belongs_to_test
Failure:
% rustc --version
rustc 1.12.0-nightly (197be89f3 2016-08-15)
% cargo test
Compiling libc v0.2.15
Compiling byteorder v0.3.13
Compiling winapi v0.2.8
Compiling regex-syntax v0.3.4
Compiling diesel_codegen_syntex v0.7.1
Compiling pq-sys v0.2.3
Compiling winapi-build v0.1.1
Compiling utf8-ranges v0.1.3
Compiling kernel32-sys v0.2.2
Compiling diesel v0.7.0 (file:///./diesel)
Compiling thread-id v2.0.0
Compiling thread_local v0.2.6
Compiling memchr v0.1.11
Compiling aho-corasick v0.5.2
Compiling regex v0.1.73
Compiling dotenv v0.8.0
Compiling dotenv_codegen v0.9.0
Compiling dotenv_macros v0.9.0
Compiling diesel_codegen v0.7.0 (file:////diesel_codegen)
Compiling test_diesel v0.1.0 (file:///./test_diesel)
error: expected ident, found ,
--> :28:70
|
28 | ( foreign_key_kind = $ foreign_key_kind , $ ( $ remaining_args ) * ) , fields
|                                                                      ^
error: expected ident, found ,
--> :28:70
|
28 | ( foreign_key_kind = $ foreign_key_kind , $ ( $ remaining_args ) * ) , fields
|                                                                      ^
Build failed, waiting for other jobs to finish...
error: Could not compile test_diesel.
To learn more, run the command again with --verbose.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/diesel-rs/diesel/issues/406, or mute the thread
https://github.com/notifications/unsubscribe-auth/AABOX8C5XhiH9qa5Te1dREaWCZi7_t6bks5qgrAegaJpZM4JmJbv
.

I actually can't reproduce this either with the supported nightly or the latest nightly. Your given example compiles fine for me on both. I did notice that the output you've pasted isn't using the code from crates.io, it's using files on your local machine (Compiling diesel_codegen v0.7.0 (file:////diesel_codegen)). Do you have a .cargo/config which is overriding it? Either way I also notice that the version it states is out of date. The latest version in 0.7.1 so either pointing at the released version from crates.io or updating your local copy should fix this issue for you.

Ah, actually I can see exactly what the problem is. You're using diesel_codegen_syntex 0.7.1 which will be generating code for the macro provided by diesel 0.7.1 but the version of diesel being used is 0.7.0 (unfortunately I can't prevent this, restricting the version of diesel in Cargo.toml would just cause it to use two copies of diesel, not force you to update the version used in your app). But basically some of your dependencies are missing https://github.com/diesel-rs/diesel/commit/ccd2a75ee1bdc1195272e22b1b8abccbe5c7ba74

Ah, ok. I'm using .cargo/config to override diesel and diesel_codegen to use master. Sounds like I need to add diesel_codegen_syntex to that list too. :)


does not handle initialize function overloading

Interface.getFunction() needs to use the full initialize function signature to pull the correct abi https://docs.ethers.io/v5/api/utils/abi/interface/#Interface--specifying-fragments

need to modify this line to use signature or sighash
https://github.com/OpenZeppelin/openzeppelin-upgrades/blob/master/packages/plugin-buidler/src/deploy-proxy.ts#L59

Hi @thegostep!  I’m sorry that you had this issue.
We have been able to reproduce this issue by following these steps:

Create a simple contract with two functions with the same name but different number of parameters
In a Buidler test, attempt to call deployProxy specifying the function as the initializer and provide the appropriate parameters.
The test errors with Error: multiple matching functions (argument="name", value="store", code=INVALID_ARGUMENT, version=abi/5.0.2)

This appears to be only with the Buidler plugin and I couldn't reproduce with the Truffle plugin.
Thanks so much for reporting it! The project owner will review and triage this issue as soon as they can.

Thanks a lot for reporting this @thegostep!
I opened https://github.com/OpenZeppelin/openzeppelin-upgrades/pull/149 to fix it by using Interface.getFunction from Ethers. Once we release the fix it should be possible to use all of function name, signature, or selector to specify the chosen initializer.

The fix has been released in<EMAIL_ADDRESS>


Implementing and using new syntactic features breaks linting

Because the new linter, eslint can't be built against the nightly TS parser and can't understand any newly introduced syntax. Found over at #33290. To properly handle this, we'd need to maintain a nightly-available fork of the parser eslint uses to support TS that we keep in-repo (so when you're working on a new syntactic feature you don't have broken lints) and version alongside our main parser, I think.

 For now, I'll go disable linting on tarball to unblock them, but we really need a plan here.

As discussed offline, there's no reason it can't use the nightly TS parser and in fact, it should.

I'm gunna close this - I think this was mostly just different from before because I skipped the LKG phase of feature rollout in #33290 and the tarball build didn't initially fail (pre-eslint) to warn me about it. I've fixed up the build to fail correctly as here. Now I'm just fixing up the order of operations in the PR so it can LKG, and I think I should be able to reenable linting then.


Centralised archive with multiple todo.txt file

When having multiple txt files and a centralised archive, it would be good to have in archived task name the name of the original file it came from.
Example:
tennis.txt -> these is a task called "find a coach +Lessons"
Then if I decide to archive this task, ideally on archive.txt there should be:
"find a coach +Lessons file:tennis.txt"

Implemented in 3.19.0 with the archive-origin-marker option and the archive-origin-is option. When you set archive-origin-marker to something (e.g. origin), that will be used as the property in the archived task (i.e. origin:/path/to/original/todo.txt).
You can set archive-origin-is to name if origin: should only be todo.txt or set it to stem to reduce origin: down to todo.
Sorry it took so long.
Does this solve the issue?


Update includes after rcutils/get_env.h deprecation

https://github.com/ros2/rcutils/pull/340 deprecates rcutils/get_env.h. This PR doesn't need to wait until the rcutils PR is merged, though.

CI from https://github.com/ros2/rcutils/pull/340#issuecomment-831327939, building & testing everything:

Linux
Linux-aarch64
macOS
Windows

@asorbini FYI, I'm going to go ahead and merge this one so I can also merge https://github.com/ros2/rcutils/pull/340 .  Feel free to keep commenting or let me know if you have any objections.


Unable to add Leap Motion to MRTK Project w/ Leap Motion Orion 4.51

Describe the bug
Unable to configure an MRTK project for Leap Motion when using Leap Motion 4.5.1
To reproduce
Steps to reproduce the behavior:

Start a brand new project.  (I'm using Unity 2019.4.6f1)
Import the Leap Motion Core module from 4.5.1,
Optionally import the Graphic Renderer, Hands, and Interaction Engine packages.
Import MRTK Foundations 2.4.0.
Optionally import MRTK Extensions
Notice errors indicating that the 'Type or namespace name...' appears for many types and namespaces.

Expected behavior
I expected that the code would compile correctly.
Screenshots
Your setup (please complete the following information)

Unity Version 2019.4.6f1
MRTK Version 2.4.0
Leap Motion Orion 4.5.1

Target platform (please complete the following information)

HoloLens
HoloLens 2
WMR immersive
OpenVR

Additional context

Hi @rlbogue, MRTK 2.4 only supports version 4.4.0 and 4.5.0 for the leap motion core assets, but support for the 4.5.1 assets will be added for the 2.5 release.

@CDiaz-MS 4.5.0 doesn't appear to work either.  Are special configurations that are not documented required?

Issue resolved on the HoloDevelopers slack


python 3.9 does not support tensorflow 1.x

Installing with the latest version of python (3.9) will fail with
ERROR: Could not find a version that satisfies the requirement tensorflow (from brain-score)
ERROR: No matching distribution found for tensorflow

Closing for now since it works with the 3.7 version specified in setup.py
https://github.com/brain-score/brain-score/blob/bb5b054e7e97ce0f3191a084c9a46ea2f00053c0/setup.py#L56


Window.Open() blocked by iframe not having "allow-top-navigation" flag

Use case:
A user clicks a chart bar and is navigated to another report view with a custom filter set via query string parameters based on the bar they clicked.
Problem:
The iframe that contains the visual doesn't have the flag "allow-top-navigation" and blocks the navigation.
Access Denied
Reproduction steps:

Add a click handler to the chart bar.  bars.on('click', function(d) {}
Variable "d.Category" is the value required.
Window.Open("UrlOfReport" + "?filter=Table/Column eq d.Category", "_blank");

Hello mcowley5 -
This is done on purpose presently for security\isolation reasons. We are currently working on adding Url navigation APIs from the custom visual.
Hopefully this will reach SU09 which is the Sept release.
thx.

Thanks for the response AviSander


[Bug] SegFault on gradvector test when run with multithreading

Describe the bug
The  gradvector test fails when run with multi-threading.
To Reproduce

Go to test/functionals/areaintegral
Run morpho6 -w4 gradvector.morpho

Expected behavior
Output:
0
[ 1 ]
[ 0 ]
[ 0 ]
[ 2 ]

Actual output:
0
zsh: segmentation fault  morpho6 -w4 gradvector.morpho

This issue also calls for a separate automated test with the run command being morpho6 -w2 or something. Will have to look into its feasibility.

This issue is due to illegal behavior in the test; integrands should not modify state. We will enforce this eventually, for now we've modify the test to remove the possibility of a race condition.
@joshichaitanya3 implemented the new CI test that has caught this and other remaining race conditions in functionals.


3RY1

Key
Value

View
https://database.shabados.com/sources/1/page/862/line/14

Source
Sri Guru Granth Sahib Ji

Page
862

ID
3RY1

Line
PwikE. mIn kipk kI inAweI; qU auriJ rihE. ksuMBwiely ]

Shabadarth (Vol. 3) | July 2010 | SGPC, Sri Amritsar

#95 #1480


Where is the requirement.txt?

I can't find where is the requirement.txt.
And I meet a problem about build fsdet in Windows.

Could you help me about this problem?

Did you find the solution for the requirements.txt file ?


VDYP-176 VRIStart data parsing

#33 included a change to the formatter but did not apply the changes.  This PR did apply the formatter leading to lots of unrelated formatting changes, mostly comment text reflowing to use the larger width. I've made PR #35 that just applies those changes.  Merging that first should make this PR much more manageable for review.


With Payment Intent API, Payment Request Button cannot sit with CardElement in the same Element

Feature request or idea? Consider opening an
API review!

Summary
Imagine we want a form with the Payment Request Button and a CardElement - to offer users a choice of how to pay. we start building this form with React and Stripe Elements.

We put these two elements in the single Element (because it's not possible to have two Elements on the same page as I understand).
Now, when a user pays with a 3D secure card using the Payment Request Button, we first send the token to confirmPaymentIntent from which we get a payment intent with the status "requires_action".
We take this payment method and we send it to handleCardPayment.
However, since we have the CardElement in the same Element, handleCardPayment either attempts to read the data from the CardElement (which will be empty), or, if we provide payment method to handleCardPayment, it'll complain that we can't feed it with a payment method when it sees CardElement on the same page.

A workaround is to hide the CardElement if the payment intent comes with a 3D secure auth request. But that looks like a bug to me. Or am I missing something?

Hi @dlihhats
That does indeed sound like a bug, thanks for reporting!
We'll investigate and keep you updated.
If you happen to have a link to an example were we can reproduce this behavior, that would be helpful as well.

Hi @dlihhats
I was able to reproduce your issue.

it's not possible to have two Elements on the same page as I understand

It actually is possible to do that and it sound like a reasonable workaround to wrap the Payment Request Button and the CardForm in their own separate <Elements> components.
We will fix this issue and release a new version of react-stripe-elements.

Hi @cweiss-stripe!
Thank you for looking into this problem, and sorry that I didn't come back sooner.
When I mentioned that per my understanding it was impossible to put two <Elements> containers on the same page, I was thinking of an error that React threw out saying something like "violated single child check".
(I'm just getting started with React, so can't remember exactly, sorry).
But anyway, when you say you'll fix this issue, will we need to wrap Payment Request Button and CardForm into individual <Elements (perhaps with the different key property?), or will they be able to sit within the same <Elements>?
And yes, sorry for the typo in the original post. I meant as you said: "descendants of the same <Elements>".

Hi @dlihhats
Our plan is to fix this. Using two separate <Elements> containers should not be required. That said, this work-around should work for now. Please let me know if you are still seeing issues with the two <Elements> containers approach.

Hey issue watchers, no news on this yet but we are working to make it much easier to get at the underlying Stripe and Elements instance. Tagging this issue so that we can follow up when it's resolved.

Closing this as this project has migrated to React Stripe.js. If you believe this is still important, please re-open it there.


fix: Websocket Connection failed with ComfyUI server over HTTPS

Pull Request Checklist

[x] Description: Briefly describe the changes in this pull request.
[x] Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
[ ] Documentation: Have you updated relevant documentation?
[ ] Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?

Description
This PR fixes an issue where the websocket fails to connect when the ComfyUI server is secured with HTTPS
(through a reverse proxy or similar).

Changelog Entry
Added

N/A

Fixed

Updated the WebSocket protocol handling to use ws:// for HTTP and wss:// for HTTPS in the comfyui_generate_image function to correct WebSocket connections.

Changed

N/A

Removed

N/A

LGTM, Thanks!


Checks for the blind message generators and PoK during signing

Related to #16.

Added an explicit use of BlindMessageProofVerify in BlindSign to validate the PoK of committed messages, and a check that the generators used in the commitment are not also used for the known messages.
Changed some of the notation to accommodate those changes above. The new notation can be also used more easily if we want to add support for blind signing with multiple commitments.

Discussed on WG call on 15th of dec @BasileiosKal has an action to make some changes requested by @andrewwhitehead and @mikelodder7

I agree and I updated this pr.  However, it does seem that a wider update to the spec may be needed.  It may also be useful to specify the domain parameters used by each function, similarly to the format from here.
It also seems like this  issue (#19 ) will be relevant i.e., if we don't change the way the generators are created, it may be confusing to refer to them as domain parameters and have them be both issuer and number of messages specific.

Agreed. Perhaps we use the current spec as an example for creating the domain parameters. The point is they need to be distinct points whose discrete log is unknown

Related to #19, @andrewwhitehead to supply a review to this PR


Cozylife platform integration

I use the getconfig.py and got this:
$ python getconfig.py 192.xxx.xxx.1 192.xxx.xxx.253
IP scan from 192.xxx.xxx.1, end with 192.xxx.xxx.253
light:
- platform: cozylife
  lights:
  - ip: 192.xxx.xxx.118
    did: 69333560acd82913d7cf
    pid: p93sfg
    dmn: Smart Bulb Light
    dpid: [1, 2, 3, 4, 5, 7, 8, 9, 13, 14]
switch:
- platform: cozylife
  switches:

copy into configuration.yaml and got this...
Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:230
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 4:11:36 AM (1 occurrences)
Last logged: 4:11:36 AM

[139961059280832] The system cannot restart because the configuration is not valid: Platform error switch.cozylife - Integration 'cozylife' not found. Platform error light.cozylife - Integration 'cozylife' not found.
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 230, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 2035, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2072, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 986, in admin_handler
    await result
  File "/usr/src/homeassistant/homeassistant/components/homeassistant/__init__.py", line 191, in async_handle_core_service
    raise HomeAssistantError(
homeassistant.exceptions.HomeAssistantError: The system cannot restart because the configuration is not valid: Platform error switch.cozylife - Integration 'cozylife' not found.
Platform error light.cozylife - Integration 'cozylife' not found.

Tried to find an integration for cozylife but nothing in HASS
What am I missing?

NVM... solved

I know that sounds dumb but it took me some time to figure out I needed a Terminal addon to run python over HAOS and the device has to be IP reserved in router. Could you please add this to README.md

I use HA on a proxmox LXC container so I have access to the OS for running the getconfig.py.  My problem was that the plugin did not get installed into the custom_components directory properly.  It has been running flawlessly since I correct the directory install error.  Not sure what you mean by "bolt button"

Thanks @yangqian for all your work.  If you need some help to further the development let me know.


App should automatically split audio files >25 MB and transcribe each part

Currently, the app can only handle audio files that are less than 25 MB. This is a limitation due to Whisper's max input file size of 25 MB. However, we can get around this limitation by splitting audio files greater than 25 MB into several files which can each be transcribed by the API. Then, the results can be combined into one transcript.
It should be noted that I believe there still needs to be an upper limit on the file size to preserve Whisper API cost. In addition, if the transcript becomes too large (# of tokens), then Groq API rate limits may cause errors on Groq API calls. There should be a check on this size as well.

@mentatbot
Currently, the app can only handle audio files that are less than 25 MB. This is a limitation due to Whisper's max input file size of 25 MB. However, we can get around this limitation by splitting audio files greater than 25 MB into several files which can each be transcribed by the API. Then, the results can be combined into one transcript.
It should be noted that I believe there still needs to be an upper limit on the file size to preserve Whisper API cost. In addition, if the transcript becomes too large (# of tokens), then Groq API rate limits may cause errors on Groq API calls. There should be a check on this size as well.


user/password proxy support for google cloud storage client

Hi,
I'm writing a client for google cloud storage using the java jar google-cloud-storage version 1.37.1
maven excerpt

com.google.cloud
google-cloud-storage
1.37.1

I want to configure it work with proxy which uses username and password, but the code does not work.
I configure my code to use proxy using System.setProperty method
System.setProperty("http.proxyUser", my_username);
System.setProperty("http.proxyPassword", my_password);
However, this does not work.
If I work with proxy without username/password, it works fine.
Also using proxy which expect to get SSL connections, does not work.
My question is:

What shall I do to enable working with proxy with username/password, and with SSL.
I know I'm using an old jar, if I upgrade it to the latest one, will this give me support for proxy with user/password?

Thanks.
Eliyahu

Hi @Eliyahu-Machluf,
Thank you for your question.
Could you check if setting https.proxyUser / https.proxyPassword hepls

quoting: Java Networking and Proxies
The https (http over SSL) protocol handler has its own set of properties:
https.proxyHost
https.proxyPort

I configure both http.proxyUser,http.proxyPassword and https.proxyUser,https.proxyPassword,
But it does not work.
Regarding the cite from 'Java Networking and Proxies', I want my http request be tunneled through proxy, and connect with the proxy using SSL.
To clarify:
My proxy support the following ways to connect with it.

http
http+ user/password
https
https+user/password.

I want my google cloud storage client to be able to connect the proxy at all 4 ways. currently only the first one works.

@Eliyahu-Machluf, could you confirm that your proxy server is configured to meet requirements from Google Storage > Documentation > Support > Proxy Servers
If it's configured properly, could you provide more details about the errors you see working with your proxy.

Hi,
I validated the proxy is configured well, by using curl and executing some get command (loading pages from the web) using proxy.
We have proxy with 4 configurations, listening on different ports:

http
http and user/password
https
https and user/password.

All configurations were checked using curl, and are all fine.
However, when I create my google cloud storage client, only the first configuration (http without user/password) works. Other configurations fails.
Here is a code snippet with the relevant code:
        System.setProperty("http.proxyHost", proxyHost);
        System.setProperty("http.proxyPort", String.valueOf(proxyPort));
        System.setProperty("https.proxyHost", proxyHost);
        System.setProperty("https.proxyPort", String.valueOf(proxyPort));
        if (proxyUser.length() > 0) {
     //System.out.println("Configured proxy user” + proxyUser );
            System.setProperty("http.proxyUser", proxyUser);
            System.setProperty("https.proxyUser", proxyUser);
        }
        if (proxyPassword.length() > 0) {
     //System.out.println("Configured proxy password” + proxyPassword );
            System.setProperty("http.proxyPassword", proxyPassword);
            System.setProperty("https.proxyPassword", proxyPassword);
        }

        GoogleCredentials credentials;
        try {
            credentials = GoogleCredentials.fromStream(new ByteArrayInputStream(googlestorageConfiguration.getJsonCredentials().getBytes()))
                    .createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"));
        } catch (IOException e) {
            throw new RuntimeException(e.getMessage());
        }
        storage = StorageOptions.newBuilder().setCredentials(credentials).build().getService();
        bucket = storage.get(bucketName);

The last line (bucket=storage.get(bucketName) fails when code is configured to use proxy of user/password with http protocol:
When trying to connect proxy http protocol with user/password, I get the following warning:
17, 2020 2:19:10 PM com.google.api.client.http.HttpRequest execute
WARNING: exception thrown while executing request
java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:383)
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:175)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:161)
at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:161)
at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:120)
at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:379)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:176)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:173)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
and finally the error “Error getting access token for service account”
[FILE_FACTORY    ]T:  JAVA EXCEPTION: com.google.cloud.storage.StorageException: Error getting access token for service account:
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:220)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:382)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:176)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:173)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:172)
at com.attunity.AttGCPStorageFileFactory.doSetParams(AttGCPStorageFileFactory.java:364)
at com.attunity.FileFactoryLoggerDecorator.setParams(FileFactoryLoggerDecorator.java:78)
at com.attunity.AttGCPStorageFileFactory.setParams(AttGCPStorageFileFactory.java:74)
Caused by: java.io.IOException: Error getting access token for service account:
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:385)
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:175)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:161)
at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:161)
at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:120)
at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:379)
... 9 more
Caused by: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:383)
... 20 more  (AttLogger.java)
When configured to work with proxy https port I get the following warning
??? 17, 2020 2:04:37 PM com.google.api.client.http.HttpRequest execute
WARNING: exception thrown while executing request
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:383)
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:175)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:161)
at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:161)
at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:120)
at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:379)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:176)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:173)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:172)
And finally an exception failed to get access token:
com.google.cloud.storage.StorageException: Error getting access token for service account:
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:220)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:382)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:176)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:173)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run(RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:172)
at com.attunity.AttGCPStorageFileFactory.doSetParams(AttGCPStorageFileFactory.java:364)
at com.attunity.FileFactoryLoggerDecorator.setParams(FileFactoryLoggerDecorator.java:78)
at com.attunity.AttGCPStorageFileFactory.setParams(AttGCPStorageFileFactory.java:74)
Caused by: java.io.IOException: Error getting access token for service account:
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:385)
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:175)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:161)
at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:161)
at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:120)
at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:379)
... 9 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(Unknown Source)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:383)

But before digging into the exceptions, I have a simple question:
Does the java client for google cloud storage support proxy with user/password, and proxy with https connection?

But before digging into the exceptions, I have a simple question:
Does the java client for google cloud storage support proxy with user/password, and proxy with https connection?

Hi, I think google storage works fine with proxies, at least I've never heard opposite. I believe your problems could be fixed by proper configuration. I'll try to help you out.

java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
As recommended in Oracle 8u111 Update Release Notes

try run java with:
-Djdk.http.auth.tunneling.disabledSchemes=
or add to your code:
System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");

I tried adding System.setProperty("jdk.http.auth.tunneling.disabledSchemes", ""); statement to the code, but this did not solve the issue, i get the same errors when using proxy with http+user/password, or when using proxy with https.

Hi @Eliyahu-Machluf ,
Sad to hear that disabling tunneling tunneling didn't help
Regarding:

StorageException: Error getting access token for service account

It looks like the storage server doesn't get token from your request.
Quoting Cloud Storage authentication > API authentication:

If your requests are being routed through a proxy, you may need to check with your network administrator to ensure that the Authorization header containing your credentials is not stripped out by the proxy.

Because neither storage client nor server do anything proxy specific, I think, the root cause should be searched within the proxy parameters. To eliminate java from the path you can check if gsutil (the command line utility to deal with storage)  works fine behind the proxy.
The following links can help you to install and setup gsutil to work with proxy:
Install gsutil
Configuring gcloud, gsutil and bq to use Proxy Servers

Hi Dmitry,
Thanks for your assistance. Currently I'm busy with other tasks, I hope to have time at next week to check the proxy user/password with gsutil utility.

Hi Eliyahu,
I'll be glad to help. Please keep me informed of the progress.
Thanks

Hi Dmitry,
I had some time today and I did some additional tests:

I've updated the Google Cloud Storage jar I use to version 1.105.2 (from March 2020), and tested it again. It gave the same results: with http proxy without user/password it worked. with user/password, and with https proxy it did not work.

As per your suggestion, I installed the gcloud utility CLI, configured it and run gsutil session. I succeeded to upload file using proxy http with and without user/password.
I don't know how to configure it to run with https proxy. as I cannot configure gcloud to run with https proxy. running the command:
gcloud config set proxy/type https
gives error:
ERROR: (gcloud.config.set) The proxy type property value [https] is not valid. Possible values: [http, http_no_tunnel, socks4, socks5].

To summarize:

http proxy with user/password works with gsutil CLI, and does not work with the Java SDK.
https proxy - I don't know whether this is supported by gsutil CLI, and the Java SDK fails to connect using this configuration.

I want to ask again: Does the Java SDK support http proxy with user/password and https proxy? As it seems it doesn't
Thanks.
Eliyahu

Hi Eliyahu,
Exactly the same case is described here: https://github.com/forcedotcom/wsc/issues/87
Would you try to start your java with: -Djdk.http.auth.tunneling.disabledSchemes=""
Thanks,
Dmitry

Hi Dmitry,
I've added my code the line System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
But still I get an exception: com.google.cloud.storage.StorageException: Error getting access token for service account: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:229)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:406)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:217)
at com.google.cloud.storage.StorageImpl$4.call(StorageImpl.java:214)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:213)
....
Caused by: java.io.IOException: Error getting access token for service account: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:444)
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:157)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:145)
at com.google.auth.oauth2.ServiceAccountCredentials.getRequestMetadata(ServiceAccountCredentials.java:603)
at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:91)
at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:159)
at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:109)
at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:88)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:423)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:542)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:403)
... 9 more

What Java version do you use?

Hi Dmitry,
I use JRE 1.8.0_211, running on windows Server 2012

Would you try to start your java with: -Djdk.http.auth.tunneling.disabledSchemes=""
Your System.setProperty() might be invoked too late and have no effect. JDK library classes are probably already initialized by that moment.

I've tried adding starting the JVM with the option -Djdk.http.auth.tunneling.disabledSchemes="", but it did not help. fails with the same exception

Another possible reason why this option doesn't help, the VM running code to access external resource through proxy is launched separately: you start one application passing extra JVM option and this JVM starts another one without that option.

Hi Eliyahu,
I installed proxy locally and managed to reproduce your failure.
It's much easier for me to find a solution now.

Great.
I'm happy to hear.
Note that Java has a default behavior that if it fails to connect a host using the proxy, it will try to connect directly to that host. So I reproduced the issue, with a machine which does not have direct access to the internet, but only using a proxy.

Good news, I managed to get on through proxy, the following lines are required:
        System.setProperty("http.proxyHost", PROXY_HOST);
        System.setProperty("http.proxyPort", PROXY_PORT);
        System.setProperty("https.proxyHost", PROXY_HOST);
        System.setProperty("https.proxyPort", PROXY_PORT);
        System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
        Authenticator.setDefault(new Authenticator() {
            protected PasswordAuthentication getPasswordAuthentication() {
                return new PasswordAuthentication(PROXY_USER, PROXY_PASS.toCharArray());
            }
        });

Below is the entire program:
package proxy;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.net.URL;

public class Test {
    private static final String PROXY_HOST = "<IP_ADDRESS>";
    private static final String PROXY_PORT = "3128";
    private static final String PROXY_USER = "user";
    private static final String PROXY_PASS = "1234";

    public static void main(String... args) throws Exception {
        System.setProperty("http.proxyHost", PROXY_HOST);
        System.setProperty("http.proxyPort", PROXY_PORT);
        System.setProperty("https.proxyHost", PROXY_HOST);
        System.setProperty("https.proxyPort", PROXY_PORT);
        System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
        Authenticator.setDefault(new Authenticator() {
            protected PasswordAuthentication getPasswordAuthentication() {
                return new PasswordAuthentication(PROXY_USER, PROXY_PASS.toCharArray());
            }
        });
        URL url = new URL("https://example.com/");
        try (BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()))) {
            String inputLine;
            while ((inputLine = in.readLine()) != null)
                System.out.println(inputLine);

        }
    }
}

Hi Dmitry,
These are great news. Thanks for the effort.
I copied the changes to my test utility and managed to work with http proxy with user/password.
Great!
I still cannot work with https proxy. I use Squid proxy, and it is configured also to be accessed using https connection. These connections does not work with and without user/password.
If you have a working environment can you test this as well?
Thanks.

The code I provided works with both: http and https. You can access http://example.com as well as https://example.com
Storage is also accessible through the proxy.

Hi Eliyahu,
I'm afraid I can't help you with establishing HTTPS connection between your client and proxy.
Probably, the following link would be helpful for you:
https://stackoverflow.com/questions/10440690/pros-and-cons-of-using-a-http-proxy-v-s-https-proxy/10442767
I guess, the thing you are looking for is called: Connecting to a proxy over SSL/TLS.
Are you sure that you need this?

Hi @Eliyahu-Machluf ,
Have you managed to resolve your problems with proxy?
Can we close this issue?

Hi @dmitry-fa,
Thanks for reproducing the problem, and showing how to solve it.
I'm closing the issue.

Hi @Eliyahu-Machluf,
The new documentation has been just published:
https://github.com/googleapis/google-auth-library-java/blob/master/README.md#configuring-a-proxy
I hope it will help you to setup an authenticated proxy.

Great.
Thanks.


Added Box python interface

Signed-off-by: ahcorde<EMAIL_ADDRESS>🎉 New feature
This PR is part of this mete ticket https://github.com/ignitionrobotics/sdformat/issues/931
Summary
Added Box Python interface
Test it
from sdformat import Box

box = Box()

Checklist

[x] Signed all commits for DCO
[x] Added tests
[ ] Added example and/or tutorial
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[x] codecheck passed (See contributing)
[x] All tests passed (See test coverage)
[x] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Codecov Report

Merging #937 (1f10483) into main (6c9b4ad) will decrease coverage by 0.01%.
The diff coverage is 68.42%.

:exclamation: Current head 1f10483 differs from pull request most recent head f7e5602. Consider uploading reports for the commit f7e5602 to get more accurate results

@@            Coverage Diff             @@
##             main     #937      +/-   ##
==========================================
- Coverage   88.14%   88.12%   -0.02%
==========================================
  Files         102      103       +1
  Lines       14706    14725      +19
==========================================
+ Hits        12962    12977      +15
- Misses       1744     1748       +4

Impacted Files
Coverage Δ

python/src/sdf/pyBox.cc
66.66% <66.66%> (ø)

python/src/sdf/_ignition_sdformat_pybind11.cc
100.00% <100.00%> (ø)

src/Box.cc
97.22% <0.00%> (+5.55%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c9b4ad...f7e5602. Read the comment docs.


[Feature]: Support different Grayscaling Algorithms

Describe the Feature
Currently the crate uses the grayscale()-method implemented by image-rs. The issue is that this grayscale algorithm uses Rec. 709 coefficients to compute the grayscale, however other popular image libraries like Pillow use Rec. 601 to compute the grayscale. This causes hashes to deviate from their counterparts generated by Python packages like imagehash due to the different grayscales.
As a resolution, the crate should implement is own grayscaling algorithm where we can control the exact coefficients used to compute the grayscale. The resulting image should then be reloaded into the image-rs crate for further resizing and processing.

This is closed with version 1.2.0


ocamlyacc/ocamllex/menhir support

mll/mly generate ml/mli in build dir and output to dist dir

bs-loader is no longer under active development. If you'd like this feature in reason-scripts, please fork bs-loader, implement the feature there, and manually override the bs-loader version.


Submit tarunjana.in

Here is the result:
https://tools.pingdom.com/#5f55f5f659000000

Sorry, but this is considered an "ultra lightweight" website - therefore it cannot be added.

Thanks for your feedback!
Most probably, by the term "ultra-lightweight", you are trying to mean that the website has no real content. This is mostly a website with non-English content, here at https://www.tarunjana.in/bn/posts/.


[Pages]: Update Mailchannel plugin doc

This PR adds steps to generating and using DomainKeys Identified Mail (DKIM) with our Mailchannel plugin.
Addresses #4943

Ok


Testing: The Swagger integration in Ortelius 10 (Service Catalog)

Testing needs to be completed on the Swagger integration.

Hi @TracyRagan @sbtaylor15 Can I take this up? Can you assign it to me?

completed


namespace of OperationTransientException

What's the namespace of OperationTransientException? Is this correct? Is it OperationCanceledException?
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 497c1436-d32c-1fb5-3209-22058fece4b2
Version Independent ID: 4818d7be-c866-39bc-f2c3-d7af59d2cd4e
Content: Retry pattern - Cloud Design Patterns
Content Source: docs/patterns/retry.md
Service: architecture-center
GitHub Login: @dragon119
Microsoft Alias: pnp

Hi @DriezL Thank you for your feedback! We will review and update as appropriate.

I would recommend using Polly
https://github.com/App-vNext/Polly/wiki/Retry

@DriezL I think this came from the old Transient Fault Handling Application Block library. In general it will depend on your specific code base. And to @reddyrajanala 's point, it's generally better to use a general purpose retry library. The code in this article is meant to illustrate the pattern.

Thanks @reddyrajanala , @MikeWasson

I still am not seeing the namespace for this. @mike-urnun-msft how was this issue addressed?  and there is no pull requests linked.


[SPARK-31918][R][BUILD] Upgrade R version to 4.0.2 in the release DockerFiile

What changes were proposed in this pull request?
This PR proposes to upgrade R version to 4.0.2 in the release docker image. As of SPARK-31918, we should make a release with R 4.0.0+ which works with R 3.5+ too.
Why are the changes needed?
To unblock releases on CRAN.
Does this PR introduce any user-facing change?
No, dev-only.
How was this patch tested?
Manually tested via scripts under dev/create-release, manually attaching to the container and checking the R version.

cc @shivaram

Test build #124484 has started for PR 28922 at commit 11648fb.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/29104/
Test PASSed.

Test build #124484 has finished for PR 28922 at commit 11648fb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124484/
Test PASSed.

thanks, merging to master/3.0!

Nice, @HyukjinKwon .

@HyukjinKwon could we also merge this in branch-2.4? If we build a 2.4.7 release we should use R 4.0 for that
cc @holdenk

@shivaram, I ported back by logically resolving conflicts; however, it wasn't tested.


Better support for 0 and 1 categoricals

Unfortunately, many datasets use 0 and 1 for booleans in columns. These must be autodetected as categoricals and also need to be e.g. one-hot encoded when requested.
What do we do when columns have 0, 1 and 2 for example? Are these already detected in the current implementation @Imipenem ?

I guess  0 and 1 are common representations for booleans and might be detected as such by pandas.
0,1 and 2 are counted as numericals, but that's an endless topic, like: What about 0,1,2,3 or 3-15 (some scores like GCS use this scala).
We can try to detect such columns, but therefore we also provide the custom encodings besides autodetect.

0,1 and 2 are counted as numericals, but that's an endless topic, like: What about 0,1,2,3 or 3-15 (some scores like GCS use this scala). We can try to detect such columns, but therefore we also provide the custom encodings besides autodetect.

Totally get what you're saying. Then I guess this should still not be autodetected and users would have to custom encode this. But the column would be numerical, right? Does this work with out encoding currently?

But the column would be numerical, right? Does this work with out encoding currently?

The original column would be numerical (after encoding as well).

As discussed, we need an autodetection for these as well.


more no covers

Codecov Report

Merging #1 (d38ef17) into main (641c2ad) will increase coverage by 0.61%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main       #1      +/-   ##
==========================================
+ Coverage   87.97%   88.59%   +0.61%
==========================================
  Files           4        4
  Lines         574      570       -4
  Branches       70       68       -2
==========================================
  Hits          505      505
+ Misses         30       28       -2
+ Partials       39       37       -2

Impacted Files
Coverage Δ

release_helper/cli.py
80.56% <ø> (+0.99%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 641c2ad...d38ef17. Read the comment docs.


feat(env): add URL validation schema to API_BASE_URL

This update ensures that API_BASE_URL includes the proper protocol (such as http:// or https://). Previously, if the protocol was missing, actions/cache would return an invalid URL error. To prevent this, I have implemented URL validation using zod to verify that the URL is correctly formatted.

Thank you very much! :heart:


Failed to load TLF lists

For some reason, I'm getting a multiple errors when I try to view files keybase on my mac.  Specifically, I'm seeing "ERROR CODE 218", "Failed to load TLF lists", and "An Unexplainable Error has occurred".  I've tried a complete uninstall and reinstall to no avail.  Please let me know if there is anything I can do to resolve this.
Thank you
my log id: 56872cff5ea12ee18881591c

Hi, thanks for the report and sorry for the bad experience.  Your device is hitting a crash in our code when it is trying to load the edit history of one of your folders.  I've put up a fix (#20865) that should make it into the next release (probably next week).
In this case, you might be able to work around it by creating 11 new files in your team directory from a mobile device.  I will message you the team name on Keybase so I don't expose it here.  I'm not 100% sure it will work, but it's my best idea right now.  Otherwise, the next release should fix it.


Use framework7+vue , the theme color and layout of the secondary page navbar cannot be changed.

The theme color and layout of this page，when link to the Statusbar page, the theme color and layout cannot be changed.

This is the routes page.

This is a (multiple allowed):

[x] bug

[ ] enhancement

[ ] feature-discussion (RFC)

Framework7 Version: EXACT RELEASE VERSION OR COMMIT HASH, HERE.

Live Link or JSFiddle: PREFERABLY (IF YOU WANT YOUR ISSUE TO BE RESOLVED ASAP). You can start from forking the following JSFiddle template: JSFiddle

What you did
EXPLAIN WHAT YOU DID, PREFERABLY WITH CODE EXAMPLES, HERE.
Expected Behavior
EXPLAIN WHAT IS TO BE EXPECTED, HERE.
Actual Behavior
EXPLAIN WHAT IS ACTUALLY HAPPENING, HERE.
P.S. Remember, an issue is not the place to ask questions. You can use Stack Overflow
for that or join the #framework7 channel on irc.freenode.net, where we will be more
than happy to help answer your questions.
Before you open an issue, please check if a similar issue already exists or has been closed before.

iOS theme uses different navbar in case you have enabled ios-dynamic-navbar. Check solution here https://github.com/framework7io/framework7-vue/issues/333


more relax about peer lodash version, now < 5

Thanks. Pushed a new version. I had to update the demo as well (some naming change in lodash).


Provided links in documentation are dead

Thanks for letting us know - Can you be a bit more specific please? All the ones in the README work for me.

Under https://github.com/microsoft/azure-api-management-monetization/blob/main/documentation/stripe-details.md, the "monetization overview", "API Management Delegation", "user registration delegation" and "product registration delegation" were just a handful of the links I found to be dead. They all sent me to github's 404 page.
Thank you for looking into this so promptly. Really do appreciate it!

Thanks for reporting! For delegation you might want to look at https://docs.microsoft.com/en-us/azure/api-management/api-management-howto-setup-delegation but we'll update the links.

Thank you!


Some values in annotations in load balancer view are showing up in with black background

Rancher versions: 2.1.1 rc5
Steps to Reproduce:

Create a load balancer
View the load balancer
Open the Labels & Annotations section

Results:  The system label is showing a black background instead of white

Version: 2.2 master 1/2
Label view is now hanging.  Annotation and label section is missing
0:46575 Uncaught TypeError: Cannot read property 'workloads' of undefined


Fix empty reply from server on stop

Sometimes when http://server:9898/stop is called you will (52) Empty reply from server. Prevent that by telling the writer its done.

hold off on this one please, see what Tyson said


Add a position setting to the popovers

While using the tour on a website of mine, a client noticed on small screens the popover will get pretty squished up if the tour is trying is on the right-most portion of the page. be nice to tell it to position the popover to the left so that it doesn't compress so much.

I totally missed it in the API page, closing.


[REEF-876] Update the website for 0.13.0-incubating release

JIRA:
REEF-876
Pull request:
This closes #

@tcNickolas . It looks good to me. I'll test and merge.

Hi, @tcNickolas . It's merged. Now, you can publish the website.


Errors in allcourses.json

there are numerous errors in the allcourses.json file. Although I did tried fixing them but after fixing all the errors the localhost does not start. I would really apprecitate if someone can fix this.

Hey @biswabaibhab007 I fixed the issue #594

Assign me the issue so that I can create PR. Thanks!

I have created a PR. After merge you can start working @biswabaibhab007


Prevent service instance sharers from attempting to share with themselves

As an app dev (sharer), I cannot share service instances with myself. #152109683
NOTE: This PR builds on top of #989, which should be merged first. The actual changes on top of #989 can be viewed in this diff.
What
This PR closes a loophole in the validation logic when a service instance share is created. If the service instance's own space GUID is in the body of the request to the sharing service instances endpoint, a 422 will be returned and no service instance shares will be recorded (or audited). This approach is consistent with the pattern of failing a request if any GUIDs in the body of the request do not correspond to valid spaces that the user can read.
Changes:

Raise 422 when source space is included in list of target spaces

PR

[X] I have viewed signed and have submitted the Contributor License Agreement
[X] I have made this pull request to the master branch
[X] I have run all the unit tests using bundle exec rake
[X] I have run CF Acceptance Tests on bosh lite

Thanks, sapi

Maybe moving all the checks in ServiceInstanceShare#create to a helper method will remove the code complexity level. It's not a complex method per se, just long.

Hi @deniseyu - we accidentally merged this PR. Could you please open a new PR with the same changes?


Haskell Packages Versions Mismatches

Issue description
I've noticed no numerous occasions now that various haskell packages fail to build because they depend on older versions of other packages, which are no longer in nixpkgs.
Sometimes the package has actually been updated to use the newer version, but nixpkgs has an older version of one package and a newer version of a different one, and the version mismatch causes problems.
For example, right now nixpkgs version of diagrams-builder requires lens < 4.16, but only has lens 4.16. The most recent version of diagrams-builder actually requires lens < 4.17, but nixpkgs isn't using that yet. I tried to make a patch for diagrams-builder to update the package requirements, but diagrams-postscript has the same problem, and it seems like I shouldn't have to go down this rabbit hole of patches every time there's an update.
Is there a way to fix this? It seems to me that nixpkgs should have access to older versions of haskell packages to use as dependencies. Isn't that part of the point of nix, to let you mix and match different versions of software?
Steps to reproduce
This happens with a variety of packages, but for a specific example:

cd into nixpkgs repo
run nix-build . -A diagrams-builder

Technical details

system: "x86_64-linux"
host os: Linux 4.14.42, NixOS, 18.09pre140682.5da85431fb1 (Jellyfish)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.0.2
channels(root): "nixos"
channels(qolen): "nixos-18.09pre140682.5da85431fb1"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

I had once a similar , in some cases, jailbreak can solve it, for instance in your overlay
   yst = super.haskellPackages.yst.overrideAttrs (oldAttrs: {
     jailbreak = true;
  });

Oh, that seems useful. Is there an easy way to propogate it to the build inputs? Because diagrams-builder relies on several other haskell libraries that suffer from the same problem.

https://nixos.org/nixpkgs/manual/#how-to-build-with-profiling-enabled
You can do something like this, but with jailbreak = true; which will jailbreak all of your packages.