wandb.config is empty when I use hyperparameter sweeps[Q]

I'm a beginner with wandb. It's a convenient tool and I like it. However, when I try the hyperparameter sweeps, I encouter a problem. I have no idea why the config is empty. The simple version of my code is as follows. As you can see, it's quite readable:
class Trainer:
    def __init__(self):
        sweep_id = wandb.sweep(sweep_config, project=PROJECT_NAME)
        wandb.init()
        wandb.agent(sweep_id, self.run, count=1)

    def run(self):
        with wandb.init():
            config = wandb.config
            opt = self.build_optim(config.optimizer, config.lr)

    def build_optim(self, optim, lr):

        optimizer = None

        if optim == "sgd":
            optimizer = nn.SGD(self.net.trainable_params(), lr)
        elif optim == "adam":
            optimizer = nn.Adam(self.net.trainable_params(), lr)
        elif optim == "adagrad":
            optimizer = nn.Adagrad(self.net.trainable_params(), lr)
        elif optim == "momentum":
            optimizer = nn.Momentum(self.net.trainable_params(), lr)

        return optimizer

The sweep config is like:
        "sweep_config": {
            "method": "grid",
            "metric": {
                "name": "accuracy",
                "goal": "maximize"
            },
            "parameters": {
                "optimizer": {
                    "values": ["adam", "sgd", "adagrad", "momentum"]
                },
                "lr": {
                    "values": [0.01, 0.001, 0.0001]
                }
            }
        }

However, I got this problem:
wandb: Agent Starting Run: hm2y06z8 with config:
wandb: 	lr: 0.01
wandb: 	optimizer: adam
wandb: ERROR Problem finishing run
Traceback (most recent call last):
  File "/root/autodl-tmp/PycharmProjects/code/trainer.py", line 58, in run_sweep
    opt = self.build_optim(config.optimizer, config.lr)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_config.py", line 160, in __getattr__
    return self.__getitem__(key)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_config.py", line 128, in __getitem__
    return self._items[key]
KeyError: 'optimizer'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1876, in _atexit_cleanup
    self._on_finish()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1977, in _on_finish
    self._console_stop()  # TODO: there's a race here with jupyter console logging
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1916, in _console_stop
    self._restore()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1845, in _restore
    self._err_redir.uninstall()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/lib/redirect.py", line 747, in uninstall
    _WSCH.remove_fd(self._pipe_read_fd)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/lib/redirect.py", line 660, in remove_fd
    self._unregister()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/lib/redirect.py", line 648, in _unregister
    signal.signal(signal.SIGWINCH, self._old_handler)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread

I tried printing the wandb.config, but I got {}.
following 2854#issuecomment-955045336, I tried :
import os
os.environ["WANDB_CONSOLE"] = "off"

However, it still doesn't help.

Hi @cuiyc2000,
I'm sorry this is happening to you. Could you share the debug.log and debug-internal.log files associated with one of the runs where this is happening? It will give us some more visibility into your issue and will allow me to provide you better assistance.
Thanks,
Ramit

Hi @cuiyc2000,
I'm sorry this is happening to you. Could you share the debug.log and debug-internal.log files associated with one of the runs where this is happening? It will give us some more visibility into your issue and will allow me to provide you better assistance.
Thanks, Ramit

(base) root@container-bc4e118dac-0ee45f86:~/autodl-tmp/PycharmProjects/code/wandb# cat debug.log
2022-05-15 17:43:56,332 INFO    Thread-12 :4144487 [wandb_setup.py:_flush():76] Loading settings from /root/.config/wandb/settings
2022-05-15 17:43:56,332 INFO    Thread-12 :4144487 [wandb_setup.py:_flush():76] Loading settings from wandb/settings
2022-05-15 17:43:56,332 INFO    Thread-12 :4144487 [wandb_setup.py:_flush():76] Loading settings from environment variables: {'project': 'DLFIN', 'entity': 'cugcuiyc', 'root_dir': '/root/autodl-tmp/PycharmProjects/code', 'sweep_id': 'p037ledq', 'run_id': 'hm2y06z8', 'sweep_param_path': '/root/autodl-tmp/PycharmProjects/code/wandb/sweep-p037ledq/config-hm2y06z8.yaml'}
2022-05-15 17:43:56,332 INFO    Thread-12 :4144487 [wandb_setup.py:_flush():76] Inferring run settings from compute environment: {'program_relpath': 'main.py', 'program': '/root/autodl-tmp/PycharmProjects/code/main.py'}
2022-05-15 17:43:56,332 INFO    Thread-12 :4144487 [wandb_init.py:_log_setup():428] Logging user logs to /root/autodl-tmp/PycharmProjects/code/wandb/run-20220515_174356-hm2y06z8/logs/debug.log
2022-05-15 17:43:56,333 INFO    Thread-12 :4144487 [wandb_init.py:_log_setup():429] Logging internal logs to /root/autodl-tmp/PycharmProjects/code/wandb/run-20220515_174356-hm2y06z8/logs/debug-internal.log
2022-05-15 17:43:56,333 INFO    Thread-12 :4144487 [wandb_init.py:init():462] calling init triggers
2022-05-15 17:43:56,333 INFO    Thread-12 :4144487 [wandb_init.py:init():466] wandb.init called with sweep_config: {'lr': 0.01, 'optimizer': 'adam'}
config: {}
2022-05-15 17:43:56,333 INFO    Thread-12 :4144487 [wandb_init.py:init():510] wandb.init() called when a run is still active
2022-05-15 17:43:56,338 INFO    Thread-12 :4144487 [wandb_run.py:_finish():1685] finishing run cugcuiyc/DLFIN/enq7k2lp
2022-05-15 17:43:56,339 INFO    Thread-12 :4144487 [wandb_run.py:_atexit_cleanup():1867] got exitcode: 1
2022-05-15 17:43:56,342 INFO    Thread-12 :4144487 [wandb_run.py:_restore():1839] restore
2022-05-15 17:43:58,616 INFO    Thread-12 :4144487 [wandb_run.py:_restore():1839] restore
2022-05-15 17:44:04,578 ERROR   Thread-12 :4144487 [wandb_run.py:_atexit_cleanup():1888] Problem finishing run
Traceback (most recent call last):
  File "/root/autodl-tmp/PycharmProjects/code/trainer.py", line 58, in run_sweep
    opt = self.build_optim(config.optimizer, config.lr)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_config.py", line 160, in __getattr__
    return self.__getitem__(key)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_config.py", line 128, in __getitem__
    return self._items[key]
KeyError: 'optimizer'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1876, in _atexit_cleanup
    self._on_finish()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1977, in _on_finish
    self._console_stop()  # TODO: there's a race here with jupyter console logging
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1916, in _console_stop
    self._restore()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/wandb_run.py", line 1845, in _restore
    self._err_redir.uninstall()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/lib/redirect.py", line 747, in uninstall
    _WSCH.remove_fd(self._pipe_read_fd)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/lib/redirect.py", line 660, in remove_fd
    self._unregister()
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/site-packages/wandb/sdk/lib/redirect.py", line 648, in _unregister
    signal.signal(signal.SIGWINCH, self._old_handler)
  File "/root/autodl-tmp/miniconda3/envs/ms/lib/python3.7/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread

(base) root@container-bc4e118dac-0ee45f86:~/autodl-tmp/PycharmProjects/code/wandb# cat debug-internal.log
cat: debug-internal.log: No such file or directory

Not good, no debug-internal.log in my computer.

solved, shouldn't use wandb.init() in __init__

Hello, I put "wandb.init()" into the trainer or self.run() rather than init and still get the empty dict. The wandb version I use is 0.12.21. If I change the version to 0.13.11, I will get the consistent error of "BrokenPipeError: [Errno 32] Broken pipe". Did anyone meet this error?

Update: It has been solved by setting "wandb.run=None" before "run=wandb.init()" in the training function or self.run in this example with version 0.12.21.

Can you provide the full code (or the full simple version) for your corrected code? Running into the same issue


Phone number handling overhaul

Overhauls how we handle phone numbers, promoting them to first class identifiers of individuals. We now use them to de-duplicate at join form submission time, as well as at import script time.
We also now validate phone numbers at all ingress points, rejecting API calls that submit invalid numbers. We also normalize to a standard format, regardless of how the client provides the number, so that string comparisons are sufficient for uniqueness determination.
Also overhauls join form de-duplication, including name change detection, which triggers a notification on slack for an admin to resolve

Closes #374

Closes #374

Whoops I left auto-merge on 😓


Add drawObjectFactory.circle() method

drawObjectFactory uses conus() method for drawing both circle and conus objects.

done in 0.0.4


Update filepath rules documentation

Clarify wildcard syntax and new -Level options

I'm a WDAC feature team member. These updates are in order to resolve issue #4261

The only constructive detail I could imagine would be nice (but maybe cosmetic only?) would be to have the cardinal numbers 0-19 in Table 2. Windows Defender Application Control policy - policy rule options changed to ordinals (0.-19.) for better visual spacing. On the other hand, maybe that could change the logical interpretation of the numbers, I am not sure. What do you think?
( I presume the choice of starting the numbering from 0 signifies that the first one (0) is some kind of major default rule, otherwise I would have questioned why not use 1-20.)

Hi @illfated -- I agree that changing to ordinals might look better, but might also cause confusion. In order to set Enabled:UMCI, you'd need to run the command Set-RuleOption 0. Our current documentation also matches the style of the -help output users can see in PowerShell.

Very well, fair enough. Thank you. Based on your feedback, I see no need to pursue my previous train of thoughts.
@e0i : May we have a final review & merge by the document author or manager?

cc @jvsam

Hi @Dansimp this pull request is ready for your review and approval. Please check, thanks.

Hi @Dansimp-- friendly request to review this PR


Rename announcements to pinax.announcements

Rename django-announcements to uses new pinax conventions.
Changelog:

Drop support for django < 1.5
Rewrite docs for readthedocs with markdown (template based on pinax-blog)
Update test config (travis, tox, coveragerc, ...)
Update .gitignore
Generate django migrations
Move templates to pinax/announcements/

Fixes #31

Great stuff @arthur-wsw!  Thanks!
Just a few items of feedback:

Please see comment on urls
Can you resolve the Travis issue so that "All checks pass"?  I think it's a simple flake8 issue.
Can you add in the makemigrations script from pinax-starter-app?  You'll need to make a couple modifications as this script is actually a template.  Let me know if you need some help.

Again, thanks for this awesome pull request!  Looking forward to seeing it merged.

@arthur-wsw I actually just merged #29 which addressed item 1 in my comment as well as the lint issue identified and mentioned in item 2.  This does mean that you'll need to update your branch with master.  There are instructions for this here.

I update the branch from pinax/master and add the makemigrations.py script from pinax-starter-app.
Let me now if more changes are needed for this pull request to be merge.


Homework 5c

Drift :- https://video.drift.com/v/abG1tXcOguDUVJuOiRbjrY0zgSOD7nj8UQ60aoTTYtQ/

Metric
Value

alert_status
b:white_check_mark: OK/b

bugs
0

code_smells
2

reliability_rating
A (0 Bugs)br/Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)br/Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
b:white_check_mark: OK/b

bugs
0

code_smells
2

reliability_rating
A (0 Bugs)br/Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)br/Note: A being best, E being worst

vulnerabilities
0


Rename dealloc_ref [stack] and enable StackPromotion for OSSA

The blocker for enabling StackPromotion for OSSA was that the ownership of dealloc_ref [stack] was wrong. It should have NonUse ownership, because it's just an end-of-lifetime marker for an alloc_ref [stack].
Also, the instruction syntax dealloc_ref [stack] is very confusing, because it does not do anything like a deallocation.
It's renamed to end_stack_ref_lifetime, which better describes its purpose.
Renaming means: removed the [stack] flag from dealloc_ref and introduced a new instruction end_stack_ref_lifetime (with correct operand ownership of NonUse)

@swift-ci test

@swift-ci benchmark

Performance (x86_64): -O
Code size: -O
Performance (x86_64): -Osize

Improvement
OLD
NEW
DELTA
RATIO

FlattenListFlatMap
5634
5108
-9.3%
1.10x (?)

ObjectiveCBridgeFromNSString
1310
1210
-7.6%
1.08x (?)

Code size: -Osize
Performance (x86_64): -Onone
Code size: -swiftlibs

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 8-Core Intel Xeon E5
  Processor Speed: 3 GHz
  Number of Processors: 1
  Total Number of Cores: 8
  L2 Cache (per Core): 256 KB
  L3 Cache: 25 MB
  Memory: 64 GB

Meghana fixed this here, then closed the PR:
https://github.com/apple/swift/pull/39803
dealloc_ref [stack] semantically deallocates stack space for a deinitialized object (of course there's no runtime deallocation for stack deallocation). dealloc_ref [stack] has has nothing to do with the object's lifetime. Except that the lifetime must obviously end before storage is deallocated.
If we care about the SIL making sense, then it would look like this:
%ref, %addr = alloc_stack_ref $Class // %ref and %addr are physically the same pointers
…
consume %ref     // in practice, this is always an @owned apply argument
dealloc_stack %addr : $Class

After devirtualization:
%ref, %addr = alloc_stack_ref $Class // %ref and %addr are physically the same pointers
…
set_deallocating %ref
end_lifetime %ref     // nop disappears after lowering ownership
dealloc_stack %addr : $Class

The OSSA lifetime utilities might end up working in practice if dealloc_stack takes the reference value as a NonUse and we're careful to check NonUse everywhere instead of isTypeDependentOperand. But there may also be places we iterate over the non-typedependent operands and assume they are real uses (which I think is a nice invariant). Again, those places could be fixed case-by-case, but that's a downside to having an illogical hack in the representation. That just needs to be weighed against the extra work involved in implementing a representation that does make sense. The PR author can decide that.
Regardless, end_stack_ref_lifetime is the wrong name because the object lifetime must end before this instruction. The instruction just marks the deallocation of stack space.

I though about the two-return-value variant of alloc_ref [stack], too. But this would be a massive change and would complicate many other parts of the SIL and optimizer. IMO we don't have to over-engineer this.
@atrick what do you think of dealloc_stack_ref? (I don't want to use the existing dealloc_stack because that's assumed to be paired to alloc_stack in so many places in the compiler)

I though about the two-return-value variant of alloc_ref [stack], too. But this would be a massive change and would complicate many other parts of the SIL and optimizer. IMO we don't have to over-engineer this.

That's a reasonable position. We shouldn't block forward progress on doing that work.

@atrick what do you think of dealloc_stack_ref? (I don't want to use the existing dealloc_stack because that's assumed to be paired to alloc_stack in so many places in the compiler)

Yes, that's fine!
I'm also fine with just adding a property to dealloc_stack that can be checked by the code that searches for alloc_stack. I can't imagine it's many places. And I wonder if it makes sense to be looking for alloc_ref [stack] in those cases.
Then we'd have
(1) dealloc_stack
Where a non-address argument type implies that the stack location was allocated and initialized together
(2) dealloc_stack [ref]
Has exactly same issue as #1. Just a bit more explicit

@swift-ci smoke test

@swift-ci smoke test

@swift-ci smoke test


"Choose File" it doesn't look like a button

In "dark-mode", <input class="form-control" type="file"> it does not stand out. File upload button "Choose File" is not clearly a button. See https://github.com/phpmyadmin/themes/issues/92

Option 1, extend '.btn-dark'

Option 2, extend '.btn-outline-info'

input {
  &[type=file]::-webkit-file-upload-button {
    @extend .btn-dark;
    // @extend .btn-outline-info;
    border-width: 1px;
    border-style: solid;
  }
}

I prefer option 1. Tested in Chrome.


XCode 7 Warnings

Hello,
After updating to XCode 7 and updating the GBDeviceInfo pod we are receiving the following warning:
ld: warning: object file (/Users/USER_NAME/Documents/Git Repositories/APP_NAME/ios/Pods/GBJailbreakDetection/GBJailbreakDetection.framework/GBJailbreakDetection(GBJailbreakDetection.o)) was built for newer iOS version (8.2) than being linked (7.1)
We would still like to support iOS 7.1 and >, so changing the minimum version is not an option.
And about 16 warnings like this:
warning: /Users/lm/Library/Developer/Xcode/DerivedData/ModuleCache/3VKQVBP6KHSD5/Foundation-3DFYNEBRQSXST.pcm: No such file or directory
We have tried clearing the derived data without any change.
Any ideas would be greatly appreciated.

Should be fixed with the latest version 1.1.1

Actually, 1.1.1 only fixed the first set of warnings, the second set is fixed in 1.2.0


Still problems with CORS-Anywhere (deployed to my server)

Hi,
I have deployed CORS-Anywhere to one of my servers.  The environment has a DNS server that I host myself.
The server has both Apache and node on it.  The server's IP is 192.168.xxx.yy.
The CORS-Anywhere server.js is running under the node, listening on port 8080.
The Apache is listening on port 7777.  I have an HTML page that I serve from the Apache at /xhrtest/xhr-fakewava.html (see below).
I am doing a request to: http://192.168.xxx.yy:8080/http://fakewava.whatever.com:7777/xhrtest/xhr-fakewava.html
Webpage after response:  "Missing required request header. Must specify one of: origin,x-requested-with"
Console shows: GET http://192.168.xxx.yy:8080/http://fakewava.whatever.com:7777/xhrtest/xhr-fakewava.html 400 (Header required)
However, the xhr-fakewava.html looks like (i.e., I already have the xhr.setRequestHeader() in it):
<script>
// 1. Create a new XMLHttpRequest object
let xhr = new XMLHttpRequest();

// 2. Configure it: GET-request for the URL /article/.../load
//xhr.open('GET', 'http://fakewava.whatever.com:7777/wavatarget/index.html');
xhr.open('GET', 'http://192.168.xxx.yy:7777/wavatarget/index.html');
xhr.setRequestHeader('X-Requested-With', 'XMLHttpRequest');

// 3. Send the request over the network
xhr.send();

// 4. This will be called after the response is received
xhr.onload = function() {
  if (xhr.status != 200) { // analyze HTTP status of the response
    alert(`Error ${xhr.status}: ${xhr.statusText}`); // e.g. 404: Not Found
  } else { // show the result
    alert(`Done, got ${xhr.response.length} bytes`); // response is the server response
  }
};

xhr.onprogress = function(event) {
  if (event.lengthComputable) {
    alert(`Received ${event.loaded} of ${event.total} bytes`);
  } else {
    alert(`Received ${event.loaded} bytes`); // no Content-Length
  }

};

xhr.onerror = function() {
  alert("Request failed");
};

</script>

Since I already have the setRequestHeader(), why am I still getting the "Missing required request header. Must specify one of: origin,x-requested-with" error?
I also have the demo.html hosted on the same Apache, at /cors-anywhere/demo.html, and I have changed the cors_api_url:
  var cors_api_url = 'http://192.168.xxx.yy:8080/';
But if I go to the demo page, and enter the URL:  http://fakewava.whatever.com:7777/xhrtest/xhr-fakewava.html
and then click GET, I am seeing a strange behavior.  Instead of the Javascript in that xhr-fakewava.html file being executed to do the GET, the web page is showing the contents of the xhr-fakewav.html, i.e., the demo.html seems to be loading the xhr-fakewava.html page, but the Javascript in that page is not executing.
Why might that be?
Thanks!  I hope that I have provided enough information to help figure out how to fix these problems?
Jim
EDIT 1:  BTW, one additional piece of information:
If I use the browser to get that xhr-fakewava.html page directly (i.e., without the CORS-Anywhere URL in front of the xhr-fakewava.html URL), the Javascript that is in the xhr-fakewava.html page DOES execute!
http://192.168.xxx.yy:7777/xhrtest/xhr-fakewava.html

Your code does clearly set the X-Requested-With header.
If you're still getting the Missing required request header. Must specify one of: origin,x-requested-with response, then that means that the server did not receive the header.
A possible reason for this is if you configured your Apache to redirect to the proxy, instead of just passing through the request. With a redirect, custom client-side headers are dropped.
PS. Instead of opening multiple issues about the issues that you've encountered while deploying your instance, please post on the same issue. Even if the issue is closed, I can still see comments and respond.

Hi,
Re. posting on same issue - sorry about that.  I had seen you had closed an earlier issue after you had commented, so I assumed that was because you wanted to see a new issue for any followups.
Re. "if you configured your Apache to redirect to the proxy, instead of just passing through the request", I am not sure what you meant?
The xhr-fakewava.html file is on the Apache local filesystem under the htdocs directory, so the URL http://xxxxx:7777/xhrtest/xhr-fakewava.html should cause the Apache to just serve that HTML page from Apache machine's filesystem.  So no redirect or rewrite there.
I am also unclear about what you meant when you said "instead of just passing through the request"...  What(?) should be passing which(?) request to what (?)?
Also, I had asked about why the Javascript/XHR that is in the xhr-fakewava.html page is not being executed, but rather the content of the xhr-fakewava.html page is just being displayed.
What would cause that kind of behavior??
Jim

Your code snippet contains xhr.setRequestHeader('X-Requested-With', 'XMLHttpRequest');.
You claim that you're getting Missing required request header. Must specify one of: origin,x-requested-with, which means two things: 1) the response is from CORS Anywhere and 2) the header that you have set is not present.
From these two, it follows that the final request to CORS Anywhere is not the original request, which can happen if you redirected the initial request.
Have you tried to use the Network tab of your browser's developer tools to see what's going on?

Yes, I use the network tab constantly, but when I test, actually see nothing in the Network tab.

Here's a shot from a test, with the web developer =>network:

Opening the URL in the browser is NOT a good way to verify whether it works.
CORS Anywhere intentionally requires the request to be made with CORS or XMLHttpRequest/fetch, because the alternative (loading as a document) is a security concern - that would allow the proxied response to execute scripts.

Do you have any idea about what else I can try/change/test?

Is it possible that with the way I have this test setup, with both the CORS Anywhere and the Apache serving the page on the same machine, and the "faking" that there are 2 different hosts, by leveraging the DNS server, is causing some problem that we don't realize?
FYI, I am going to try to stand up a new Apache on anothere machine, and make that new machine the "fakewava.whatever.com" machine and see if that makes a difference...
Jim

Hi Rob,
I think that you were right!!
I was just doing some testing with the demo.html (hosted here) and I had live headers on, and then I put in a URL like:
http://xxx.yyy.com:7777/
In live headers I am seeing a 301 response:
GET /http://xxx.yyy.com:7777/ HTTP/1.1
Host: 192.168.aaa.bb:8080
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36

HTTP/1.1 **301** Please use a direct request
access-control-allow-origin: *
access-control-expose-headers: access-control-allow-origin
cache-control: private
Date: Thu, 23 Sep 2021 20:38:25 GMT
location: http://xxx.yyy.com:7777/
vary: origin

In the web developer==> network, I see 2 requests:

Request URL: http://192.168.aaa.bb:8080/http://xxx.yyy:7777/

and then:

http://xxx.yyy.com:7777/

I assume that the 301 response AND the location header will cause a redirect, but I am unclear about why that redirect is happening?  As far as I am aware, there isn't anything special about this Apache that cause a redirect.
I have seen that Apache does sometimes cause a redirect when the URL is missing a trailing "/" (so Apache causes a redirect which adds the "/" at the end).
Do you know or have a guess?
Also, assuming it is redirecting, would that cause the header to be gone/removed?
Jim

The very basic API of the CORS Anywhere server is very simple: Put the URL of the destination URL after the base URL of the proxy server.
Your experiment shows that you are concatenating many URLs. That doesn't make sense.
The 301 redirect happens because the first part of the URL is the same as the website itself, and that is deemed a coding error that would lead to an inefficient use of resources (without the 301 redirect an internal redirect would happen).

I am really confused by what you said, when you said "the first part of the URL is the same as the website itself"????
Can you specifically show me where I had that?  I may've have messed up pasting URLs in my msg, because it's a kind of pain to obfuscate them, but I don't think that I am doing what you said above.
BTW, I started doing that last set of tests because on one of the issues (not mine, but someone else's thread), you said to bring up the demo page and enter like "http://<IP_ADDRESS>", so I tried that, plus several other examples from you main page like "http://google.com", etc. and I just tried that again using my demo page here, and, for example, http://google.com, and that work.
Since the demo page is working with google, etc., can we start from that (we can circle back to the / case, later?
So anyway, then I entered the following URL into the demo  page text box:
http://charlieeastweb.....com:7777/xhrtest/xhr-fakewava.html
and click GET, and what I see in the response text box is:
GET http://charlieeastweb.....com:7777/xhrtest/xhr-fakewava.html
200 OK

<html>

<body>

<script>
// 1. Create a new XMLHttpRequest object
let xhr = new XMLHttpRequest();

// 2. Configure it: GET-request for the URL /article/.../load
//xhr.open('GET', 'http://fakewava.whatever.com:7777/wavatarget/index.html');
//xhr.open('GET', 'http://<IP_ADDRESS>:7777/wavatarget/index.html');
xhr.open('GET', 'http://charlieeastweb08.sbx.gxaws.com:7777/wavatarget/index.html');
xhr.setRequestHeader('X-Requested-With', 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXMLHttpRequest');

alert(location.host);

// 3. Send the request over the network
xhr.send();
.
.
.

So this test, I was able to retrieve the xhr-fakewava.html page, which contains Javascript and XHR, but it is just treating that as text and not executing the Javascript/XHR.
Why is that?
When I look at the web developer =>Network I see 2 GET requests:

Request URL: http://192.168.xxx.yyy:8080/http://charlieeastweb....com:7777/xhrtest/xhr-fakewava.html
Status code: 301

Request URL: http://charlieeastweb....com:7777/xhrtest/xhr-fakewava.html
Status code: 200 OK
The, the demo is showing the text contents of the xhr-fakewava.html page.
I would've expected another request after #2 above, where the Javascript/XHR code inside the xhr-fakewava.html page was doing the XHR GET using the code I posted earlier, but there is no 3rd request in the web developer=>network display.
So using the demo page, it is at least not blowing up, but even though that is the case, the demo page is still not functioning fully correctly.
BTW, the URL I posted above:
http://charlieeastweb.....com:7777/xhrtest/xhr-fakewava.html
is literally (except for the obfuscated part I removed) the URL I put into the text box in the demo page.  That is not a double URL... it is just the URL to get the HTML page that contains the Javascript.
NOTE that the 1st request above, has this URL:
http://192.168.xxx.yyy:8080/http://charlieeastweb....com:7777/xhrtest/xhr-fakewava.html
IS a double-URL (/, but that double URL was output from the web developer ==>network display.
I DID NOT enter the entire double-URL... into the demo page.  I only entered the last part into the demo page text box:
http://charlieeastweb....com:7777/xhrtest/xhr-fakewava.html
Now, I have a question.  I keep wondering I am misunderstanding how this is supposed to work.
Isn't the URL that I put into the demo page text box suppose to point to an (HTML) page that contains Javascript/XHR code to retrieve some resource (from a different domain)?
That was my understanding, so can you confirm that is correct or not?
Sorry for the long post.
Jim

I have another question that I wanted to try to confirm with you:
When I am getting the "Missing required request header. Must specify one of: origin,x-requested-with" message, I understand that the recommendation was to add the xhr.setRequestHeader() to the Javascript.
I have been ASSUMING that the Javascript that needs to be modified is the Javascript in the xhr-fakewava.html HTML page (which is the HTML page that contains Javascript to do the XHR to get a target HTML test page I have, the "wavatarget.html" page).
Is the "xhr-fakewava.html" page the correct page where the xhr.setRequestHeader() needs to be added?
Thanks,
Jim

Actually, I was just testing with the demo page on your/this website, e.g., putting "http://google.com", and I am unclear - is it actually executing Javascript on the age that it is retrieving?  It seems like it is working the way it is working on my test environment, i.e., it is just retrieving the text of the page and not executing anything?

Rob,
I THINK that I have just had a potential "AHA" moment.  I've been reading all kinds of stuff trying to understand more about CORS Anywhere, and I found this SO thread:
https://stackoverflow.com/questions/29670703/how-to-use-cors-anywhere-to-reverse-proxy-and-add-cors-headers
specifically the post by "Graham Hannington" on Dec 14, 2016.
Reading (and thinking about) what Graham said, I think that I may've had a misconception about what CORS Anywhere does (or how it does it), and I think at least on my side, I've been "coming from" that misconception so I had a really hard time understanding what you've been saying.
Let me explain (and you can tell me if what I am thinking now is right or wrong).
Ok, originally I thought that the CORS Anywhere is facilitate us to retrieve the HTML/Javascript page that we have from a different domain, and that that/our HTML/Javascript page would then execute the Javascript and use XHR to retrieve (for example) a page  from another different domain.
However, I think that what I was thinking is the wrong scenario.
From reading Graham's post (among others), I think that in the scenario above I was using CORS Anywhere and the double URL in the WRONG PLACE.
I think that your intention (and what CORS anywhere helps do) is that it helps us in scenarios where, if we have a HTML/Javascript/XHR/Fetch page that is needing to retrieve a resource, and the RESOURCE is in a different domain, then we would modify the original URL that is pointing to the resource, in the code, and make that into a double URL, and then after that is done, the Javascript/XHR/Fetch will be able to retrieve the resource from the other domain without dealing with the CORs request headers (e.g., Origin) and response header (e.g., the ACAO header).
Is that correct???
If it it, then, man, I have to apologize for being so confused :(!!!!
And if that is correct, then it explains why, with my test, I am just getting the text of my Javascript page, instead of that Javascript code executing!!
Please let me know, because if the above is correct, then I have a lot of thinking to do to try to see if we can leverage CORS Anywhere (I think the new concept does, but I don't want to evaluate how it fits until I know whether I got the right understanding or not).
Thanks,
Jim


Doesn't show or record body from multipart/form-data requests

I installed it via npm. I see:
 content-type: "application/xml; charset=utf-8"
 content-length: "27"

but it isn't printed. The db.json does not have the body either. And the files dir is empty.


/opt/rbenv/versions and /opt/rbenv/shims are owned by root

When rbenv is installed sometimes /opt/rbenv/versions and /opt/rbenv/shims are owned by root.  This causes a permissions error later when a new version is installed.

I have the same problem, opt/rbenv belongs to rbenv user but all the files and folders in rbenv belongs to root, what is the fix?


bug: estimate swap incorrect calculation

from discord: https://discord.com/channels/838986635756044328/842252939149967382/1058063440616374372
the calculation of "outboundFee" for eth (thor.rune -> eth.eth) is somehow too small (for example, it returns 0.00048, although it should be 0.0048), apparently one more "decimal" is missing somewhere.

hmm not sure if this is an issue, in inbound_addresses its stated to be Gwei at 480000 which works out to be eth amount 0.00048, see https://www.alchemy.com/gwei-calculator,

Hi, I would like to suggest to change this line (https://github.com/xchainjs/xchainjs-lib/blob/b1fcdc50ccb1bf5031991aeb7619e273825e0694/packages/xchain-thorchain-query/src/thorchain-query.ts#L232) to this line: let outboundFeeInOutboundGasAsset = calcNetworkFee(params.destinationAsset, destinationInboundDetails).times(3);.
Why:

According to description in https://dev.thorchain.org/thorchain-dev/concepts/fees#outbound-fee the outboundFee = txSize* gasRate * 3
It's more relevant with https://thornode.ninerealms.com/thorchain/quote/swap endpoint output
My test transactions in mainnet with BTC->USDT swap has always refunded since limit value is always greater than real output value after all outbound fees

plus I think that we don't need to substract inbound fee from not chain asset, ie. if we swapping USDT to BTC this fee must be included only with native chain asset (eth, btc, avax, etc) and netoutput and affiliate fee will be more relevant.
so we can transform this line https://github.com/xchainjs/xchainjs-lib/blob/master/packages/xchain-thorchain-query/src/thorchain-query.ts#L253  into following code:
const inputMinusInboundFeeInAsset = isChainAsset(input.asset) ? input.minus(inboundFeeInInboundAsset) : input;

add method isChainAsset helper into utils/swap or thorchain-query:
export const isChainAsset = (asset: Asset): boolean => eqAsset(asset, getChainAsset(asset.chain));

What do you think?

Lets break down the second comment.
First, swapping USDT to BTC.
You need to account for the inbound tx cost when giving the quote. So need to find the ETH tx cost and subtract it away from the USDT amount sent in before you remove the rest of the fees. This will be higher than an ether cost as ERC20 transfers are bigger - contract call.
isChainAsset could work and clean things up a bit, will have to look into that more later.
To the last point, there is always an inbound tx cost to any asset, so you couldn't have inboundFee = 0.

Lets break down the second comment.
First, swapping USDT to BTC. You need to account for the inbound tx cost when giving the quote. So need to find the ETH tx cost and subtract it away from the USDT amount sent in before you remove the rest of the fees. This will be higher than an ether cost as ERC20 transfers are bigger - contract call.
isChainAsset could work and clean things up a bit, will have to look into that more later.
To the last point, there is always an inbound tx cost to any asset, so you couldn't have inboundFee = 0.

Hi @GrassRootsCrypto, thank you for your comments.
Regarding the last point about inbound tx cost, of course inboundFee can't be zero, but we don't need to substract it from input token amount in case of not chain asset since it will skew the real net output and affiliate fee calculation, plus check in covering fee.
So when I as developer calling the estimate method for USDT->BTC I want to receive results as following:

inbound fee in source network chain asset (ETH)
oubound fee in target network chain asset (BTC)
netoutput in target  asset (BTC)

PS: I think that substracting inbound fee in ETH converted to BTC is wrong and will skew real output results

@CodeFunta, thanks for the reply.
I now see it.
When sending 30 USDT, gas is paid in Ether directly from the user's wallet and not taken from the 30 USDT. So inbound fee will be 0 however the user will be paying about $33.43 overall (30 USDT + Gas). The user should be made aware of the gas requirements though when the quote returns.

@CodeFunta, thanks for the reply. I now see it. When sending 30 USDT, gas is paid in Ether directly from the user's wallet and not taken from the 30 USDT. So inbound fee will be 0 however the user will be paying about $33.43 overall (30 USDT + Gas). The user should be made aware of the gas requirements though when the quote returns.

Hi @GrassRootsCrypto , yes and I as developer taking inbound fee , outbound fee and netoutput with UI so user is aware for totals fee and estimated output value in destination token


Need tests and CI

The error in the most recent release could probably have been avoided

agreed, but not something I will have time for in the near future - if you know someone willing to add those, or want to take a stab at it yourself, I'll be more than happy to merge that work in.

I'll take a stab at this.  Will report back

the test framework's been merged in, and while there is a low number of tests (two, right now), adding more tests based on bugs found will now be much easier (as in, possible, rather than not even possible)


VSTest Target

In my SnowShovel project I'm using an slnproj file to run my tests:
  <Target Name="VSTest">
    <ItemGroup>
      <TestProjects Include="tests/**/*.csproj" />
    </ItemGroup>
    <MSBuild Projects="@(TestProjects)" Targets="VSTest" />
  </Target>

Tests can then be executed with the dotnet cli:
dotnet test SnowShovel.slnproj

I feel like this should be in the box. I'm willing to create a pull request if you can assist me in how to test changes in the project.

Good idea, the VSTest target should be in the box indeed. However I don't want to force a convention by having to put the test projects in a tests/ folder and I'd prefer if it just worked without having to specify the test projects.
All test project define the $(IsTestProject) msbuild property, I think the VSTest target could automagically include only these projects.
I have started a refactoring to use the msbuild sdk, this allows to query the csproj properties and make this feature possible.
I should be able to find some time in a couple of weeks to finish the refactoring and implement this feature, I'll leave the issue open for tracking.
Thanks for the proposal

Cool, I didn't know about this property $(IsTestProject). If you can use that to identify the test projects then that is better than enforcing the convention that all test projects are under the tests directory.


Ctrl+C on Ubuntu 14.04 LTS doesn't kill geth, geth takes massive amounts of CPU

@iurimatias I've already made this issue clear via Gitter, but I want to see if there would be other people willing to help tackle this.
This happens after I try to end the session following a embark blockchain command. It kills geth in the shell, however, geth is not dead and lives. PIDs show that it takes up to 50% of CPU resources. Prior to this, the automatic miner works just as you would expect. It's kind of an annoyance having to go into the task manager and seek out and kill geth, so any and all help in solving this is greatly appreciated. Much thanks guys.

I've got this problem too. I'm using Vagrant to set up a VM to run this stuff. See: https://gist.github.com/tom-seddon/407df5a095043a954448
The fix I've found seems to be to set mine_when_needed: false in blockchain.yml. Obviously this disables mining on demand, but at least it works.
When geth does get stuck, you can attach remotely with something like geth attach ipc:/tmp/embark/geth.ipc, type web3, then exit with Ctrl+D. Then both copies shut down. I can only assume something gets in some funny state somewhere.
--Tom

Another option might be to clone https://github.com/jesuscript/vagrant-dapp-env and replace its setup.yml with this very similar one that just specifies versions for a few components: https://gist.github.com/tom-seddon/0dd45250f95121451229
(I decided which versions to use in both these cases by running a VM that does work - vagrant-dapp-env, but built a couple of weeks ago - noting version numbers for geth and solc, and then installing packages on a new VM until I found ones that installed versions of geth and solc that purported to be the same version. Though I note that the geth I get today was supposedly built with go 1.5, whereas the one from 2 weeks ago was built with go 1.4.2.)
--Tom

Interesting thing I also discovered.
If you set console: true with mine_when_needed: false, then you will not only kill the mine_when_needed function (it mines constantly once that is activated), but you will also no longer get this error. I definitely think it has something to do with either console or mine_when_needed

Having the same issue on #123 . Removing and reinstalling embark will make it possible to run embark blockchain again without problems. But obviously shutting down the geth leads to the same issues of geth being stuck at  IPC Service Started.
Setting console: true in turns makes it impossible to start again as it will be stuck at IPC Service Started.

this was an issue with geth that has been fixed since then. as for the /tmp/ folder, embark 2.0 no longer uses that folder and shouldn't have the same issue


Adding "lazy" save case when no meshes were ever loaded

I encountered permission issues on mesh files when calling the save() method when no meshes were ever loaded in my code (using the lazy_load argument).
The problem was fixed by adding a check in the Mesh class to_xml() method, and works like a charm on my side :)
Thank you for maintaining urdfpy alive !

Hey @CarolinePascal , thanks for the PR! I'm sorry it took me so long to get to it--I missed the email. I'm going to work on having a faster turnaround (three months is a very low bar so I'm sure I can beat it).
Thanks for contributing!

@fishbotics Thanks for the merge ! Glad I could help :)


[BUG] I am trying to reset password using hash.bat file on windows system but it is not working

I am trying to reset password using hash.bat file on windows system but it is not working i am getting the following error, could you please help me? or suggest me how to reset admin password

i think it looking for Open Distro folder which i can't see on openserach folder

@opensearch-project/transfer-request Can someone move this to the security repo?


Fix and update link-check

Context: https://github.com/iterative/dvc.org/pull/2932

This PR fixes the odd failures that are happening on PRs by using fetch-depth: 0 on the checkout action, allowing full access to all branches' git history and allowing so we can do a proper github-like diff that doesn't include origin/master changes that have happened since the PR in question.
Separately, this PR upgrades link-check to a new version that uses a small change that uses a single command of git diff origin/master...HEAD instead of a two-step merge-base and diff. Functionally the same thing (probably didn't need a minor bump, just a patch, but oh well we're still in 0.x.x), just more simple and elegant- the only reason we didn't do it before is I wasn't aware of triple-dot syntax. I tested this new version a couple times to make sure it doesn't just break everything, and it works fine.
One big caveat is that merging master (or any Gatsby 3 commit) into our current Gatsby 2 PRs that currently have this issue will cause caching havoc that needs manual intervention to be remedied. I can do so for PRs that really need it, but I'd suggest using local link-check instead since everyone probably has full commit history locally and it should work. At least we'll have it solved for the future!

Thanks @rogermparent ! Looks good i.e. I approve but

merging master (or any Gatsby 3 commit) into our current Gatsby 2 PRs that currently have this issue will cause caching havoc that needs manual intervention to be remedied

That sounds scary haha. No matter when we merge this there will be several PRs open that may require merging master to solve conflicts. Are you suggesting we just ignore the CI link-checker for those?


dm.thinpooldev lacks error handling

I created an LVM2 thin-pool.  I then created a logical volume inside that pool, just to play with it.
I deleted my logical volume and then tried to use docker daemon with it and it would crash:
◀root@elastic01 ~▶ rm -rf /var/lib/docker ; /usr/bin/docker daemon --debug --storage-driver 'devicemapper' --storage-opt 'dm.thinpooldev=/dev/mapper/docker-pool0' --storage-opt 'dm.fs=xfs' --storage-opt 'dm.use_deferred_removal=true'
DEBU[0000] Registering PUT, /containers/{name:.*}/archive
DEBU[0000] Registering DELETE, /images/{name:.*}
DEBU[0000] Registering DELETE, /containers/{name:.*}
DEBU[0000] Registering OPTIONS,
DEBU[0000] Registering HEAD, /containers/{name:.*}/archive
DEBU[0000] Registering GET, /info
DEBU[0000] Registering GET, /images/{name:.*}/get
DEBU[0000] Registering GET, /images/{name:.*}/history
DEBU[0000] Registering GET, /containers/{name:.*}/changes
DEBU[0000] Registering GET, /containers/{name:.*}/json
DEBU[0000] Registering GET, /containers/{name:.*}/stats
DEBU[0000] Registering GET, /_ping
DEBU[0000] Registering GET, /images/search
DEBU[0000] Registering GET, /containers/{name:.*}/top
DEBU[0000] Registering GET, /exec/{id:.*}/json
DEBU[0000] Registering GET, /version
DEBU[0000] Registering GET, /images/json
DEBU[0000] Registering GET, /containers/ps
DEBU[0000] Registering GET, /containers/{name:.*}/export
DEBU[0000] Registering GET, /containers/{name:.*}/logs
DEBU[0000] Registering GET, /containers/{name:.*}/archive
DEBU[0000] Registering GET, /events
DEBU[0000] Registering GET, /images/{name:.*}/json
DEBU[0000] Registering GET, /containers/json
DEBU[0000] Registering GET, /containers/{name:.*}/attach/ws
DEBU[0000] Registering GET, /images/get
DEBU[0000] Registering POST, /containers/create
DEBU[0000] Registering POST, /containers/{name:.*}/stop
DEBU[0000] Registering POST, /containers/{name:.*}/wait
DEBU[0000] Registering POST, /containers/{name:.*}/rename
DEBU[0000] Registering POST, /images/create
DEBU[0000] Registering POST, /images/{name:.*}/push
DEBU[0000] Registering POST, /images/{name:.*}/tag
DEBU[0000] Registering POST, /containers/{name:.*}/attach
DEBU[0000] Registering POST, /auth
DEBU[0000] Registering POST, /containers/{name:.*}/kill
DEBU[0000] Registering POST, /containers/{name:.*}/pause
DEBU[0000] Registering POST, /containers/{name:.*}/unpause
DEBU[0000] Registering POST, /containers/{name:.*}/restart
DEBU[0000] Registering POST, /containers/{name:.*}/start
DEBU[0000] Registering POST, /exec/{name:.*}/start
DEBU[0000] Registering POST, /images/load
DEBU[0000] Registering POST, /build
DEBU[0000] Registering POST, /containers/{name:.*}/resize
DEBU[0000] Registering POST, /containers/{name:.*}/copy
DEBU[0000] Registering POST, /containers/{name:.*}/exec
DEBU[0000] Registering POST, /exec/{name:.*}/resize
DEBU[0000] Registering POST, /commit
DEBU[0000] [graphdriver] trying provided driver "devicemapper"
DEBU[0000] devicemapper: driver version is 4.29.0
DEBU[0000] devmapper: Deferred removal support enabled.
DEBU[0000] Generated prefix: docker-253:0-375713300
DEBU[0000] Checking for existence of the pool 'docker-pool0'
FATA[0000] Error starting daemon: error initializing graphdriver: EOF

After much digging around, I found the problem in the commit message for 2b10749c:

Docker will not take ownership of the specified thin-pool device unless it has 0 data blocks used and a transaction id of 0.  This should help guard against using a thin-pool that is already in use.

I rebuilt my LVM2 thin pool and now it works.
I would expect that instead of error initializing graphdriver: EOF it would display a message about the LVM2 thin pool have been used already or that it was invalid or something I could use to figure out the problem.
I also (stupidly, I admit) didn't understand the documentation and instead of doing a man lvmthin and working things out from there, I tried using /dev/sdb2 as the thinpool.
I would suggest a good error message for that would be handy. :-)

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

Thanks for reporting!
ping @rhvgoyal do you think we can do something better with the error message?

 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.7-coreos
Operating System: CoreOS 766.4.0
CPUs: 1
Total Memory: 3.618 GiB

uname -a
Linux master-pgsql1 4.1.7-coreos #2 SMP Wed Sep 16 22:54:37 UTC 2015 x86_64 Intel(R) Xeon(R) CPU @ 2.30GHz GenuineIntel GNU/Linux

i have configured a lvm thinpool using this
parted -s ${DISK} mklabel gpt mkpart primary 1 100%
/usr/sbin/pvcreate ${DISK}1
/usr/bin/echo -e "y\n" | /usr/sbin/wipefs -f ${DISK}1
/usr/sbin/vgcreate docker ${DISK}1
/usr/sbin/lvcreate -l 99%VG --type thin-pool --thinpool thin_pool docker
METASIZE=$(/sbin/vgdisplay docker | awk '/Free  PE/ {print $(NF-1),$NF}' | sed 's/ //g')
lvresize --poolmetada ${METASIZE} docker/thin_pool

which i have confirmed that is ok
# lvs
  LV        VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  thin_pool docker twi-a-tz-- 198.00g             0.00   0.03
# dmsetup status
docker-thin_pool-tpool: 0 415227904 thin-pool 0 136/472064 0/1621984 - rw discard_passdown queue_if_no_space
docker-thin_pool_tdata: 0 415227904 linear
docker-thin_pool_tmeta: 0 3776512 linear
docker-thin_pool: 0 415227904 linear

but when i try to start docker as daemon to use this thinpool, i get this
/usr/bin/docker --daemon --debug --storage-driver 'devicemapper' --storage-opt 'dm.thinpooldev=/dev/mapper/docker-thin_pool'
DEBU[0000] Registering GET, /containers/{name:.*}/json
DEBU[0000] Registering GET, /_ping
DEBU[0000] Registering GET, /images/{name:.*}/get
DEBU[0000] Registering GET, /containers/json
DEBU[0000] Registering GET, /containers/{name:.*}/changes
DEBU[0000] Registering GET, /containers/{name:.*}/stats
DEBU[0000] Registering GET, /events
DEBU[0000] Registering GET, /images/{name:.*}/json
DEBU[0000] Registering GET, /containers/{name:.*}/export
DEBU[0000] Registering GET, /images/search
DEBU[0000] Registering GET, /images/{name:.*}/history
DEBU[0000] Registering GET, /containers/ps
DEBU[0000] Registering GET, /containers/{name:.*}/top
DEBU[0000] Registering GET, /info
DEBU[0000] Registering GET, /version
DEBU[0000] Registering GET, /images/json
DEBU[0000] Registering GET, /exec/{id:.*}/json
DEBU[0000] Registering GET, /images/get
DEBU[0000] Registering GET, /containers/{name:.*}/logs
DEBU[0000] Registering GET, /containers/{name:.*}/attach/ws
DEBU[0000] Registering POST, /images/create
DEBU[0000] Registering POST, /images/{name:.*}/push
DEBU[0000] Registering POST, /containers/{name:.*}/kill
DEBU[0000] Registering POST, /containers/{name:.*}/start
DEBU[0000] Registering POST, /containers/{name:.*}/exec
DEBU[0000] Registering POST, /exec/{name:.*}/start
DEBU[0000] Registering POST, /containers/{name:.*}/rename
DEBU[0000] Registering POST, /build
DEBU[0000] Registering POST, /images/{name:.*}/tag
DEBU[0000] Registering POST, /containers/{name:.*}/pause
DEBU[0000] Registering POST, /containers/{name:.*}/unpause
DEBU[0000] Registering POST, /containers/{name:.*}/stop
DEBU[0000] Registering POST, /containers/{name:.*}/copy
DEBU[0000] Registering POST, /exec/{name:.*}/resize
DEBU[0000] Registering POST, /containers/{name:.*}/restart
DEBU[0000] Registering POST, /containers/{name:.*}/wait
DEBU[0000] Registering POST, /auth
DEBU[0000] Registering POST, /commit
DEBU[0000] Registering POST, /images/load
DEBU[0000] Registering POST, /containers/create
DEBU[0000] Registering POST, /containers/{name:.*}/resize
DEBU[0000] Registering POST, /containers/{name:.*}/attach
DEBU[0000] Registering DELETE, /containers/{name:.*}
DEBU[0000] Registering DELETE, /images/{name:.*}
DEBU[0000] Registering OPTIONS,
DEBU[0000] [graphdriver] trying provided driver "devicemapper"
DEBU[0000] devicemapper: driver version is 4.31.0
DEBU[0000] Generated prefix: docker-8:9-168
DEBU[0000] Checking for existence of the pool 'docker-thin_pool'
FATA[0000] Error starting daemon: error initializing graphdriver: EOF

how can i know that is causing this?

We seem to have code to return right error if number of data blocks are not 0 or transaction id is not 0. I will debug it and see why it is not being returned.
unc (devices *DeviceSet) checkThinPool() error {
_, transactionID, dataUsed, _, _, _, err := devices.poolStatus()
if err != nil {
return err
}
if dataUsed != 0 {
return fmt.Errorf("Unable to take ownership of thin-pool (%s) that already has used data blocks",
devices.thinPoolDevice)
}
if transactionID != 0 {
return fmt.Errorf("Unable to take ownership of thin-pool (%s) with non-zero transaction ID",
devices.thinPoolDevice)
}
return nil
}

@docwhat @Andre-Freitas can you please try the patch attached to PR #16823 and see if it fixes the issue for you.

@rhvgoyal atm i can't test it atm  because my host is a coreos (and they don't have any compiler tools). I will try to see if i can build docker from source it in another linux distro...

Hitting this with Docker version 1.8.2, build a01dc02/1.8.2 on RHEL 7.2 (latest). Ended up having to delete the docker-pool LV. Eg:
systemctl stop docker
rm -rf /var/lib/docker
lvremove /dev/volumegroupxyz/docker-pool
docker-storage-setup
systemctl start docker

@mglantz This got fixed in 1.9. Try it.

I work with version 1.10 and receive a meaningful error message "error initializing graphdriver: devmapper: Unable to take ownership of thin-pool (docker-pool0) that already has used data blocks" but unfortunately I don't understand how to create the thin-pool LV in the correct way and there is very little documentation about how to set it up for Docker when not on a RedHat/CentOS machine with the docker-storage-setup script.

@geku there's information on configuring it in the "storage drivers" section of the documentation; https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#device-mapper-and-docker-performance

@thaJeztah thanks a lot for the quick reply. I found this documentation already but according to the daemon flags documentation the usage of datadev and metadatadev is deprecated and replaced by thinpooldev. And thinpooldev is not documented afaik.

So far I have found the following working solution:
pvcreate /dev/xvdb
vgcreate docker /dev/xvdb
lvcreate -y -l 90%VG --thin docker/pool0

But then, the metadata volume is too small I think:
  # lvs -a
  LV              VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  [lvol0_pmspare] docker ewi-------  68.00m
  pool0           docker twi-a-tz-- 135.00g             0.64   1.46
  [pool0_tdata]   docker Twi-ao---- 135.00g
  [pool0_tmeta]   docker ewi-ao----  68.00m

Finally I have a working solution where I can set the size of the data and metadata volume:
pvcreate /dev/xvdb
vgcreate docker /dev/xvdb
lvcreate -y -l 5%VG -n pool0meta docker
lvcreate -y -l 90%VG -n pool0 docker
lvconvert -y --thinpool docker/pool0 --poolmetadata docker/pool0meta

The Docker config is --storage-driver=devicemapper --storage-opt dm.thinpooldev=/dev/mapper/docker-pool0

@geku, if you are not using docker-storage-setup, then refer to man lvmthin to figure out how to setup thin pool.

FYI, in my systemctl status docker, I was getting error:
time="2016-06-19T05:18:17.906673293-04:00" level=fatal msg="Error starting daemon: error initializing graphdriver: Unable to take ownership of thin-pool (docker_vg-docker--pool) that already has used data blocks"
running docker-storage-setup again did not fix error. I just kept getting output:
Logical volume "docker-pool" changed.
I had to manually remove logical volume with
lvremove /dev/mapper/docker_vg-docker--pool
then rerun docker-storage-setup, then restart docker and I am good to go.


Inline values provider isn't called with new view port when scrolling editor content

from https://github.com/microsoft/vscode/issues/105690#issuecomment-804710058
DebugEditorContribution.updateInlineValueDecorations needs only be called when a provider is registered. The built-in fallback solution does not need to be called on scrolling because it does not try to only show inline values for the visible area.

@isidorn thanks, the fix works great.
Initially I thought that the viewport passed to the inline provider is a bit off, but then I noticed that I'm passing getVisibleRangesPlusViewportAboveBelow() which adds more lines above and below which is a good idea as my experiments show:
With getVisibleRanges() new lines appear without inline values and then after 200ms the values appear.

With getVisibleRangesPlusViewportAboveBelow() the inline values scroll smoothly into the view because they were already added in the "PlusViewportAboveBelow" area.

Of course that only helps if scrolling stays within the "PlusViewportAboveBelow" area. For page wise scrolling it won't help.

@weinand thanks for the gifs, they show the two strategies quite well.
Based on your comment I am adding verfied label.

@testforstephen please note that we debounce this call. So we only call the provider once the user stops scrolling.
I think in practice there will not be too many calls. Did you have a chance to try it out?

Yes, I tried it in Java debugger and debounce is taking effect. It's not the debounce I'm worried about, it's the fact that the inline values returned are volatile and not cached. As I scrolled up and down in the editor, new requests kept coming and I had to recalculate the same lines again.
Here is an example. In the very first window, the inline values in the stack frame are fully displayed, then scrolling down and back, the previous results are lost and I have to recalculate the results again. Ideally, it should be like lazy loading. I would prefer not to refresh the view again if the results have been provided before.
https://user-images.githubusercontent.com/14052197/113988755-5ab65000-9882-11eb-8a1f-b4665effe355.mov

@testforstephen thanks for the details.
I've created https://github.com/microsoft/vscode/issues/120803

@testforstephen thanks this is helpful!
@weinand let me know if I can help with the issue once we decide to tackle it.


Unsupported measurement unit: cm

Since the last few updates, it does not work here. It seems it don't recognise the metric values and if I move it outside the metric value loop, it recognise it as GAL instead of CM.
Logger: homeassistant
Source: custom_components/eyeonwater/eow.py:155
Integration: EyeOnWater (documentation)
First occurred: 06.44.14 (108 occurrences)
Last logged: 15.39.18
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 182, in async_refresh
await self._async_refresh(log_failures=True)
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 283, in _async_refresh
self.async_update_listeners()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 110, in async_update_listeners
update_callback()
File "/config/custom_components/eyeonwater/sensor.py", line 85, in _state_update
self._state = self.meter.reading
File "/config/custom_components/eyeonwater/eow.py", line 155, in reading
raise EyeOnWaterAPIError(f"Unsupported measurement unit: {read_unit}")
custom_components.eyeonwater.eow.EyeOnWaterAPIError: Unsupported measurement unit: CM

Hi,
Are you in Canada or US?
Something weird happens here:
    try:
        domain = data[CONF_DOMAIN]
    except KeyError:
        domain = "com"

    if domain == "com":
        eow_hostname = "eyeonwater.com"
        metric_measurement_system = False
    elif domain == "ca":
        eow_hostname = "eyeonwater.ca"
        metric_measurement_system = True
    else:
        raise WrongDomain(f"Unsupported domain {domain}. Only 'com' and 'ca' are supported")

Do you mind to remove the integration from HA and to add it again?
Hopefully it will fix this weird domain issue.

Hi,
Are you in Canada or US?
Something weird happens here:

    try:

        domain = data[CONF_DOMAIN]

    except KeyError:

        domain = "com"

    if domain == "com":

        eow_hostname = "eyeonwater.com"

        metric_measurement_system = False

    elif domain == "ca":

        eow_hostname = "eyeonwater.ca"

        metric_measurement_system = True

    else:

        raise WrongDomain(f"Unsupported domain {domain}. Only 'com' and 'ca' are supported")

Do you mind to remove the integration from HA and to add it again?
Hopefully it will fix this weird domain issue.

I live in Denmark, but we are using the com domain. I've tried to login on the ca website, but that's not working.
Just changed it from False to True and now it's working 👍
Perhaps an extra option (checkbox) for the setup could be an solution? (Metric / Imperial)

Cool, I had no idea that EOW is used in Europe.
Yes, the assumption ".com == Imperial" was definitely wrong. That's how development without specs looks like :-)
I will add one more extra property for solving the issue.
Thank you for reaching out.

Cool, I had no idea that EOW is used in Europe.
Yes, the assumption ".com == Imperial" was definitely wrong. That's how development without specs looks like :-)
I will add one more extra property for solving the issue.
Thank you for reaching out.

I had no idea to.
But from what I can look up, it's so far the only water meter here that's approved by the authorities, that are not somehow mains powered.
The ones that are not mains powered, are not "true" smart meters, because they use an battery powered adapter on an existing meter, and the RF is only short range, so it require the water company to drive around and collect the data, and by law, customers here, are entitled to access real time data (max 24 hours old) so it would be near impossible for the water company to collect data otherwise.

@smulle48 I've pushed changes for EOW hostname and measurement system decoupling.
Please update and verify

@smulle48 I've pushed changes for EOW hostname and measurement system decoupling.
Please update and verify

It's working perfect 👍
Thx for the fix 😁


Change shebang to /bin/bash

In Ubuntu the default shell points to dash rather than bash. As such, this substitution (line 375):
    GUEST_IFNAME=${GUEST_IFNAME:0:15}

returns an error as that syntax is not supported by dash. Setting the shebang do /bin/bash fixes this.

Good point, thanks for catching this!

I have been using this script on Alpine, where the default shell is ash and there's no bash command.
I'd suggest reverting this change and rewriting the ${GUEST_IFNAME:0:15} with awk or similar utility.

I use a busybox container without bash too

For now, my recommendation would be to add bash to your container; but if someone wants to replace the ${GUEST_IFNAME:0:15} with a call to cut, I'll be happy to receive a PR! 👍


Support nested objects for grouping tables

Grouping tables with ngTable works with flat dataset like this:
usersWithArticles = [
{user: 'aaa', articleTitle: 'article a1', articleContent: '....'},
{user: 'aaa', articleTitle: 'article a2', articleContent: '....'},
{user: 'bbb', articleTitle: 'article b1', articleContent: '....'}
...
];

and ngTable rebuild list to nested form with attributes data and value.
IMHO nested json objects is more natural in js something like:
usersWithArticles = [
{user: 'aaa', articles: [
    {title: 'article a1', content: '....'},
    {title: 'article a2', content: '....'}
]},
{user: 'bbb', articles: [
    {title: 'article b1', content: '....'}
]}
...
];

Why did you choose first flat way? Second type of nested objects will be supported? I thing nested object for group list will be very useful for many developers.

Now I tried to hack it:

    {{group.data[0].user}}

    {{article.title}}

it works but it is tricky ;-)
So same question: Do you have plan support nested objects? I know for nested objects is impossible to change grouping column, but for me it is not important.

I tend to create nested ng-table's for the nested objects that you describe. EG:

Here's the typical html markup to define such nesting:

perfect, thanks a lot. is it possible to use it now?

Yes. I've been using this approach since an around v0.5

BTW, the grid you saw in the screen shot had custom css styles. In other words there are no styles from ng-table specific to nesting tables.


provide opaque tensor type ttl::Tensor

The class ttl::Tensor should have no template parameters.
discussions:

Q: What will be the relationship between std::Tensor and std::raw_tensor?
A: currently raw_tensor only hides dtype and rank but not device

A: should we use ttl::raw_tensor instead of ttl::Tensor?

Q: what else does ttl::Tensor provide?
A: tensor with non-consecutive storage.

The Tensor type is currently implemented under stdml namespace.

should the opaque tensor be under the namespace ttl? e.g. ttl::opaque_tensor (without any template parameters)


Loose Coupling

Can flask admin be used without SQLAchemy? I'm really not a fan of the project and would prefer to use a different ORM such as masonite-orm.

This was a goal of Flask-Admin. SQLAlchemy support is just an extension that's built on top of Flask-Admin core. See here: https://github.com/flask-admin/flask-admin/blob/master/examples/simple/app.py
If you want to get fancy and support your ORM, check this doc: https://flask-admin.readthedocs.io/en/latest/advanced/#using-different-database-backends


Starting the editor delay regression

Godot version
3.x (3.5 RC4?, not present RC3)
System information
Linux Mint 20.1
Intel Core i5-7500T
8gb RAM
Intel HD Graphics 630
Issue description
After building the editor after #61995, I noticed that the editor seemed to be taking longer at startup (and the project manager). Switching between "synchronous" and "asynchronous" seems to change the delay.
Steps to reproduce
Switching from "synchronous" to "asynchronous" seems to create the delay.
Measured startup in debug:
3828 ms (sync)
4278 ms (async)
I haven't thoroughly investigated this, but was worth a mention as any increase in startup time can be problematic for debugging / development.
Minimal reproduction project
N/A.

CC @RandomShaper @Calinou

What kind of asynchronous support does the editor report on your system? Full native or via secondary context?

What kind of asynchronous support does the editor report on your system? Full native or via secondary context?

Full native support.

Ok this is much worse. BeatInvaders which is a more complex game, took:
Async : 16401 ms
Sync: 1977 ms

Can you try clearing the shader cache? I have a suspicion.

Shader caching in general tends to have a lot of overhead on the first run, also on 4.0.alpha. I think we'll need to add something like godotengine/godot-proposals#3657 to 3.x as well.

This isn't just the first run though, even once cached, it still seems quite a bit slower. While it may be advantageous to run in the editor as far as catching bugs is concerned (especially on betas), are there any other advantages? Because if it impedes startup time / stability it might be nice to be able to easily switch it off in the editor / project manager. Should there be a separate setting for editor and for projects?

@lawnjelly, @timothyqiu, thanks for the info you provided. I think it all boils down to a change I mistakenly reverted from my last PR.

Might be fixed by #62251, please confirm and close.

Just pulled latest 3.x, unfortunately no change. And no output in shader_cache again. I'll look more in the morning.

To be sure we're on the same page, is the slowdown the same as before the latest bug my recent PR caused and which was fixed by another? In other words, is the current slowdown the same as when the change to async in the editor was just merged?

To be sure we're on the same page, is the slowdown the same as before the latest bug my recent PR caused and which was fixed by another? In other words, is the current slowdown the same as when the change to async in the editor was just merged?

Yup, delay seems the same.

Fixed by #62302 .


Added sleep to introduce delay between full and incremental runs.

Description of what I changed
Cloud build fails intermittently on dockerised pipeline's incremental run stage. Upon investigation, I found that the thread remains active after full run and when the controller receives request for incremental run, it throws error saying that A pipeline is still running. This can be because it is trying to create resource tables.
Introducing delay between the runs to see if this solves the problem.
E2E test
Tested
TESTED:
N/A
Checklist: I completed these to help reviewers :)

[x] I have read and will follow the review process.

[x] I am familiar with Google Style Guides for the language I have coded in.
No? Please take some time and review Java and Python style guides.

[x] My IDE is configured to follow the Google code styles.
No? Unsure? -> configure your IDE.

[x] I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)

[x] I ran mvn clean package right before creating this pull request and added all formatting changes to my commit.

[x] All new and existing tests passed.

[x] My pull request is based on the latest changes of the master branch.
No? Unsure? -> execute command git pull --rebase upstream master

Codecov Report

Merging #709 (81d71e0) into master (a57f13b) will increase coverage by 4.02%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master     #709      +/-   ##
============================================
+ Coverage     34.04%   38.07%   +4.02%
+ Complexity      291      251      -40
============================================
  Files            71       51      -20
  Lines          3818     2663    -1155
  Branches        437      311     -126
============================================
- Hits           1300     1014     -286
+ Misses         2397     1529     -868
+ Partials        121      120       -1

Impacted Files
Coverage Δ

...in/java/org/openmrs/analytics/PipelineManager.java
13.74% <0.00%> (-1.09%)
:arrow_down:

... and 29 files with indirect coverage changes


Feature/pomodoro update

I hope it's OK that I renamed it from Promodoro to Pomodoro.
Plus there is new feature and UI fix.

Looks great - thanks!

I just noticed:
setWatch(() => { console.log('BTN1') }, BTN1, { repeat: true });
        setWatch(() => { console.log('BTN2') }, BTN2, { repeat: true });	        setWatch(() => { console.log('BTN2') }, BTN2, { repeat: true });
        setWatch(() => { console.log('BTN3') }, BTN3, { repeat: true });	        setWatch(() => { console.log('BTN3') }, BTN3, { repeat: true });
        setWatch(() => { console.log('BTN4') }, BTN4, { repeat: true });
        setWatch(() => { console.log('BTN5') }, BTN5, { repeat: true });

Is there any need for this? Looks like debug code to me - won't do anything except spam the console?

@gfwilliams with console logs I just followed previous pattern. So it makes sense to remove it. And I still need to update changelog anyway, add notes about BTN4 and BTN5

Thanks!


Height of inputs change when errorText show or hide.

Problem description
Height of inputs change when errorText show or hide.
According to material-design it shouldn't change.

Current Behaviour

That's not what Google is doing on his new login page. I believe that we should let user the flexibility to chose the behaviour. That's what #7085 is trying to achieve.

@oliviertassinari  I don't understand how minHeight works to make height not change when errorText shows and hides

The idea is to provide an empty helper text in order to get a placeholder for a potential future message.

However, FormHelperText is not rendered until there is an error in most use cases and libraries such as redux-form-material-ui
Also I want the one with error text to as short as the one without error text, not the opposite.

https://github.com/erikras/redux-form provides high flexibility, we can easily write our own wrapper around Material-UI or even move that new logic into redux-form-material-ui.

Another point: LOGIN form is different from normal forms.
A normal form need to avoid any form of surprises like element position change or element height change.  A login form need to be as concise as possible for the first impression. Also, when user try to login, they seldom need the error messages, which is totally opposite in normal forms.
@oliviertassinari So what google is doing in their login form is not breaking their own design guide.


MirroredStrategy error with Object detection retraining

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04.4 LTS
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source
TensorFlow version (use command below):b'v1.11.0-0-gc19e293' 1.11.0
Python version:3.6.5
Bazel version (if compiling from source): 0.15.0
GCC/Compiler version (if compiling from source):5.4.0
CUDA/cuDNN version:9.1/7.1.2
GPU model and memory: AWS g2.8large Grid K520 4036MiB

Describe the current behavior
Training stops with the attached error log when retraining Mobilenet V2 SSD model.
Describe the expected behavior
Trainig should start and use the available 4 GPUs
Code to reproduce the issue
Following is the code change I made to https://github.com/tensorflow/models/blob/master/research/object_detection/model_main.py
def main(unused_argv):
  flags.mark_flag_as_required('model_dir')
  flags.mark_flag_as_required('pipeline_config_path')

  distribution = tf.contrib.distribute.MirroredStrategy()
  config = tf.estimator.RunConfig(model_dir=FLAGS.model_dir,train_distribute=distribution)

Other info / logs
Error Log:
TF_MirroredStrategy_error.txt

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.
If you think we've misinterpreted a bug, please comment again with a clear explanation, as well as all of the information requested in the issue template. Thanks!

@ymodak I will post the issue on StackOverflow as well. Can you please tell what other information is needed? I thought I provided all details asked by the issue template. Thanks.

@anilmaddala Yes you did provide all the info. requested by the template. Actually I didn't update the snippet completely. Now I did.  Apologies.


group return value is ignored

Calling a command defined as @click.group() with standalone_mode=False does not propagate the return value as expected.
For instance:
import click
import sys

@click.group(invoke_without_command=True)
@click.option("-l", "--local-configuration", type=str)
def main(local_configuration):
    print('Inside main')
    return 1

if __name__ == '__main__':
    print(main(sys.argv[1:], standalone_mode=False))

$ python test.py
Inside main
None

It does work when the command is defined with @click.command(), hence I suppose this is a bug.
Environment:

Python version: 3.7.5
Click version: 8.0.3

Thanks for opening the issue, I ran a git bisect to see where this behavior changed.
It showed https://github.com/pallets/click/commit/079f91a2396c5c9264f34f95894259b7b15eb89c as when the behavior changed I'm not sure why moving to a src directory would cause this.

It seems related to
https://github.com/pallets/click/blob/d5932d674d804cf67b20e4f1c3546abf5aa65494/src/click/core.py#L1636-L1638
where return value of super().invoke(ctx) is replaced with [] or None before passed to _process_result.

Not sure what to do with this one. The difference in behavior between groups and commands wasn't documented, and it's not clear why group returns weren't used when adding support for command return values.
The return value of a command is the return value, if any. The return value of a group is the result of calling its result_callback on the return value(s) of it subcommand(s). result_callback only gets the subcommand result, it won't get a value returned by the group itself.
I think it could make sense to use the group's return value as well though. The problem is making sure existing code continues to work after the change, at least for a deprecation period.

A group can be invoked without a subcommand, and returns a value. Would the value be returned directly, or still passed through the result callback? What if the callback only expects values returned by subcommands?
A subcommand is required, and only the subcommand returns a value. This is the current expected behavior.
A subcommand is required, and both the group and the subcommand return values. This is tricky because the result callback would only be expecting one value based on current behavior. Should the subcommand value overwrite the group value?
The group is in chain mode, the return value of each subcommand is collected in a list. Should the return value of the group be the first item in the list? What if the group doesn't return anything, and the result callback expects that every item is not None (that's how the example in the docs works)?

A possible solution that sidesteps the questions above: only use the group return value if invoked without subcommand. That is, if the group is treated like a command itself, return its value as if it were a command.
Regardless of the solution, the behavior of return values needs to be much better documented.
For now, you could add a result callback to the group that returns what you would have returned from the group if it was used without a subcommand.
@main.result_callback
@pass_context
def group_result(ctx, value):
    if ctx.invoked_subcommand is None:
        return 1

For #1178, in 8.0 we changed the behavior so that the result callback was called even if there was no subcommand, passing None.
Previously, the group return value was returned, as long as the group wasn't in chain mode and was called without a subcommand. In chain mode, the group return was always ignored since the intention in that mode was to have a pipeline of return values.
The behavior before 8.0 was basically what I just described as a possible solution to this issue. Now that I've looked over this more, I think that makes more sense, I think the change should be reverted in 8.1.

A compromise that should satisfy that issue and this one would be to pass the group return value to the result callback if it's invoked directly. As shown in the workaround I posted above, you can tell whether the value is from a subcommand or not based on ctx.invoked_subcommand.


settings websocket

Summary
PR Checklist
~- Is this a multi-tenancy feature/bug?
- [ ] Yes, the relevant RBAC changes are at:~
~- Do we need to backport changes to the old Rancher UI, such as RKE1?
- [ ] Yes, the relevant PR is at:~
~- Are backend engineers aware of UI changes?
- [ ] Yes, the backend owner is:~
Fixes #

https://github.com/harvester/harvester/issues/4463

test step:

Set additive-ca to empty
Configure backup-targe S3
If a websocket pushes backup-target data, check whether the UI display meets the expectations。
websocket may not push data, so you need to repeat the above operation

Occurred changes and/or fixed issues

Technical notes summary

Areas or cases that should be tested

Areas which could experience regressions

Screenshot/Video

Use new PR to fix : https://github.com/harvester/dashboard/pull/1129


support clova speech

Plugin for Clova Speech-to-text from Naver cloud platform,
Documentation:
ai-application-service-clovaspeech

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for your contributions @josephkieu!
How can we validate that the plugin is working as intended? could you provide API keys to test this solution?


Debugger: JS object property order changes in the debugger

Type: Bug
The order of properties on a JS object changes in the debugger which can be confusing if you're not aware of this. This has no run-time effect though, just a visual bug on how the object is displayed in the debugger.
Steps:

Create at least Javascript object with at least 3 properties
Assign a string name on each property.
e.g.

let persons = {
    "20 jan":{},
    "10 Mar":{},
    "30 Dec":{},
    "test": 2,
    3: 2,
    yolita: 10,
    eden: "test"
}

console.log each property and it logs it in this order expect for the property with a number as its key

Run with Debugger and the order of the properties in Persons variable changes.

It has no run-time effect but i was confused when I was debugging and I thought one of my codes changed the order of the properties in the object.
VS Code version: Code 1.70.1 (6d9b74a70ca9c7733b29f0456fd8195364076dda, 2022-08-10T06:08:33.642Z)
OS version: Windows_NT x64 10.0.19044
Modes:

System Info

Item
Value

CPUs
AMD Ryzen 5 5600X 6-Core Processor              (12 x 3693)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.89GB (9.57GB free)

Process Argv
--crash-reporter-id 9534e654-e9c9-408b-91b8-86117f703b83

Screen Reader
no

VM
0%

Extensions (2)

Extension
Author (truncated)
Version

sqltools
mtx
0.23.0

sqltools-driver-mysql
mtx
0.2.0

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30542924
vsaa593:30376534
pythonvs932:30410667
wslgetstarted:30449410
cppdebug:30492333
pylanb8912:30545647
vsclangdc:30486549
c4g48928:30535728
d7fjc500:30543455
dsvsc012:30540252
azure-dev_surveyonecf:30545436

First of all why are you worried so much about property order in a JS object ?
By design JS objects' keys / propertis isn't ordered.
More info: JS | RFC 7159 | JSON

Properties in the debug console are sorted first by visibility, then alphabetically.

The described behavior is how it is expected to work. If you disagree, please explain what is expected and what is not in more detail. See also our issue reporting guidelines.
Happy Coding!

First of all why are you worried about property order in a JS object ? By design JS objects' keys / properties isn't ordered. It's an unordered collection of key-pairs unlike arrays which is an ordered collection.
More info: JS | RFC 7159 | JSON

I see. I wasn't aware of this. I just kinda confusing when you make keys out of dates.
For example in the project that I'm doing. I have to make a key format based on [departure date - return date].
Upon getting the data from the database, they are sorted by date, the order if these date is from the nearest date to the farthest date. This array of data goes through some code that attaches the a key based on [departure date - return date] an is turned into an object.
But when I check them on the debugger, they are now re-arranged in this way.
{
 "01 Dec 2023-07 Dec 2023": "trip details here",
 "02 Jun 2023-08 Jun 2023": "trip details here",
"30 Dec 2022-05 Jan 2023: "trip details here",
}

Anyway, thanks for pointing this out.


[Customizer] is down

When i try to build package on https://foundation.zurb.com/sites/download.html/

Did you retry it?

Works on my machine.

Same. Maybe a compilation error but we will need the logs. I'm waiting for accesses to Foundation servers from Zurb, this could help.
Closing this for now. @sashabeep please provide more informations (version, settings) if you are unable to download it after several tries.

Customizer f-d  up again...

https://foundation.zurb.com/sites/download.html/custom-f6

components used:

It happens time to time when the archive is not generated fast enough.

Looks like, some hash assigned with browser session to prevent duplicates or so, but in all of my browsers nothing works for now

See https://github.com/foundation/node-foundation-customizer/pull/13


chore(dns): example cloud dns reverse lookup

Add example for DNS reverse lookup
Example reference - https://cloud.google.com/dns/docs/zones/managed-reverse-lookup-zones#console

@msampathkumar
Thanks for the PR! 🚀✅ Lint checks have passed.

@msampathkumar
Thanks for the PR! 🚀✅ Lint checks have passed.


error running overlaphighways linter

Running from latest master of overlaphighways linters I get error 👇

230 tiles processed in 1s/Users/maning/projects/git/osmlint/node_modules/@turf/turf/turf.js:179
    if (!isNumber(coordinates[0]) || !isNumber(coordinates[1])) throw new Error('coordinates must contain numbers');
                                                                ^

Error: coordinates must contain numbers
    at Object.point (/Users/maning/projects/git/osmlint/node_modules/@turf/turf/turf.js:179:71)
    at module.exports (/Users/maning/projects/git/osmlint/validators/overlapHighways/map.js:130:40)
    at gotData (/Users/maning/projects/git/osmlint/node_modules/@mapbox/tile-reduce/src/worker.js:68:5)
    at notify (/Users/maning/projects/git/osmlint/node_modules/queue-async/build/queue.js:62:29)
    at /Users/maning/projects/git/osmlint/node_modules/queue-async/build/queue.js:49:29
    at tileUnzipped (/Users/maning/projects/git/osmlint/node_modules/@mapbox/tile-reduce/src/mbtiles.js:38:7)
    at Unzip.onEnd (zlib.js:227:5)
    at emitNone (events.js:91:20)
    at Unzip.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
254 tiles processed in 1sevents.js:160
      throw er; // Unhandled 'error' event
      ^

Error: channel closed
    at ChildProcess.target.send (internal/child_process.js:584:16)
    at handleTile (/Users/maning/projects/git/osmlint/node_modules/@mapbox/tile-reduce/src/index.js:131:23)
    at DestroyableTransform.handleZXYLine (/Users/maning/projects/git/osmlint/node_modules/@mapbox/tile-reduce/src/index.js:148:5)
    at emitOne (events.js:96:13)
    at DestroyableTransform.emit (events.js:188:7)
    at addChunk (/Users/maning/projects/git/osmlint/node_modules/readable-stream/lib/_stream_readable.js:291:12)
    at readableAddChunk (/Users/maning/projects/git/osmlint/node_modules/readable-stream/lib/_stream_readable.js:278:11)
    at DestroyableTransform.Readable.push (/Users/maning/projects/git/osmlint/node_modules/readable-stream/lib/_stream_readable.js:245:10)
    at DestroyableTransform.Transform.push (/Users/maning/projects/git/osmlint/node_modules/readable-stream/lib/_stream_transform.js:148:32)
    at DestroyableTransform.write [as _transform] (/Users/maning/projects/git/osmlint/node_modules/binary-split/index.js:24:14)

@maning you should use

node: v6.12.3
npm : 3.10.10

Those mostly come with nvm.

Tried again, I get an output but process did not finish.  Same issue as above.
However, the output doesn't seem to be overlapping highways.
See https://gist.github.com/maning/3f421f7bf1d07c47e36affc5d7637501


Create AST node to represent AT TIME ZONE expressions

Address #4289

Implemented AT TIME ZONE as an Expression and applied the changes we discussed.

Addressed comments, translate AtTimeZone to FunctionCall after planing as an optimizer

last few comments, otherwise looks good

Thanks a lot!
Just remove the unnecessary constructors to address the comments.

Addressed the comments.

Merged!


Add profile picture enhancements

Added code to remove the current profile pictures in favor of higher-resolution unstretched profile pictures

@HitItAll please edit the target branch from mlool:main to mlool:develop


Restrict CORS origins for socket.io connections

In #98, I removed strict checking for CORS requests. My reasoning is that it's not providing meaningful protection at the moment, but it's creating significant usability issues (see #97).
CORS origin checking currently does not much protection because:

The user base of TinyPilot is so small that CSRF attacks are highly unlikely.
Even with CORS origin checking, third-party origins can still send requests to the /socket.io route from the user's browser. CORS enforcement prevents them from reading responses, but there's nothing useful in the responses anyway.

We should revisit this decision in the future to ensure that we're not sacrificing real protection by dropping origin checking, even if (1) is no longer true.

True, I think CORS has almost no benefit especially since there is no authentication and traffic is not going over HTTPS. So I'm more worried about passwords that are  possibly going over an unencrypted connection.
If HTTPS would be added (not easy to do and takes additional resources), activating CORS would be a no brainer as using the correct hostname becomes a requirement anyway.

Reopening as the previous fix caused a regression. When users accessed TinyPilot from a different URL, it broke the websocket connection.
Need to investigate more how to make SocketIO cooperate.

From tinkering some more, it seems to be related to nginx. Connecting directly to TinyPilot without nginx works as expected.
I need to review the nginx config here:
https://flask-socketio.readthedocs.io/en/latest/#using-nginx-as-a-websocket-reverse-proxy


Add release flag

1- add a --release flag, that disables debugging features (logging prints, api return codes and runtime errors).
2- make all debugging features on by default. (make current debugging flags disable debugging)

These changes to the CLI must be described in the docs. I also believe the example from the READ should tell developers to use the --release flag.


Error when trying to run distance_between_vehicles.py with the metrics module and my own logfile

I am getting this error message when trying to run distance_between_vehicles.py and I cant figure out where my mistake is

OS: [Windows10]
CARLA Version [e.g. CARLA 0.9.11]
Python version [e.g. 2.7]
Version [e.g. 0.9.11]

Hey @Brandeyy I've never seen that before, can you pass me the .log file? I'll try to run it, see what's happenning

@glopezdiest this is the log file, this also happened with other log files I recorded myself. The example one worked just right.
VehicleConstructionSite.log

After some digging, it seems like the spectator camera, as well as sensors and other objects, has a bounding box of 'nan' (and an extent of 'inf' (Id: 432 origin: (-nan, -nan, -nan) extension: (-inf, -inf, -inf))
The weird thing is that does type of string should be correctly parsed and is indeed what's happening to me, as I just run your test and it didn't fail. Anyway, I'll just add a check for that case, so that it doesn't happen, as the bounding box of these actors is pretty much irrelevant. For your case, you should check this function and do the same, something like:
def parse_bounding_box(info):
    if 'nan' in info[3]:
        location = carla.Location()
    else:
        location = carla.Location(
            float(info[3][1:-1])/100,
            float(info[4][:-1])/100,
            float(info[5][:-1])/100,
        )

    if 'inf' in info[7]:
        extent = carla.Vector3D()
    else:
        extent = carla.Vector3D(
            float(info[7][1:-1])/100,
            float(info[8][:-1])/100,
            float(info[9][:-1])/100,
        )

    return carla.BoundingBox(location, extent)

Thanks a lot !


Missing default shortcut keys for "ToggleNoteAccent" and "ToggleNoteGhost" inputs

DefaultControls.json
@TheNathannator

Got any suggestions for this? I don't know which keys are available and which aren't, so I didn't worry about it at the time.

If you run the "InputEditor" scene you'll be able to edit and save the default controls with all the usual warnings about input conflicts as you would run the game normally.
Note that you might hit a couple of asserts running this scene, it's a little wonky but you can ignore them.

Good to know, thanks.

I'm personally not too fussed as to what the actual keys should be. "G" for ghost would have been nice but that's already taken by the global view toggle, so it's a bit of a free-for-all

Yeah there's not much key real estate left lol, from what I can tell all that's left is ZV,.;/[]'\789, along with F1-F12, the numpad, and Insert/Home/End

They don't have to be completely unique, you can duplicate some keys as long as the input groups they're set to aren't configured to be conflicting. Just pushed up a small fix to the drums group panel inputs as well to free that up.
Also note the SetNoteAccent and SetNoteGhost are also missing default inputs as well I've just noticed.


ENV support without additional wrappers

Reopen of #938, because I there is no button reopen.

Is your feature request related to a problem? Please describe.
I want use human-readable config, without additional wrappers. Instead I need to wrap every option to ${oc.env:KEY,default_value} and its looks ugly:
token: ${oc.env:TOKEN,???}
locale: ${oc.env:LOCALE,en}
debug: ${oc.env:DEBUG,false}
verbose: ${oc.env:VERBOSE,false}
db_url: ${oc.env:DB_URL,sqlite:///data.db}

I want just:
token: ???
locale: en
debug: false
verbose: false
db_url: sqlite:///data.db

Provide every option also with ENV variable can be helpful, example, for docker when the best will be just add arguments to docker run instead of add volumes and etc.
Describe the solution you'd like
Something like overwrite_with_env argument on OmegaConf.create and OmegaConf.load methods.
So this will just wrap every option to "$\{oc.env:{0},{1}\}".format(key.upper(), value) on parsing time.
Describe alternatives you've considered
Monkey patch as workaround, but this isn't good working, and produce additional bugs.
Additional context
python-decouple provide support for ENV overwriting from box, but their architecture, in all cases, different from your.

I implement https://github.com/omry/omegaconf/issues/938#issuecomment-1138626501, but in some time noticed that this don't work if one of values ??? (https://github.com/omry/omegaconf/issues/938#issuecomment-1140508996).

Sorry my original solution did not handle missing values properly. Please give this a try:
import os
from omegaconf import OmegaConf

cfg = OmegaConf.create(...)
for key in cfg:
    if key in os.environ:
        cfg[key] = os.environ[key]  # overwrite with environment var

Working, thanks again. Really don't know why I didn't think about this.