Error when running with docker

  File "/usr/src/app/agent/run.py", line 50, in run_agent
2023-08-17T07:20:35.442545533Z     await assistant.conduct_research()
2023-08-17T07:20:35.442547293Z   File "/usr/src/app/agent/research_agent.py", line 143, in conduct_research
2023-08-17T07:20:35.442549203Z     research_result = await self.run_search_summary(query)
2023-08-17T07:20:35.442550993Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-08-17T07:20:35.442552853Z   File "/usr/src/app/agent/research_agent.py", line 128, in run_search_summary
2023-08-17T07:20:35.442554863Z     os.makedirs(os.path.dirname(f"./outputs/{self.directory_name}/research-{query}.txt"), exist_ok=True)
2023-08-17T07:20:35.442556793Z   File "<frozen os>", line 225, in makedirs
2023-08-17T07:20:35.442558993Z PermissionError: [Errno 13] Permission denied: './outputs/89a73f7e-705a-4ee6-9583-5f78845a2fcb

Try to use gpt-reasercher with docker and query some simple question but encounter this error.

I have the same exact error! Hmmmm
This is ChatGPT's response:
The error message suggests that the directory /usr/src/app/outputs does not exist when trying to change ownership.
From the Dockerfile you've provided, it seems you've removed the line to create the outputs directory. The chown command is trying to change the ownership of a directory that doesn't exist, thus resulting in an error.
To fix this, you need to re-add the line to create the outputs directory before you try to change its ownership.
I dunno if thats a very accurate solution or not

Hey @TC23345 this issue is currently known on Windows apparently. Docker is trying to create a directory without permissions from the root user. Can you try running in terminal a simple command to mkdir a directory and see if it persists?

@assafelovic
I have exactly the same issue. running on docker 4.2.0. I'm upgradeted to 4.22.1 but reproduced the same behavior
I think it occuers when gpt-researcher is trying to output the result.
In the log window I see
c:\Users\alex\Desktop\gpt-researcher>docker-compose up
Recreating gpt-researcher_gpt-researcher_1 ... done
Attaching to gpt-researcher_gpt-researcher_1
gpt-researcher_1  | INFO:     Started server process [1]
gpt-researcher_1  | INFO:     Waiting for application startup.
gpt-researcher_1  | INFO:     Application startup complete.
gpt-researcher_1  | INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
gpt-researcher_1  | INFO:     <IP_ADDRESS>:42406 - "GET / HTTP/1.1" 200 OK
gpt-researcher_1  | INFO:     ('<IP_ADDRESS>', 42408) - "WebSocket /ws" [accepted]
gpt-researcher_1  | ERROR:    Exception in ASGI application
...
gpt-researcher_1  |   File "/usr/src/app/agent/research_agent.py", line 128, in run_search_summary
gpt-researcher_1  |     os.makedirs(os.path.dirname(f"./outputs/{self.directory_name}/research-{query}.txt"), exist_ok=True)
gpt-researcher_1  |   File "", line 225, in makedirs
gpt-researcher_1  | PermissionError: [Errno 13] Permission denied: './outputs/227403b7-d6f2-4468-a4d1-ae8c17c1a406'
I did as instructed and open a bash window from Docker and I reproduce the same behavior in the bash window
$ ls
CODE_OF_CONDUCT.md  Dockerfile  README.md    actions  client  docker-compose.yml  main.py  processing        venv
CONTRIBUTING.md     LICENSE     pycache  agent    config  js                  outputs  requirements.txt
$ cd outputs
$ ls
11016334-9ca4-4bf7-9748-5417d603363a  690c5c52-50fb-4cf3-8ee7-ea165e97b778  adad9a83-b544-402b-9b6d-b83bd0df87d8
1c519e4c-22f6-4dc2-864f-58c018fd77b9  8aa7f8c7-080d-437f-b9dc-eccd047d5b57
5792f061-56f7-49c3-9416-cdbcf20f4332  a9da62ce-028f-4f12-af38-639dcb0ba6ef
$ mkdir testdir
mkdir: cannot create directory ‘testdir’: Permission denied
$
For the record, I tried the docker-compose up cmd from a normal cmd window + also when start a cmd window as windows admin => same result.
Here is some additional context about why I'm trying with docker in the first place...
I tried gpt-researcher about 2 weeks ago and I was quite pleased with it. I was running it on windows natively without using docker at all. However, I think it was a few days ago, I started having "chrome crashed" errors in gpt-researcher log instead of nice reports. Looking in the README it says many people have chrome-driver issues and suggests to downgrade chrome.
However, my chrome has automatic updates and I intend to keep it that way. indeed, over the past 2 weeks, I think chrome upgraded twice silently (now running 116.0.5845.111 (Official Build) (64-bit))
so I'm not really comfortable with degrading, let alone sticking to a specific version. I did try to see if I could upgrade the chrome-driver version or at least identify a chrome version very close to mine.
looking into the code dependancies
gpt-researcher\venv\Lib\site-packages\webdriver_manager\drivers\chrome.py
it looks for versions in
https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json (line 67)
and in this json I notice
116.0.5845.96 => so a .96 and not a .111 => i'm ok doing this slight downgrade for now.
Looking how to perform this downgrade, it looked like google does not really allow you to do this (to be confirmed). The first result seems to be based on 3rd party software => there is no way I'm using 3rd party software for chrome upgrades/downgrades.
So I edited config/config.py file
and changed
self.selenium_web_browser = os.getenv("USE_WEB_BROWSER", "chrome")
to
self.selenium_web_browser = os.getenv("USE_WEB_BROWSER", "firefox")
(line 23)
so now the reports can be written again, but no web scrapping seems to work. I always get errors in the log like
An error occurred while processing the url https://www.businesscoot.com/en/study/the-accounting-software-market-france: Message: Error: NS_ERROR_UNKNOWN_HOST
Stacktrace: [...]
or
Scraping url https://www.frenchbusinessadvice.com/accounting-guides/ with question Daily activities of an accountant and a chartered accountant in France
An error occurred while processing the url https://www.ifac.org/about-ifac/membership/profile/france: Message: 'geckodriver.exe' executable may have wrong permissions.
So I tried "edge" (running Version 116.0.1938.62 (Official build) (64-bit))
This time, there is no error message in the log...
but the reports starts with saying: "Due to the unavailability of the requested URLs, I am unable to process the direct information from the listed pages.  However, based on general knowledge about xxx, I can provide an analysis."
So I think it did not work either because the initial reports I got from gpt-research were a lot more elaborate, would cite sources, etc.
I have not tried to download/install opera and see what happens. I would prefer that we fix the docker File permission error instead. I like the idea to have everything tidy inside a container.
I just hope this more detailed feedback helps solve the issue 🤞

Hi @assafelovic, I am use wsl2 on windows 11. I have to run the docker contain using root user to handle the permission error.
After added user: root inside docker-compose.yml, gpt-researcher run as expected.
I am not very familiar with docker, but I enter the gpt-researcher container's shell, try to create file inside outputs, the permission error is also encounter. Perhaps something wrong in user permission configuration?

The error is due to a permissions issue with creating the outputs directory. The directory does not initially exist and is generated in runtime. I just added a proposed fix to latest version #158 . Can you please try pulling and try again?
If that doesn't work, I suggest trying to install regularly (see README) and manually adding the outputs folder to better debug where the issue is coming from. Lmk if this solves it!
@lkyhfx @alexisdal @TC23345

Hi @assafelovic
I was under the feeling that version #158 had been merged into master, so I just tried deleting my entire gpt-researcher directory and did the git clone and docker-compose up commands again as instructed in the README
I hid my openai key for obvious reason but I can see the runtime seems to work until the result needs to be written in 'output'
check output below
Not sure if that's relevant by i'm using cygwin64 and bash because the default terminal in windows is terrible to work with (cmd)
$ rm -rf gpt-researcher

$ rm -rf gpt-researcher

$ git clone https://github.com/assafelovic/gpt-researcher.git
Cloning into 'gpt-researcher'...
remote: Enumerating objects: 784, done.
remote: Counting objects: 100% (463/463), done.
remote: Compressing objects: 100% (189/189), done.
remote: Total 784 (delta 317), reused 357 (delta 266), pack-reused 321
Receiving objects: 100% (784/784), 1.54 MiB | 8.31 MiB/s, done.
Resolving deltas: 100% (439/439), done.

$ cd gpt-researcher/

$ export OPENAI_API_KEY="************"

$ docker-compose up
[+] Running 2/2
 ✔ Network gpt-researcher_default             Created                                                                                0.1s
 ✔ Container gpt-researcher-gpt-researcher-1  Created                                                                                0.1s
Attaching to gpt-researcher-gpt-researcher-1
gpt-researcher-gpt-researcher-1  | INFO:     Started server process [1]
gpt-researcher-gpt-researcher-1  | INFO:     Waiting for application startup.
gpt-researcher-gpt-researcher-1  | INFO:     Application startup complete.
gpt-researcher-gpt-researcher-1  | INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
gpt-researcher-gpt-researcher-1  | INFO:     <IP_ADDRESS>:59976 - "GET / HTTP/1.1" 200 OK
gpt-researcher-gpt-researcher-1  | INFO:     ('<IP_ADDRESS>', 59978) - "WebSocket /ws" [accepted]
gpt-researcher-gpt-researcher-1  | ERROR:    Exception in ASGI application
gpt-researcher-gpt-researcher-1  | Traceback (most recent call last):
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/websockets/wsproto_impl.py", line 234, in run_asgi
gpt-researcher-gpt-researcher-1  |     result = await self.app(self.scope, self.receive, self.send)
gpt-researcher-gpt-researcher-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
gpt-researcher-gpt-researcher-1  |     return await self.app(scope, receive, send)
gpt-researcher-gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 292, in __call__
gpt-researcher-gpt-researcher-1  |     await super().__call__(scope, receive, send)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
gpt-researcher-gpt-researcher-1  |     await self.middleware_stack(scope, receive, send)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 149, in __call__
gpt-researcher-gpt-researcher-1  |     await self.app(scope, receive, send)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
gpt-researcher-gpt-researcher-1  |     raise exc
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
gpt-researcher-gpt-researcher-1  |     await self.app(scope, receive, sender)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
gpt-researcher-gpt-researcher-1  |     raise e
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
gpt-researcher-gpt-researcher-1  |     await self.app(scope, receive, send)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
gpt-researcher-gpt-researcher-1  |     await route.handle(scope, receive, send)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 341, in handle
gpt-researcher-gpt-researcher-1  |     await self.app(scope, receive, send)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 82, in app
gpt-researcher-gpt-researcher-1  |     await func(session)
gpt-researcher-gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 324, in app
gpt-researcher-gpt-researcher-1  |     await dependant.call(**values)
gpt-researcher-gpt-researcher-1  |   File "/usr/src/app/main.py", line 60, in websocket_endpoint
gpt-researcher-gpt-researcher-1  |     await manager.start_streaming(task, report_type, agent, agent_role_prompt, websocket)
gpt-researcher-gpt-researcher-1  |   File "/usr/src/app/agent/run.py", line 38, in start_streaming
gpt-researcher-gpt-researcher-1  |     report, path = await run_agent(task, report_type, agent, agent_role_prompt, websocket)
gpt-researcher-gpt-researcher-1  |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-gpt-researcher-1  |   File "/usr/src/app/agent/run.py", line 50, in run_agent
gpt-researcher-gpt-researcher-1  |     await assistant.conduct_research()
gpt-researcher-gpt-researcher-1  |   File "/usr/src/app/agent/research_agent.py", line 143, in conduct_research
gpt-researcher-gpt-researcher-1  |     research_result = await self.run_search_summary(query)
gpt-researcher-gpt-researcher-1  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-gpt-researcher-1  |   File "/usr/src/app/agent/research_agent.py", line 128, in run_search_summary
gpt-researcher-gpt-researcher-1  |     os.makedirs(os.path.dirname(f"./outputs/{self.directory_name}/research-{query}.txt"), exist_ok=True)
gpt-researcher-gpt-researcher-1  |   File "<frozen os>", line 225, in makedirs
gpt-researcher-gpt-researcher-1  | PermissionError: [Errno 13] Permission denied: './outputs/cb77dbe5-ad40-4a22-8ef0-cec2464b4fbb'
Gracefully stopping... (press Ctrl+C again to force)
Aborting on container exit...
[+] Stopping 0/1
[+] Stopping 1/1-researcher-gpt-researcher-1  Stopping                                                                                1.0s
[+] Killing 1/1t-researcher-gpt-researcher-1  Killing                                                                                0.8s
 ✔ Container gpt-researcher-gpt-researcher-1  Killed                                                                                0.8s
time="2023-09-07T17:22:30+02:00" level=error msg="got 3 SIGTERM/SIGINTs, forcing shutdown"

$ git log | head -n 10
commit b89fdaa0aee4e710af594cd82ca88792d1b88e0a
Author: Assaf Elovic<EMAIL_ADDRESS>Date:   Thu Sep 7 15:54:20 2023 +0300

    Update README.md

    Removed troubleshooting issues from main README and instead added guidance.

commit 9083223e79fed2201e0ea6313f533f7dc765376e
Merge: 85821b1 0f9f962

$

This seems like a local permissions issue. Or in other words an environment problem on your end. Closing for now or until further opened from others.

I am running GPT researcher with the below pip install
pip install gpt-researcher
and the below python code
from gpt_researcher import GPTResearcher
query = "why is the latest research on Tabular GANs and what are the scopes on improvements on tabular GANs?"
researcher = GPTResearcher(query=query, report_type="research_report")
Conduct research on the given query
research_result = await researcher.conduct_research()
Write the report
report = await researcher.write_report()
Getting this error :-
Traceback (most recent call last):
File c:\Users\dipgh\anaconda3\lib\site-packages\IPython\core\interactiveshell.py:3369 in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
Input   In [2] in <cell line: 2>
from gpt_researcher import GPTResearcher
File c:\Users\dipgh\anaconda3\lib\site-packages\gpt_researcher_init_.py:1 in
from .master import GPTResearcher
File c:\Users\dipgh\anaconda3\lib\site-packages\gpt_researcher\master_init_.py:1 in
from .agent import GPTResearcher
File c:\Users\dipgh\anaconda3\lib\site-packages\gpt_researcher\master\agent.py:7 in
from gpt_researcher.master.functions import *
File c:\Users\dipgh\anaconda3\lib\site-packages\gpt_researcher\master\functions.py:21
match retriever:
^
SyntaxError: invalid syntax
Getting the same error while running the command uvicorn main:app --reload after git cloning , creating new environment with necessary packages and populating the necessary env variables for the OpenAI and Tivali API keys.
Can someone pls help?


mon: upgrade calamari when running rolling_update playbook

Prior to this change, ansible was only checking for the existence of the
package, now if upgrade_ceph_packages is true this means we are
performing an upgrade.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1442016
Signed-off-by: Sébastien Han<EMAIL_ADDRESS>

jenkins test kraken-ansible2.2-centos7_cluster

@andrewschoen Would you please take a look at this?

Where is the stable-2.2 backport for this change?

@ktdreyer here: https://github.com/ceph/ceph-ansible/pull/1470


[tooltip][material-ui] hide tooltip on scroll

[x] I have followed (at least) the PR section of the contributing guide.

solves this issue: https://github.com/mui/material-ui/issues/38379

I attempted to address issue #38379 earlier, but unfortunately, I couldn't find a solution. As this pull request is now inactive, let's close it to give others a chance to address the issue.


Upload-only shares must not overwrite but create a separate file

curl -D - 'https://localhost:9200/remote.php/webdav/empty%20file' \
  -X 'PUT' \
  -H 'Content-Length: 0' \
  --insecure \
  -u einstein:relativity
HTTP/1.1 204 No Content

Discovered while running tests/acceptance/features/apiSharePublicLink1/changingPublicLinkShare.feature:171

Discussion in REVA https://github.com/cs3org/reva/issues/2480


232234

cache example

Codecov Report

Merging #1452 (b6f6ab1) into master (7a6fb4a) will decrease coverage by 0.10%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1452      +/-   ##
==========================================
- Coverage   69.75%   69.65%   -0.11%
==========================================
  Files         402      403       +1
  Lines       38842    38899      +57
==========================================
  Hits        27094    27094
- Misses      10005    10062      +57
  Partials     1743     1743

Flag
Coverage Δ

go-1.14
69.60% <0.00%> (-0.09%)
:arrow_down:

go-1.15
69.63% <0.00%> (-0.08%)
:arrow_down:

go-1.16
69.63% <0.00%> (-0.10%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

os/gcache/gcache_z_example_adapter_redis.go
0.00% <0.00%> (ø)

net/gtcp/gtcp_pool.go
34.09% <0.00%> (-4.55%)
:arrow_down:

os/gcache/gcache_adapter_memory_data.go
94.48% <0.00%> (+0.78%)
:arrow_up:

os/gcache/gcache_adapter_memory.go
81.81% <0.00%> (+1.18%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7a6fb4a...b6f6ab1. Read the comment docs.

@564104865 少了gcache包方法的Example。


Extra space displayed in test output for array values

When a test fails for an array value, the actual and expected value have a leading space that isn't there for other data types:
no primes under two  ... FAIL
    Actual:    « [1]»
    Expected:  « []»

Compared to:
1 is decimal 1  ... FAIL
    Actual:    «1»
    Expected:  «2»

@axtens is this something you can fix?

Ah now this makes more sense. Is this the same as the other issue I've been responding to?

Have asked on the 8th forum: https://8th-dev.com/forum/index.php?board=28.0

Got an answer on that: stringifying an array adds a leading space. I'll fix the lib sometime this week

Related to? https://github.com/exercism/8th/issues/134

Discussed on the forum at https://forum.exercism.org/t/leading-space-on-array-output-in-8th/10779

Being addressed by https://github.com/exercism/8th/pull/135


move host to (0,0)

Move host io to origin coordinate and remove the bladerunner rom in NPA address.
This comes with bsg_manycore PR #30

Did this pass in Cosim?

It passes all except for the test_manycore_credits which is addressed in another PR.
Message is as follow:
BSG INFO: test_manycore_credits Regression Test (COSIMULATION)
ERROR:   default_eva_to_epa_tile: Invalid EVA Address 0x0. Does not map to an addressible tile memory locatiion.
ERROR:   hb_mc_manycore_eva_write: Error, requested copy to region that is smaller than buffer
ERROR:   test_manycore_credits: failed to write buffer to EVA 0x0 of coord (1,1)
BSG REGRESSION TEST FAILED
BSG COSIM FAIL: Test failed with exit code:          -1
0
$finish called at time : 7740 ns : File "/home/xlelephant/lx_git_repos/bsg_f1/cl_manycore/testbenches/cosim/cosim_wrapper.sv" Line 37
exit

Is there a corresponding change in bsg_manycore that moves the host interface to (0,0)?

It passed the simulation, But some test cases will send finish packet to coordinate (0,3):
remote_load_crc
bsg_transpose
fib
crc32
quicksort
nprimes
depend_stall_mispredict
I believe that some separate parameters are not set correctly in those tests. But I did not find them.

Did you run make clean? As far as I can tell, all of these call bsg_finish, which uses IO_X_INDEX. You've changed that definition.


crunchyroll-dl Web

Do you want to try create a web app for crunchyroll-dl? i already created the frontend design: https://louanbastos.github.io/crunchyroll-dl-en/
email me<EMAIL_ADDRESS>

Hmm, while this does seem possible, I'd rather not be involved with a website... I'd rather keep such tools locally, as I feel Crunchyroll would be less lenient with a website, compared to the multitude of tools available to download from their website on GitHub.

Hmm, while this does seem possible, I'd rather not be involved with a website... I'd rather keep such tools locally, as I feel Crunchyroll would be less lenient with a website, compared to the multitude of tools available to download from their website on GitHub.

so just create for pc only, but with User Interface.

If you're planning on making this accessible over a website, you'd probably need a premium account fetching the links. I see there was another site on that page you linked - crunchydl - doesn't that use a premium account in the backend as well?

I have a premium login, and the site don't use, my website https://louanbastos.github.io/crunchyroll-dl-en/ is a just frontend


Add an is_tuple_like trait for sequenceable type detection

Requires #2685, otherwise the unit test won't pass

Please add the new #include here: https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/include/traits.hpp

I fixed the code formatting and added the trait to the include compilation.
@hkaiser thank you for sending me the .clang-format file, it seems to be much better than the one I had.

 Please merge again, this should resolve the issue.

@hkaiser I rebased the PR accordingly, but the CI is still failing


Execute inline command?

Not sure how to describe it, so subject might be incorrect.
Basically what I'm thinking is somewhat like below
> cash ls
execute ls, then exit to original shell
>

> cash cp src dest
copy...
>

let allow invoke specific command. Is there way without installing as global viacash-global?

ah. There isn't support for that right now. But we can definitely add it!

@nfischer what do you think - if multiple params are passed to process.argv in Cash, exit the interactive shell, as opposed to staying in it?

I think the argument should be treated as file names, and then be executed. This is consistent with the behavior of bash.
It might be possible to make another utility for this purpose. Over at ShellJS, we started https://github.com/shelljs/shx for this kind of use-case. This could be used like:
$ shx ls # use ShellJS's ls command, but right on your command line
$ shx pwd
$ shx grep 'text' file.txt

Shx isn't released to npm yet, but you can expect a release in a week or two (once we release ShellJS v0.7). If something similar is a priority for cash, I would recommend releasing it as another package, since it lets cash filename.sh work very similarly to bash filename.sh.

I'm probably going to check for a filename, and if a filename is not found, check for a valid cash command, and if that doesn't exist, fall back to the bash file not found error.

That approach raises questions about what to do when multiple parameters are passed. If multiple parameters go to bash, it runs a script with the rest as arguments ($1, $2, etc) to the script.
If someone tries cash rm ls what would that do?

It would check for rm as a bash file. If it finds it, it will pass ls in as the first command.
If it doesn't, it will run rm ls into the Cash interpreter.

I'm not a fan of this approach, since I'm not aware of any precedent for doing it. I thought that's what the individual packages were meant to solve.
I think it should always execute things as scripts, or if the file can't be found, fail. That's most consistent with bash.

Okay fine, no biggie. Closing this issue then.


Install phono3py on Mac with Intel ICC and MKL

Dear Professor Togo,
I'm confused on MULTITHREADED_BLAS C macro after reading the installation instruction on phono3py website and your comment in setup.py.  Does setting MULTITHREADED_BLAS in setup.py turn on or off multithreading for inner loop calls of BLAS? My current understanding is the following setting in setup.py turns on multithread BLAS (I'm using MKL)? Is that right? Thank you.
`
if os.path.isfile("mkl.py"):
# This supposes that MKL multithread BLAS is used.
# This is invoked when mkl.py exists on the current directory.
print("MKL LAPACKE is to be used.")
print("Use of icc is assumed (CC='icc').")

from mkl import mkl_extra_link_args_lapacke, mkl_include_dirs_lapacke

extra_link_args_lapacke += mkl_extra_link_args_lapacke
include_dirs_lapacke += mkl_include_dirs_lapacke

if use_setuptools:
    extra_compile_args += ['-DMKL_LAPACKE',
                           '-DMULTITHREADED_BLAS']
else:
    define_macros += [('MKL_LAPACKE', None),
                      ('MULTITHREADED_BLAS', None)]

`
Best regards,
Wei

MULTITHREADED_BLAS is used only in c/harmonic/phonon.c.  If MULTITHREADED_BLAS is defined, OpenMP is not used for the loop over phonon modes because LAPACK is used inside this loop. If you use multithreaded BLAS in MKL (as far as I know there is non-multithreaded version in MKL), you should set -DMULTITHREADED_BLAS.

Dear Professor Togo,
Thank you for the clarification. This is now crystal clear to me.
Best
Wei

These compilation stuff is super bothering for users. If you are allowed to use CONDA in your computing system and it is a usual x86-64 system, I recommend simply using it, which has no disadvantage in phono3py when using CONDA's MKL linked numpy and scipy (these are installed as default, see more detail https://docs.anaconda.com/mkl-optimizations/). After installing miniconda, the installation of phono3py is done as follows:
% conda create --name phono3py
% source activate phono3py
% conda install -c atztogo phono3py

That's all.

Thank you again, Professor Togo. I was trying on Mac and there conda install -c atztogo did not work, because your channel only has phono3py for Linux.
'$ conda install -c atztogo phono3py
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:

phono3py'

Unfortunately, phono3py conda package for mac is not prepared. You need to build phono3py. But MKL linked numpy and scipy exist. Therefore something like following works (for me):
% conda create --name phono3py
% source activate phono3py
% conda install numpy scipy pyyaml matplotlib
% conda install -c conda-forge phonopy h5py openblas
% conda install clang_osx-64 llvm-openmp
% git clone https://github.com/atztogo/phono3py.git
% cd phono3py
% export CC=clang
% python setup.py build
% pip install -e .

phonopy conda package exists in conda-forge channel. It is strongly recommended to install h5py and openblas from the conda-forge channel at this moment.  Some package installed by dependency of openblas for mac from anaconda channel seems broken at this moment (I doubt libgfortran). I also doubt h5py related package in anaconda channel for mac. For linux no problem.

These details instructions are again, really helpful. Thank you, Professor Togo. Indeed I installed phonopy from conda-forge channel and compiled phono3py using icc and MKL (standalone package, not the one from conda, which is free for academic use). I found I need to insert the following to after line 21:
if 'icc' in os.environ['CC']:
        cc = 'icc'

Also below is my mkl.py (note the difference with mkl.py for linux is that no /intel64 after /lib is necessary):
intel_root = "/opt/intel/compilers_and_libraries/mac"
mkl_root = "%s/mkl" % intel_root
compiler_root = "%s" % intel_root

mkl_extra_link_args_lapacke = ['-L%s/lib' % mkl_root,
                               '-lmkl_rt']
mkl_extra_link_args_lapacke += ['-L%s/lib' % compiler_root,
                                '-lsvml',
                                '-liomp5',
                                '-limf',
                                '-lpthread']
mkl_include_dirs_lapacke = ["%s/include" % mkl_root]

With them, I got phono3py working on Mac like a charm. Thank you a lot for your time in helping me out to get it working.

OK! Thanks for your info. This issue became a good log for people struggling to build phono3py.

Thanks. I changed the issue title to make it easier for people to find.

Hi,
Can anyone suggest me, How to install phonopy-spectroscopy in conda environment.
Thanks&Regards
Suneetha N

Please send your question about phonopy/phono3py installtion to the phonopy mailing list. For phonopy-spectroscopy, please contact somebody at https://github.com/skelton-group/Phonopy-Spectroscopy.

Sure sir,
Thank you for your suggestion.
On Thu, Apr 27, 2023 at 4:37 AM Atsushi Togo @.***>
wrote:

Please send your question about phonopy/phono3py installtion to the
phonopy mailing list. For phonopy-spectroscopy, please contact somebody at
https://github.com/skelton-group/Phonopy-Spectroscopy.
—
Reply to this email directly, view it on GitHub
https://github.com/phonopy/phono3py/issues/11#issuecomment-1524150994,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOK2WIMXNISKYQLKEPBZKS3XDGTBZANCNFSM4F3LNW6Q
.
You are receiving this because you commented.Message ID:
@.***>


reload_at config variable ignored

Hi there,
Doing some testing of this on Cisco ME3400 devices as a method to mass upgrade.
We're noticing that the 'reload_at' setting we have in config.yaml is being ignored. We changed the default 00:00 to 06:00, but it still uses 00:00. Other things from our config.yaml are being accepted.
The reload_* portion of config.yaml looks like this (default commented out):
reload_in: false
#reload_in: 10
reload_at: '06:00'

I've tested this on a handful of devices, all me3400, always with the same results. Config syntax above perhaps?

Give the latest commit a shot.
Does the code work otherwise on ME3400s?

Yep that did it. And yes seems to work, have tested where the image was already installed from archive (so it was a subdir) and where there was a .bin in the root.
We have scp enabled on them, which also works even though the release notes/instructions imply it shouldn't.  Times are pretty much all over the place, here's some samples:
Total time elapsed: 0:17:41.948463
Total time elapsed: 0:22:26.918263
Total time elapsed: 0:26:18.993592
Total time elapsed: 0:31:05.640006
Total time elapsed: 0:46:19.443902

Additional comment. Our testing so far is just using this inline from the python3 cli. If we use it in production, it will be called by some code.
Is there a way to set reload_at (or perhaps any other config value) manually in the code to override the config? For example, if we want to upgrade 100 devices we almost certainly don't want to reload them all at the same time.

Pushed another update that allows you to disable reloads by setting both reload_in and reload_at to false in config.yaml.
Then upgrade() will do everything except schedule reload. You can manually call schedule_reload() afterward, specifying either reload_at or reload_in with whatever value you want at runtime.
Later I may add a stagger_reload option that randomizes reloads across a given interval.

I added a reload_range option to the latest push.

Sorry to poke on resolved ticket, but in issue 8 I was testing some stuff, and noticed i'm having this same issue again. That testing is in the dev branch, happen to know if this fix happened there?
options i have set (range is set low as a test):
 'reload_at': False,
 'reload_in': 0,
 'reload_range': 5,

.. but the device is scheduled to load just after midnight:
Reload scheduled for 00:01:00 CDT Thu May 28 2020 (in 10 hours and 43 minutes) by netconf
If I'm understanding properly, the settings i have, and desire, is to do an immediate reload.

Latest dev commit includes several bugfixes in schedule_reload(). Give it a shot.


UAADB revocable_tokens table is filled even if uaa.jwt.revocable is set to "false" in the YML file

My team is running a huge CF deployment and we noticed that event if the we have uaa.jwt.revocable set to false in the bosh YML file the UAADB table revocable_tokens  is filled with new records.
This was not happening in the previous version we were running (CF 235).
It seems to me that issue is in this class Link
which doesn't properly skip token persistence if the uaa.jwt.revocable is false.
This is a problem for us (on DBA side) because revocable_tokens  table contains more than 3M records.

If refresh tokens are requested, they are delivered in a revocable format, always.
There was a bug discovered where we only clean the table if the refresh token is being refresh.
With https://github.com/cloudfoundry/uaa/commit/51780c3c63fc98636f71d778c4fb8a4218ad3a82#diff-fbe33c1f3c5844b8f9482f55eaa64845 we also clean the table when we store tokens

@aalbanes  see #466 for additional diagnostics

@gberche-orange Guillaume thx for reference.
@fhanik  Filip, as far as I recall in the CF release context, CC are not going to validate if a refresh token expired or not but it it check for expiration field within the JWT itself.
It this still the case ? If so.. which is the reason why we cannot enforce a refresh token to  not be revocable.

@aalbanes I think you can set the following to true
https://github.com/cloudfoundry/uaa-release/blob/develop/jobs/uaa/spec#L362-L364
I don't think the feature was meant to behave that way, but looking at the code, it will not issue refresh tokens if this value is set to true and the client doesn't have the uaa.offline_token scope.

@fhanik not having a refresh is not a solution. I think what we really need to fix is that refresh tokens are there, BUT it is configurable whether it is revocable. The current design doesn't really work at scale.
FWIW, OpenStack had the same issues some years ago b/c they were storing too many tokens and then faced issues at scale.
@sreetummidi let me know if this change will be possible or if you want to discuss this further on a PM level

@fhanik @sreetummidi can you comment please ?

@smoser-ibm Correct, didn't suggest it as a solution, but a work around, until a solution is  in place :)
This issue will be open until a solution is in place.

@smoser-ibm We will resurrect this feature for UAA 4.0
https://www.pivotaltracker.com/story/show/138155269

@sreetummidi - why is the Issue closed now?  Has this actually been resolved?  Neither of the related tracker items:
https://www.pivotaltracker.com/n/projects/997278/stories/137028417
https://www.pivotaltracker.com/n/projects/997278/stories/138155269
appear to have been started, so confused why this Issue was closed?

@ScarletTanager Yes, issue has been resolved. And released.
https://github.com/cloudfoundry/uaa-release/releases/tag/v27


Include build dependencies in cargo install section

Additionally remove a misplaced "use" in front of the comparison table and align the code spaces within the table to enhance readability of the code (no effect on visual Markdown).
Since I just compiled it, running into 4 failures due to missing build dependencies 😉. I'd btw agree if one either wants other OSes/distros listed as well or have the dependencies listed in a more generic way, like "Rust toolchain, C compiler, libc, libssl and pkg_config" or so. So take this more as a suggestion to list build dependencies somewhere in some way, which I wanted to do in a constructive way with the copy&paste ready commands the the type or distros I use. From this, everyone should be able to derive the respective packages for the own distro, e.g. on RPM based ones it's glibc-devel and openssl-devel or so.

Makes sense. Just made a small adjustment to the way "dpkg" is spelled. On the manpage it is spelled in all lowercase letters -- even at the beginning of a sentence; so I'll stick to that convention. Other than that looks good. Thanks!

Just made a small adjustment to the way "dpkg" is spelled.

Agreed, also Wikipedia agrees: https://en.wikipedia.org/wiki/Dpkg
And the Debian Wiki; https://wiki.debian.org/dpkg
Not sure where I read "DPkg" quite regularly 😄.


Incorporate a GPU-accelerated version of Scikit-Image’s “RANSAC” into RAPIDS cuCIM library

Is your feature request related to a problem? Please describe.
One of the bottlenecks in my end-to-end Geospatial application for rapidly geolocating aerial imagery is the ability to perform image similarity quickly. Furthermore, RANSAC is critical to removing false matches by removing poor SIFT feature matches when comparing the Aerial (query) image to a Satellite (reference) sub-image.
Here is a visual depiction of applying RANSAC to improve SIFT feature matching:
RANSAC Improves SIFT Feature Matching - One Slide.pptx
So I wish cuCIM had a GPU-accelerated version of RANSAC versus my current usage of the Scikit-Image implementation. This would enable me to have another stage of my application that keeps the processing on the GPU device versus having to return from processing on the device to processing on the host's CPU.
Describe the solution you'd like
A successful GPU-accelerated version of Scikit-Image's "RANSAC" algorithm.
Describe alternatives you've considered
I haven't considered any alternative solutions, and I haven't been able to find any GPU-accelerated versions of RANSAC.
Additional context
I don't have any other context, code examples, but here is the URL to the Scikit-Image Github repo for RANSAC listed here: https://github.com/scikit-image/scikit-image/blob/v0.19.2/skimage/measure/fit.py#L628-L898

cc @gigony @grlee77 (as we discussed this today)

Thanks for raising the issue @kkgreen.
For scikit-image the ransac function can take a user-defined model. The ones provided by the library are CircleModel, LineModelND and EllipseModel.
I am trying to judge how feasible accelerating these on the GPU is. What would be a typical size N for the data input to ransac in your use case?

Evening Gregory,
Here is the command that I am using for RANSAC:
Import Scikit-image's RANSAC and various Transforms
from skimage.transform import AffineTransform, ProjectiveTransform
from skimage.measure import ransac
model_robust, inliers = ransac((src, dst), AffineTransform,
min_samples=MIN_MATCH_COUNT, residual_threshold=2, max_trials=300)
So I am not using any of those models that you mentioned.
On Wed, Nov 16, 2022 at 3:32 PM Gregory Lee @.***>
wrote:

Thanks for raising the issue @KKGreen https://github.com/KKGreen.
For scikit-image the ransac function can take a user-defined model. The
ones provided by the library are CircleModel, LineModelND and EllipseModel
.
I am trying to judge how feasible accelerating these on the GPU is. What
would be a typical size N for the data input to ransac in your use case?
—
Reply to this email directly, view it on GitHub
https://github.com/rapidsai/cucim/issues/435#issuecomment-1317632611,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJQHYVOKOMWZMVHLSYZXX3WIVAGJANCNFSM6AAAAAASBMRIKQ
.
You are receiving this because you authored the thread.Message ID:
@.***>


Generate pipeline is not loading in v2.0.4

Using the version v2.0.4 of this plugin, the generated build is output into console but not load
~~~ Running plugin monorepo-diff command hook
[2021-05-06T23:25:25Z] $ /var/lib/buildkite-agent/plugins/github-com-chronotc-monorepo-diff-buildkite-plugin-v2-0-4/hooks/command
[2021-05-06T23:25:25Z] https://github.com/chronotc/monorepo-diff-buildkite-plugin/releases/download/v2.0.4/monorepo-diff-buildkite-plugin-linux
[2021-05-06T23:25:25Z]   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
[2021-05-06T23:25:25Z]                                  Dload  Upload   Total   Spent    Left  Speed
[2021-05-06T23:25:25Z]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   644  100   644    0     0   3833      0 --:--:-- --:--:-- --:--:--  3856
[2021-05-06T23:25:26Z]
100 3302k  100 3302k    0     0  5449k      0 --:--:-- --:--:-- --:--:-- 5449k
[2021-05-06T23:25:26Z] INFO[0000] --- :one: monorepo-diff v2.0.4
[2021-05-06T23:25:26Z] INFO[2021-05-06T23:25:26Z] Running diff command: .buildkite/diff.bash
[2021-05-06T23:25:26Z] Generated Pipeline:
[2021-05-06T23:25:26Z] steps:
[2021-05-06T23:25:26Z] - trigger: webhooks
[2021-05-06T23:25:26Z]   build:
[2021-05-06T23:25:26Z]     message: Why no work
[2021-05-06T23:25:26Z]     branch: API-871
[2021-05-06T23:25:26Z]     commit: 3fa1531e2cfd976584ee5ce8bc3dde83ae2023b9
[2021-05-06T23:25:26Z] - {}
[2021-05-06T23:25:26Z] - wait

in the timeline of the step, it says it uploaded the steps:

however, none of the steps are uploaded or run. when I switch back to version v1.3.1 of this plugin, everything works as expected with no other changes.

Could you share your build config

Closing this as there has been no response.


Set more windowlike defaults

I changed the default scale and dimensions (in mm) to be more like a real window.

Thanks!


update to latest chefdk and ubuntu 16.04

Hah! Great minds think a like. Btw, now I need to fix the rubocop styling issues. Unfortunately, no access to a computer until next week.

Woo! Green!  Smartphones FTW


feat: cerberus recovery

Description
Add support for recovery.
If foreing leader fails to send the node. The pacemaker should trigger.
When it triggers the node does following

Check if we fail to get proposal from local leader (this is currently TODO)
If locals are fine. The we check for which committee we don't have a newest node. And we choose some subcommiitee of size (f+1+some epsilon value). And we send the VCGlobalSCR message to these node.
Node receives VCGlobalSCR message from someone, with shard_id. We check if we have some newer leader proposal, if we do, send it back. If we don't ignore (we are probably going to be triggered as well, because our leader failed).

How Has This Been Tested?
Not yet.

I think the point is that you can stop asking after receiving a valid  QC
On Wed, 25 Jan 2023 at 15:09, stringhandler @.***>
wrote:

@.**** commented on this pull request.
In dan_layer/core/src/workers/hotstuff_waiter.rs
https://github.com/tari-project/tari-dan/pull/311#discussion_r1086762301
:

@@ -74,6 +75,9 @@ use crate::{
};

const LOG_TARGET: &str = "tari::dan_layer::hotstuff_waiter";
+// When the foreign committee fails to provide QC, we ask f+1+EPSILON nodes for the QC (f+1 should be enough, but we
+// want to make sure, and we don't want to ask the whole committee).
You have to ask f + 1 to make sure you get an honest node. If you ask less
than f, they could say they haven't gotten it
—
Reply to this email directly, view it on GitHub
https://github.com/tari-project/tari-dan/pull/311#discussion_r1086762301,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABZZALYNFTC5FCD6KQXEZ5DWUE64LANCNFSM6AAAAAAUF6YEI4
.
You are receiving this because you commented.Message ID:
@.***>

--
Cayle Sharrock
Head of Engineering
Tari Labs
https://www.tari.com

Amazing! well done!

Thanks. There are still some todos, but they are not required to run this. They are just some improvements suggestions.


fix(nav): check for relative path

@yufeih could you build the dist folder and push when you have a moment? If not, these changes are not applied to the end result.
Thanks!


security checks, file access rights and error handling fixed

Security vulnerabilities fixed.
File access right changed.
Error handling added on file validation check.

gosec version may not be the updated one i believe that is the reason why it fails on security stage

@ReneWerner87 these changes are needed for #853


Support NetworkX 2

Currently Lightflow doesn't support the latest version of NetworkX.

maybe this might help? #16

Merged your PR and released version 1.11.0


Adding LOJ-1101 A secret Mission

Solved but could not manage time to write tutorial. Hope someone else do this if not i will be back soon


Automated cherry pick of #8502

Cherry pick of #8502 on release-5.38.
/cc  @amyblais

/release-note-none


fix: skip acl check when group mode is 0

close #4512

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.jiefeng seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

need to fix this?


Add missing functionalities to support API

Apple has left as with a table of supported endpoint in Xcode 7. It shows up that they're only providing some basic stuff but it's necessary for us to cover them all!
We're on a good way and already cover most but some of them are missing - not because we don't have code to support them but because a nice, convenience method hasn't beed written yet (you can do it).
In README you can see a table of features, so if you're planning to implement some of those missing methods, plase update README as well

Here's the list of all official endpoints:

[x] List bots on server
[x] Create a new bot
[x] Retrieve a bot by ID
[ ] Update a bot’s configuration (will be tricky, on hold till #44 is merged)
[x] Get the most recent integrations for a bot
[x] Enqueue a new integration
[x] List integrations on server
[x] Retrieve an integration by ID
[ ] List the commits included in an integration
[ ] List the build issues produced by an integration
[x] List devices connected to server
[ ] List hosted repositories on server
[ ] Create a new hosted repository

Should be an easy one for anyone with 20mins on their hands.
-- @czechboy0 in #39

If you're wondering how to determine what endpoint will be used in official API, here's the list:

Type
Path
Description

GET
/bots
List bots on server

POST
/bots
Create a new bot

GET
/bots/(id)
Retrieve a bot by ID

PATCH
/bots/(id)
Update a bot’s configuration

GET
/bots/(id)/integrations
Get the most recent integrations for a bot

POST
/bots/(id)/integrations
Enqueue a new integration

GET
/integrations
List integrations on server

GET
/integrations/(id)
Retrieve an integration by ID

GET
/integrations/(id)/commits
List the commits included in an integration

GET
/integrations/(id)/issues
List the build issues produced by an integration

GET
/devices
List devices connected to server

GET
/repositories
List hosted repositories on server

POST
/repositories
Create a new hosted repository

If you need some motivation to start doing this task, please refer to #2 😉

:+1:

I think I'll cover this if only I get OS X Server running on my MBP 😡

@czechboy0 I'm trying to update XcodeServerSDK to support missing API calls but I'm stuck on the first one... 😁
Update a bot’s configuration - how do you imagine the method signature for this one? I came with two ideas but don't know which one will be better:

public func updateBot(botId: String, withBot bot: Bot, overwriteBlueprint blueprint: Bool = true, completion: (success: Bool, error: NSError?) -> ())
public func updateBot(bot: Bot, overwriteBlueprint blueprint: Bool = true, completion: (success: Bool, error: NSError?) -> ())

In the first option you specify both: bot to update by ID and updating bot. In the latter you just provide updated bot with correct ID which will be used to PATCH existing bot. Which one do you prefer more?

Hmm this one is actually pretty tricky. I suggest we leave that one for last :laughing:

Also because some of the work I'm doing now will make the bot updating call much easier to work with. If you can take a look at List the build issues produced by an integration, List the commits included in an integration, Retrieve an integration by ID etc that'd be most useful.

Typical me - start working from the worst side 😐
Roger that, I will go for other things and leave this for later.
Thing that get stuck in my head after opening XcodeServer class - why not move those calls to separate files as extensions eg. XcodeServer+Bot, XcodeServer+Integrations. This way we'll only have generic sendRequestWithMethod() in XcodeServer class and get rid specific API calls. But I guess, we'll have to wait with this until your PR as this'll some introduce merge conflicts...

Yeah I agree, now that there are more API calls it'd make sense to start splitting it. As you said, let's wait for that until this is merged (and even create an issue so that we don't forget).

Retrieve an integration by ID covered in #48.

@czechboy0 you know anything about integration commits? I have configured bot and if I run integration I always see that there are no commits related to this integration. Any idea when those commits are assigned to the integration?

Yeah that's going to be a tricky one to test. The commits of an integration are the commits that have been pushed since the last integration. So I'd suggest you create a dummy branch, run an integration there once. Then, commit a couple of times there and then run an integration again. The second one should show all those commits. However, for debugging, you can also delete integrations (not just bots) through Xcode. This might help in development.

You're probably familiar with the output from /integrations/(id)/commits. What do you think should be returned to the end user? Only an array of commits? Because there are many, many other things...

Hmm, I guess I found answer... We're returning an array of commiters... But we probably don't have models for this, right?

Nope, that's all yours :wink: Create model classes for Commits and committers and for whatever else makes sense.

Looks like it's a job for more than 20 min 😜

Yeah that might have been an underestimate :laughing:

So if #44 is already merged, are we good to go with Updating a bot’s configuration? I'd prefer to finish non-new-stuff-required calls first and than jump on those more involving. Please, think about the method signature and let me know 😉

Actually, PATCHing an existing bot is a bit more involved. If I were you I'd leave that for last. It's not that common of a call anyway (I never had it in Buildasaur). But I'll take a look at it next week some time.

Ok, as you wish 👌

Repositories are represented by a simple JSON:
{
      "readAccessExternalIDs": [],
      "writeAccessExternalIDs": [
        "FDF283F5-B9C3-4B43-9000-EF6A54934D4E",
        "ABCDEFAB-CDEF-ABCD-EFAB-CDEF00000050"
      ],
      "name": "Test2",
      "posixPermissions": 0,
      "httpAccessType": 1
}

There's no similarity between Repository and XcodeServerEntity. So, my question is, should we anyway create it as XcodeServerEntity child or maybe create a simple struct?

I'll leave that to your judgement. I don't have a strong preference either way.

Yay only 3 more API calls to support them all! The last two should be relatively easy, it's just work adding those model classes. And the Bot patching one I'll try to look at in the next couple of days.

One last to go...

@czechboy0 what is the status of Update a bot’s configuration (will be tricky)?
Why is it tricky? And how tricky is it? this would be a great feature! since checking out code and building a project takes over 90% for each bot.  Running the unit tests takes less than 10%, that is a big overhead!
I have btw created a Swift 3 branch for XcodeServerSDK and each pod it is dependent on: https://github.com/Sajjon/XcodeServerSDK
Should I create a PR maybe?

@cojoj If I would try developing a Edit existing Bot feature, any ideas where to start? What are the primary challenges?

Whoa, it was a long time ago... And to be clear:

How I see it being tricky... You have to provide correct things to update bot. I guess the only required thing would be id, but I may be wrong (this needs to be checked). Also, updating sometimes can involve this multi stepping, so maybe some sort of builder for Bot.


[SPARK-18271][SQL]hash udf in HiveSessionCatalog.hiveFunctions is redundant

What changes were proposed in this pull request?
when lookupfunction in HiveSessionCatalog, first look up in spark's build-in functionRegistry, if it not existed, then look up the Hive'S build-in functions which are listed in Seq(hiveFunctions).
But the [hash] function is already in spark's build-in functionRegistry list, so it will never to go to look up in Hive's hiveFunctions list, so the [hash] function which is in Hive's hiveFunctions is redundant, we can remove it.

Can one of the admins verify this patch?

ok to test

LGTM

Test build #68138 has started for PR 15766 at commit c3a0389.

Test build #68138 has finished for PR 15766 at commit c3a0389.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68138/
Test FAILed.

retest this please

Test build #68152 has started for PR 15766 at commit c3a0389.

Test build #68152 has finished for PR 15766 at commit c3a0389.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68152/
Test FAILed.

Merged build finished. Test FAILed.

Don't we actually unregister this when testing Hive?

@rxin  good catch! We do unregister the spark builtin hash in test: https://github.com/apache/spark/blob/master/sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala#L60-L61
So we have a little more work to do here, we should register the hive hash function in that test suite.

I am working on the related issue in https://github.com/apache/spark/pull/14498

@rxin  @cloud-fan you are rigth,hash should be unregistered and replace with Hive's hash, or we could put the failed hash testcase into blacklist as @gatorsmile  's work #14498 . I will close the PR

@cloud-fan I will appreciate that you can help to close this PR~

Only the owner(yourself) can close this PR.


Picking from gallery in image_picker could be replaced by file_selector

With file_selector now supporting every platform besides Android. (PR in progress: https://github.com/flutter/packages/pull/3814), the functionality of the two plugin now overlap. On most platforms the API to choose an image from the gallery is the same for any other file type and is already supported by file_selector.
Replacing the functionality in image_picker with file_selector or removing it would reduce the maintenance burden of image_picker.

@stuartmorgan FYI


Extracts access limiting into its own model.

This is done in a similar manner to versioning.
Some points:

The access_limited field remains and needs to
be removed in a later ticket.
Some references to the field exist in the tests
and will also need removing when the column is
gone.
The viewable? method which I ported over from
the draft content item model is not used anywhere.
Neither was the access_limited? method which
was tested but unused.
Do we know why?

https://trello.com/c/m0TgNj9i/379-refactor-extract-accesslimitation-from-contentitems

Looks good – it's great to see the Draft and Live content item models fall closer inline. Looking forward to these models being merged together.
A few minor comments (some of which were @elliotcm's as we reviewed it together)

One thing that we discussed whilst reviewing this ticket was whether the access limited record should be destroyed when the draft content item is published. This would affect the user interface/behaviour for publishing apps.
This pull request keeps the behaviour the same as it was previously and is a refactor only, so we deferred this discussion/decision to later.

The access_limited field remains and needs to be removed in a later ticket.

Could you use the pattern we've got in Whitehall and Signon to deprecate the column? https://rubygems.org/gems/deprecated_columns

I've pushed some changes based on my prior comments.
I've also verified that the pact still passes between this branch and the content store master branch. :+1:

I have addressed all comments including @jamiecobbett's (see 1f843b87b78d3186148b2104c2ae9f6611ce2ace)
Ready for another round of reviewing.
Thanks

:+1:


Update or create skip-link component

Check if the skip-link component already exists, if not create a new component.

Can you describe what this component should do or which bitstyles component should be reflected, since it is not obvious to me.

@andreasknoepfle Link to the description of the component: https://bitcrowd.github.io/bitstyles/?path=/story/atoms-skip-link-overview--page


Grant access at billing account level

Can we add an example that describes how to do this at the billing account level or something more granular than the whole enrollment.  We are using this to create subscriptions in a devops pipeline and would like to give the service principal the least privilege possible.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 564fcf4e-69b9-a8b0-a362-c4effeca4a8b
Version Independent ID: 7ae1feea-e9ca-bc0a-c984-81b19f4339b8
Content: Grant access to create Azure Enterprise subscriptions
Content Source: articles/cost-management-billing/manage/grant-access-to-create-subscription.md
Service: cost-management-billing
Sub-service: billing
GitHub Login: @bandersmsft
Microsoft Alias: banders

@grayjeremy Is https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/assign-roles-azure-service-principals what you're looking for?

@grayjeremy I think the link I posted previously is just what you're looking for, so I'll close the issue.
#please-close


Upload limit: Display helpful error when input is malformed

Upload Limit 0  or special characters shouldn't be allowed:
In the Settings Activity  there is an option of setting the upload limit which should accept numbers fro 0 to 9 and some special characters shouldn't it display an error and disable ok button if the user has typed any such character.
Steps to reproduce:

Go to settings Activity

Tap the option SetUploadLimit

Currently if you type something irrelevant than it discards the current entered value and keeps the previous one. Instead we can display a message indicating that such numerals or characters are not allowed.
Device and Android version:
vivo 1713 API 24
Commons app version:
2.9.0-debug-sub_categories~677e0099
Would you like to work on the issue?
Yes

@maskaravivek @nicolas-raoul  For adding additional changes, that we desire to this PR we need to edit EditTextPrefernece the custom class created in the Project.

Need to a TextWatcher to the entered text
And an additional TextView to display error text

This might effect the other cases where we are using this class
Shall I make changes to the Widget or there can be other suggestions in the current improvement :)

Instead of adding another TextView below the edit text, you could use the method setError to show the error, also if EditTextPreference is used in a lot of places, you could may be extend the EditTextPreference and add the TextWatcher in this class to show the appropriate error.

TextView below the edit text, you could use the method setError

@ashishkumar0207 setError function which class have you metioned here?

TextView below the edit text, you could use the method setError

@ashishkumar0207 setError function which class have you metioned here?

@vanshikaarora EditText in android has a function setError(), actually two using which you could specify the error message(and maybe the appropriate icon), to show error messages.

@vanshikaarora EditText in android has a function setError(), actually two using which you could specify the error message(and maybe the appropriate icon), to show error messages.

Now that's clear :). So shall I modify the wrapper class to add a setError() method to the EditText?

@vanshikaarora  Any one having a reference to the wrapper class (by which I am assuming you mean the wrapper for the EditTextPreference class), will be able to directly access the setError function, right?, so I guess there would not be the need to modify the wrapper class for at least this thing.

by which I am assuming you mean the wrapper for the EditTextPreference

Yes I am talking about this Wrapper class EditTextPreference. Currently this class does not overrides the function setError().

So shall I modify the wrapper class to add a setError() method to the EditText?

Here I mean shall I override the setError() function of the class? :)

@vanshikaarora You will need to just call the method setError directly with the required params. I do not see a use case over here which requires overriding the function.

EditTextPreference$getEditText().setError

Thanks @ashishkumar468 :) I have used the same function and modified the code. Here is the preview of the final PR

Awesome!


Unconventional StationXML

Observed Issues:

sometimes magnetic field is identifed by channel code "T" rather than "F"
At least some historical station xml  files are returning units of Ampres and Tesla.  We need code to handle these cases and correct for them.  In particular, I think the Amperes reference is incorrect and the Tesla is contrary to our convention.
We need to both identify/log/report these issues and add compensatory scaling factors to calibration.
Some dipoles are Q2, Q3 for horizontal, but mt_metadata expects Q1, Q2.  How to handle this?

Most likely solution would seem to be update the stationxml rather than the code ... let's agree if that is the path and then make a plan for executing

I have added a one off script to clean up the non-conformant xml at the inventory stage before it goes into mt_metadata.

@kkappler Is there a date where the conventions changed.  For instance did FDSN update the channel codes some time and added the magnetic and electric channels to the current letter code?  Could we use the data of the metadata to use different channel code dictionaries?  If this is a one off, as Anna suggests maybe a simple workaround is fine.

I really don't know.  I know that when I used the NCEDC data circa 2010, all magnetometers were archived with "T" code. ... and vertical electric dipoles were Q1.  I reached out to Doug Neuhauser at BSL to get some insight into this issue and no response.

Jared has hacked around the "T" instead of "F" issue for magnetic channel code.
Karl's one off code is in iris_mt_scratch/sandbox/io_helpers/test_data.py
as def scan_network_for_nonconformity(inventory)

I Propose to use the workaround methods and scripts to create a Parkfield/Hollister mth5 file that we commit to the repo for testing.  That specific file can be maintained.  The scripts that make it can serve as an example for how to make an mth5 from unconventional xml, but we probably don't want to support these odd duck files in general.
I'll plan to get Gary's synthetic data and the parkfield running through the pipeline and then close this issue.

@kkappler sounds like a good plan, I would put the StationXML file in mt_metadata/data/station_xml and we can make a script in mt_metadata to convert the StationXML into an Experiment.  And another script in MTH5 to make an MTH5 file or just have one script in MTH5.

PR #142 updates reading of StationXML and most of these issues should be fixed.  Closing on PR


DDL execute fail

Sql syntax error in file:  WhereHows/data-model/DDL/ETL_DDL/kafka_tracking.sql
line 148, missing comma:
PRIMARY KEY(dataset_id,db_id,data_time_epoch,partition_grain,partition_expr)
Did you ever try execute the DDL after modify the sql?

It's fixed in #202. Thanks!


Lower case Greek should be italic

Not sure if this is a deliberate font choice, but there are some strange italicization results for greek versus roman characters in Makie. Is this a MathTeXEngine problem, or Makie?

more examples at this issue https://github.com/JuliaPlots/Makie.jl/issues/1245

cbar = Colorbar(f[1, 1][1,1],height=15,vertical=false)
Label(f[1, 1][1, 2], L"|\psi|^2\xi^2")

gives something like this

The absolute value bars are italicised. How do I make this a more realistic LaTeX font? I would expect something like this

Note that computer modern is not correctly rendered using the current implementation. It should look like this
https://tug.org/FontCatalogue/computermodern/

whereas I get, e.g.

The integral seems to have survived intact, but \pi and \mu ...

I would be happy to have a go at fixing this... any pointers on how to get started?

Thanks for the detailed report ! There are several things going on here.

Currently we are using New Computer Modern and not erm... "vanilla" Computer Modern, for technical reason since the former is a set of unicode fonts, while the latter has some pre-unicode complications. The result still looks wrong compared to the expectation anyway.
This is definitely mostly a MathTeXEngine problem, Makie trust this package about everything related to math rendering.
That being said, there is currently a dependency conflict between Makie and MathTeXEngine due to the last breaking update of GeometryBasics.jl (see https://github.com/JuliaPlots/Makie.jl/pull/1174 and related). This prevents the use of the very last version we have here. The src/protoype.jl renders think with a different pipeline, allowing to circumvent this issue and continue developpement as long at it isn't solve.
In the latest (v0.2.0) the wrong italics for delimiters (including e.g. |) should be solved.
For italicized greek letter it is more subtle. The main problem is finding where the glyphs for theses characters are located in the font files and making them available to Makie. They are not the main glyphs in the math font NewCMMath-Math nor the one from the italics font NewCM10-Italic... I fear that they may be alternative glyphs in the math font and we lack the ability to load those through TrueTypeAbstraction.jl. I unfortunately lack the latex knowledge to go read what latex engines do for this.
Alternatively, it may be simpler to implement the vanilla Computer Modern set of fonts, since it doesn't use any of the OpenType tricks AFAIK. This would mostly require introducing the table associating unicode symbol to their (non unicode) code in the correct font. Matplotlib already have a similar table here we could adapt. I'm starting to be more and more convinced using Computer Modern may be the way to go despite this added complexity.

So point 3 and 5 require a fix. For the latter, the starting point is this function
https://github.com/Kolaru/MathTeXEngine.jl/blob/88b62d1ceae3f96b40f131baa71d25262012f768/src/engine/texelements.jl#L121
It is given a Char and FontSet and additional information about the context (currently char type and I'm adding font modifier to fix #21, hopefully today) and has the job to return a TeXChar object containing the code of the char inside the font, the font, and whether this symbol should be considered slanted or not. By navigating what it does, you should get an idea of the madness of font handling :D
If you'd like to contribute and have any question about the internal, I'm happy to help.

How does LaTeXStrings.jl approach compare to the two approaches outlined in point 5? Is it doing a version of the matplotlib lookup? I ask this because LaTeXStrings result seems to produce consistently good output in my limited usage

LaTeXStrings.jl doesn't do any layouting or rendering, it is only a convenient type for strings representing LaTeX code. The hard work is always done by the backend.

It seems to me that all Greek is given a reverse slant. Could there be a systematic error in the pipeline logic?

5. For italicized greek letter it is more subtle. The main problem is finding where the glyphs for theses characters are located in the font files and making them available to Makie. They are not the main glyphs in the math font NewCMMath-Math nor the one from the italics font NewCM10-Italic... I fear that they may be alternative glyphs in the math font and we lack the ability to load those through TrueTypeAbstraction.jl.

So I should at least hunt down an example of italicised lower case greek, and figure out whether TrueTypeAbstraction.jl can load it?

how would I go about changing the default lower case greek to be italic?

@Kolaru do you think there is a way forward using NCM, or does this require deeper surgery?

The glyphs are definitely there already.
Here I used the Font Preview vscode extension to read the otf files in MathTeXEngine.jl:

Can I just set the font that MathTeXEngine would use globally to italic? I mean I would much rather specify a few roman cases using \textrm{ } than have nonitalic defaults. In fact that is the normal situation (italic defaults on everything) I think...
Is the font used for math set at the Makie level, or MathTeXEngine?
How do I tell Makie to tell MathTeXEngine to use NewCM10-italic?
Or set the default font in MathTeXEngine?

mmm. Fonts are confusing. No italic math file for NewCM.
Yet, the "Math example" here
https://tug.org/FontCatalogue/newcomputermodernroman/
is italic. I don't get this. What is going on here?

Sorry for the late answer I was quite busy recently.

These lower case greek symbols look perfect.

Yes they can do the job, they are not the glyphs used in the math example you linked below for some reason that eludes me as well. But we could use them anyway.

Can I just set the font that MathTeXEngine would use globally to italic?

Unfortunately things are a bit more complicated. Currently we separate between symbols and normal characters. Glyphs for symbols are taken from the math font and glyphs for normal character from the italic font. The catch is that greek letters are parsed as :symbol, so having them italicized require a bit more work.

Is the font used for math set at the Makie level, or MathTeXEngine?
How do I tell Makie to tell MathTeXEngine to use NewCM10-italic?
Or set the default font in MathTeXEngine?

The fonts are set in MathTexEngine there:
https://github.com/Kolaru/MathTeXEngine.jl/blob/32ce8c4cc586ca54e90c6c35af03d7924cae8fd3/src/engine/fonts.jl#L28-L44
The first dict define the default fonts with an identifiers (e.g. :italic for the italic font), while the second maps a element type (e.g. :symbol) to a font type (e.g. :math). As mentioned above, currently greek letters are parsed as :symbol so its clashes with symbols like \sum that we want to take from the :math font. The simplest way may be to parse greek letters as a separate type of object so we can easily define a font for them. Ideally I think it should be done by the parser (greek letters can be identified by looking at their unicode code).

do you think there is a way forward using NCM, or does this require deeper surgery?

Patching NCM is possible, but once I have more time I will look into the options in more details nonetheless.

awesome!

Is it fixed now? I'm using CairoMakie0.6.6 and still have the problem.

I guess https://github.com/JuliaPlots/Makie.jl/pull/1952 basically fixes this, apart from (so far) this new issue with backslash that will impact axis labels https://github.com/JuliaPlots/Makie.jl/issues/1967


Added missing attributes key to all language files

I found a problem with missing "attributes" key in language files, the same as #319  and #299. This PR adds this key to all files where it is missing.
Signed-off-by: Jakub Koudelka<EMAIL_ADDRESS>

Added in general code modifications

@koudi @mikeerickson is this already merged? There are some lang files missing the attributes key in the master branch:
be.js, bg.js, pt_BR.js and sv.js.


Add rng related methods

Add a public method to set rng seeds.
Also, I added the CMAwM._rng property for backward compatibility.

@c-bata Could you take a look when you have time?

Oh, sure, updated!


feat(sidesheet): create simple sidesheet content

Description

What's included?
Adds a simple sidesheet content component
Future work?
Remove second <mat-divider> if no actions provided
Test Steps

[ ] npm i
[ ] npm run serve
[ ] Components -> Sidesheet Content

General Tests for Every PR

[ ] npm run serve:prod still works.
[ ] npm run tslint passes.
[ ] npm run stylelint passes.
[ ] npm test passes and code coverage is not lower.
[ ] npm run build:lib still works.

closes https://github.com/Teradata/covalent/issues/1381
Screenshots or link to StackBlitz/Plunker

*reminder
Talk about td-sidesheet-content as its own directive.


BUG: Incorrect behavior of keyspace stats and DBSIZE command

Hi,
While going through the code, I observed that we are incrementing the KeyspaceStat count even if the operation is not creating new keys but overwriting the existing ones. This leads to the following behavior in dice:

In Redis:

Please assign this to me, if possible.

Hi @apoorvyadav1111
can i pick this up?

Hi @Abh-ay , I have been working on this and will be raising a PR soon. Thank you for your interest in contributing. Please keep watch on new issues.
Apoorv

Ok @apoorvyadav1111
I created PR also for this, should I close this request?
Thanks!!


Google not indexing the Makeability Lab website

For some reason, the Makeability Lab website is not being indexed correctly by Google. I noticed that if you search for a project like "HandSight Makeability Lab" that none of the results include the actual HandSight project page.
Perhaps relatedly, I recently received this notification from Google.

Google has validated your fix of Page indexing issues on site https://makeabilitylab.cs.washington.edu/. The specific issue validated was: Duplicate without user-selected canonical.

I then followed the link and it brought me here

More investigation is needed.

OK, if you use this Search Console Inspect tool on Makeabiltiy Lab urls like HandSight, you can see that robots.txt is not available and so not indexing our website

In fact, the entire Makeability Lab website does not appear on Google atm:

it appears to be on Bing:

Matt got back to me and said he tried to fix this. Something appears to now be working as I can at least submit the Makeability Lab website to be indexed:

This continues to be a problem. I can seem to get content indexed if I specifically add the page; however, as of now, we have 606 pages that are not indexed on the Makeability Lab website: 513 of these are marked as "crawled by not currently indexed"

You can see that we started to lose our indexing around Nov 2023:

This continues to be an issue for some pages:

For example, I searched "makeabilitylab handsight" and it didn't come up correctly in Google (interestingly, some newer projects are fine like "makeabilitylab mobiprint")
So, now trying to add handsight project manually

And other manually:

But many projects I spot checked were already on there...


services/horizon: Remove by_hash index, replace with index on hash prefix

PR Checklist
PR Structure

[ ] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
[ ] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
[ ] This PR's title starts with name of package that is most changed in the PR, ex.
services/friendbot, or all or doc if the changes are broad or impact many
packages.

Thoroughness

[ ] This PR adds tests for the most critical parts of the new functionality or fixes.
[ ] I've updated any docs (developer docs, .md
files, etc... affected by this change). Take a look in the docs folder for a given service,
like this one.

Release planning

[ ] I've updated the relevant CHANGELOG (here for Horizon) if
needed with deprecations, added features, breaking changes, and DB schema changes.
[ ] I've decided if this PR requires a new major/minor version according to
semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.

What
This commit removes by_hash index and introduces a new field: transaction_hash_prefix (and index on it). This make the duration of TransactionProcessor comparable to other processors (around 3-4x speed improvement) but also keeps the queries by transaction hash fast (however fast SELECT queries require reingestion).
Why
After analyzing the speed of queries sent by TransactionProcessor we understood that the majority of time is spent in updating indexes on string fields. In https://github.com/stellar/go/pull/4085 unused indexes were removed but there was one left that is actually used in /transactions/{id} endpoint: by_hash on transaction_hash field. Removing it makes the aforementioned endpoint extremely slow and, in reality, useless because of timeouts.
The solution in this commit creates a bigint field that represents a 16 characters (8 bytes) prefix of tx hash in numeric form (thus updating such index is fast). As a result all transactions are put into math.MaxUint64 buckets. The buckets are used for index scan in SELECT statement and found rows are then filtered by transaction_hash. In practice the number of buckets is vastly larger than number of existing transactions in public network so we should see more than one transaction in a bucket very rarely or never.
Known limitations
The problem with this solution is that while ingestion becomes faster the select queries are slow until the prefix column is populated by reingesiton. Also, while creating a new NULLable field is fast in Postgres, creating an index on it is actually very slow for a table with 2B rows (in stg it took around 6h to create such index).

I wonder if this would still be needed if we use https://github.com/stellar/go/pull/4094 to COPY into a temporary table without indexes and then merge it into the definite table.

I wonder if this would still be needed if we use #4094 to COPY into a temporary table without indexes and then merge it into the definite table.

I'm not sure if performance improvement in #4094 actually works well with an index on string field. The benchmark runs on a table without an index.

@stellar/horizon-committers I run some tests in stg last week and unfortunately I wasn't able to migrate history_transactions table using an SQL query: after around 10h the DB run out of this space (because all rows need to be rewritten but old ones are still kept until vacuum run).
I started think about intermediate solution that will allow people with reingested DB to be able to use improved version but these with old rows still use the old index on hash field. I came up with this:

In TransactionByHash we first run the query: select * from history_transactions where transaction_hash_prefix is null limit 1.
If the query above returns one row this means that there are some rows without prefix set. In such case we use the old query that's using transaction_hash field only.
If there are no rows it means all rows have the prefix set so we can use optimized query.
At the same time during ingestion (because it always have RW access to DB) we run the same query (select * from history_transactions where transaction_hash_prefix is null limit 1).
If no rows are found we run DROP INDEX IF EXISTS by_hash.

By doing this we remove the performance issue in TransactionProcessors as soon as all rows are updated so we don't break performance of /transactions/{id} in a mean time. This is a bit tricky and changes the schema outside migrations so wanted to check with you first before implementing.

At the same time during ingestion (because it always have RW access to DB) we run the same query (select * from history_transactions where transaction_hash_prefix is null limit 1).
If no rows are found we run DROP INDEX IF EXISTS by_hash.
Otherwise do nothing.

@bartekn I'm not sure if it's a good idea to have ingestion essentially perform part of the db migration automatically. perhaps we could have a separate horizon command to drop the index? we could also run the select query when horizon starts up and log a message about how the index should be dropped if no rows are found

Good point. I wanted it to be automagic for users but doing this via CLI command is safer/clearer.

@stellar/horizon-committers this is ready to review again. I implemented the idea here: https://github.com/stellar/go/pull/4087/#issuecomment-975981439 with @tamirms suggestion: we don't drop index automatically in ingestion. Instead, we print a message on startup that optimization is possible via horizon db optimize-schema command.


i hope real metrics，not sampling， trace can sampling

Please answer these questions before submitting your issue.

Why do you submit this issue?
[ ] Question or discussion
[ ] Bug
[ ] Requirement
[x] Feature or performance improvement

Question

What do you want to know?

Bug

Which version of SkyWalking, OS and JRE?

Which company or project?

What happen?
If possible, provide a way for reproducing the error. e.g. demo application, component version.

Requirement or improvement

Please describe about your requirements or improvement suggestions.
希望skywalking 有些指标可以在javaagent完成统计，因为这样相当于全量统计，比如url调用量，一旦采样之后，计算的99线，在出现问题的时候，没有参考意义，以及对于long service等调用提供支持，就像点评的cat监控，有些设计我认为比较好，这是自己定位问题的一点见解。

English only. If you want to contribute new feature we could discuss details.


Make user suggestions be filtered by username and ordered by best match

Task linked: QF-1955 Fix user suggestion list in "Add members" page


Back button on "Optional Actions" page of MSI installer returns to "Welcome", not to "Destination Folder"

Steps to reproduce

Download PowerShell-7.0.0-win-x64.msi.
Double click to open.
The "Welcome to the PowerShell 7-x64 Setup Wizard" page opens. Click "Next".
The "Destination Folder" page opens. Click "Next".
The "Optional Actions" page opens. Click "Back".

Expected behavior
The "Destination Folder" page opens.
Actual behavior
The "Welcome to the PowerShell 7-x64 Setup Wizard" page opens.
Environment data
Windows 10 version 1909.
$PSVersionTable is not available when PowerShell has not been installed yet.

Thanks for reporting and investigating in so much details. Your 2 comments on the 2 PRs above were useful and pointed out that my original implementation was correct but regressed in PR #8846
As you already pin-pointed the problem, can you submit a PR please?

As you already pin-pointed the problem, can you submit a PR please?

With the CLA requirement, I think not.

@KalleOlaviNiemitalo Can you please elaborate what the issue with the CLA requirement is?
I can do a PR as well but we'd want to understand first what exactly is preventing you (or anyone else in the community) from contributing.


Attention Mask Ops

The current attention seq2seq example (i.e., in seq2seq.py function attention_decoder) has a bug. The code (quoted below):
520     def attention(query):
521       """Put attention masks on hidden using hidden_features and query."""
522       ds = []  # Results of attention reads will be stored here.
523       for a in xrange(num_heads):
524         with variable_scope.variable_scope("Attention_%d" % a):
525           y = rnn_cell.linear(query, attention_vec_size, True)
526           y = array_ops.reshape(y, [-1, 1, 1, attention_vec_size])
527           # Attention mask is a softmax of v^T * tanh(...).
528           s = math_ops.reduce_sum(
529               v[a] * math_ops.tanh(hidden_features[a] + y), [2, 3])
530           a = nn_ops.softmax(s)
531           # Now calculate the attention-weighted vector d.
532           d = math_ops.reduce_sum(
533               array_ops.reshape(a, [-1, attn_length, 1, 1]) * hidden,
534               [1, 2])
535           ds.append(array_ops.reshape(d, [-1, attn_size]))
536       return ds
does not do a masking op before the softmax. This git pull request will fix this problem, i.e., we should mask the attention energies before the softmax based on the encoder sequence length.

Can one of the admins verify this patch?

this is in reference to this issue: https://github.com/tensorflow/tensorflow/issues/512

I agree that it would be good to add masking to the attention model and API. But I think adding an op and a separate cuda kernel just for that is a huge overkill. Why not just multiply pointwise by the mask in python before the softmax? Without a good reason, I find a PR like this not acceptable, the code complexity and maintenance cost looks much larger than the simple problem it solves.

sometimes the mask is generated dynamically (i.e., the mask is a function
of the previous attention)...
for example in speech recognition: Bahdanau/Chorowski et. al.,
http://arxiv.org/pdf/1508.04395.pdf
http://papers.nips.cc/paper/5847-attention-based-models-for-speech-recognition.pdf
--
William Chan
Carnegie Mellon University
(650) 450-9455
williamchan.ca
On Mon, Apr 25, 2016 at 11:28 PM, Lukasz Kaiser<EMAIL_ADDRESS>wrote:

I agree that it would be good to add masking to the attention model and
API. But I think adding an op and a separate cuda kernel just for that is a
huge overkill. Why not just multiply pointwise by the mask in python before
the softmax? Without a good reason, I find a PR like this not acceptable,
the code complexity and maintenance cost looks much larger than the simple
problem it solves.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/tensorflow/tensorflow/pull/2079#issuecomment-214628517

It is possible that I don't understand the problem, so please, clarify. But why would it need a special op to make the mask a function? If it's a tensor of 0s and 1s, it can be changed at every step, why can't this be done the simple way in python? Is there a very big speed advantage to having an extra op?

Well, the python side does not know what mask to generate (i.e., it doesnt
know the sequence_len of each sample in the minibatch). This is especially
true in speech where your minibatch has a high variance in sequence_len.
However, this problem can be somewhat alleviated to generate the mask of 1
and -FLT_MAX (not 1 and 0), if our input layer generates that for us.
However, this is not true for all attention masks, for example in speech
recognition where the encoder length can be thousands of frames, we want
the attention mask to be a function of the previous attention alignment,
i.e., in the papers I cited above, they generate the mask by looking at the
median of the previous attention and then use a window around it (in
essence, you have a somewhat monotonic sliding window on where to attend
to). Correct me if I am wrong, but I believe this can not be generated
python side?
Its not a speed advantage, (correct me if I am wrong), but I don't think
you can mask properly w/o an op like this.
--
William Chan
Carnegie Mellon University
(650) 450-9455
williamchan.ca
On Mon, Apr 25, 2016 at 11:45 PM, Lukasz Kaiser<EMAIL_ADDRESS>wrote:

It is possible that I don't understand the problem, so please, clarify.
But why would it need a special op to make the mask a function? If it's a
tensor of 0s and 1s, it can be changed at every step, why can't this be
done the simple way in python? Is there a very big speed advantage to
having an extra op?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/tensorflow/tensorflow/pull/2079#issuecomment-214635456

I was thinking about it a bit and I still don't see the problem. Here is how I'd do it:

add a parameter to attention_decoder called mask_function
this parameter is a python function that takes the step number and the previous attention mask tensor
and outputs a new attention mask tensor
every attention is multiplied by mask_function(prev_attention, step_number) (one-liner in python)

Would that work? If you want to feed-in attention, you'd just provide mask_function(_, i) = length-tensor[i] so you can feed the example lengths. And if you want to average by previous (or move it with a step), you just need to write a lambda that does it.
Or am I missing something crucial here?

sorry, didnt quite catch the last step:

you'd just provide mask_function(_, i) = length-tensor[i]

how does "length - tensor[i]" generate the mask? length is a tensor of the
sample length? and tensor[i] is what? and doesn't this result in a scalar
not a mask?
thanks!

ok. i think ur right, just need a tf.cond or something similar in the
mask_function right? and then use that to generate the mask?
--
William Chan
Carnegie Mellon University
(650) 450-9455
williamchan.ca
On Tue, Apr 26, 2016 at 11:15 PM, William Chan<EMAIL_ADDRESS>wrote:

sorry, didnt quite catch the last step:

you'd just provide mask_function(_, i) = length-tensor[i]

how does "length - tensor[i]" generate the mask? length is a tensor of the
sample length? and tensor[i] is what? and doesn't this result in a scalar
not a mask?
thanks!

To answer your question, let me be a little bit more precise. What I suggest is to modify the seq2seq.py in two ways.
(1) on lines 546, 565, 567 instead of attns = attention(state) let's do attns = attention(state, attns, inp, i) so the attention() function now takes the arguments attention(query, previous_attention, decoder_input, step_number)
(2) on line 530 instead of a = nn_ops.softmax(s) let's do a = nn_ops.softmax(s + mask_f(previous_attention, decoder_input, step_number) * -LARGE_NUMBER where mask_f is an extra argument to the decoder.
In this way the mask_f function gets all the current information in the step. Its task is to generate 1.0 (or anything positive) if we want to mask something and 0.0 otherwise. The 1.0 will be turned into a -LARGE_NUMBER bias in the softmax and effectively disable that field for attention, right?
The tensor s has shape [batch_size x attn_length], where attn_length is the size of the attention_states (length of the encoder in a seq2seq model). Let me focus on the basic case when we want mask_f to mask an input iff the decoder input symbol is a padding, which means that it is all zeros. We can do this in two ways, I think. If we're sure the decoder input is all 0s only if it's padded and has a reasonably large norm otherwise, then we can do mask_f(_, dec_inp, _) = tf.minimum(1.0, tf.nn.l2_loss(dec_inp) * LARGE). If we're not sure of that, then we can do mask_f(_, _, step) = self.decoder_mask[i] when declaring the mask_f function somewhere in our model code. In this case decoder_mask are placeholders of the same kind as decoder_inputs that are 1 in case of padding and 0 otherwise -- and we need to make sure to feed the appropriate values.
I'm sure I missed some things. In particular, previous_attention coming to the attention function should probably be the previous attention mask, not the previous attention vector as I suggested above (so the new mask needs to be returned in the function and the pointer saved in the loop, a few more lines). But does the general idea sound reasonable and flexible enough?

I agree in general your idea works. Just some minor things...

The tensor s has shape [batch_size x attn_length], where attn_length is
the size of the attention_states (length of the encoder in a seq2seq
model). Let me focus on the basic case when we want mask_f to mask an input
iff the decoder input symbol is a padding, which means that it is all
zeros. We can do this in two ways, I think. If we're sure the decoder input
is all 0s only if it's padded and has a reasonably large norm otherwise,
then we can do mask_f(_, dec_inp, ) = tf.minimum(1.0,
tf.nn.l2_loss(dec_inp) * LARGE). If we're not sure of that, then we can
do mask_f(, _, step) = self.decoder_mask[i] when declaring the mask_f
function somewhere in our model code. In this case decoder_mask are
placeholders of the same kind as decoder_inputs that are 1 in case of
padding and 0 otherwise -- and we need to make sure to feed the appropriate
values.

The problem w/ this approach is (where self.decoder_inputs is generated by
the input layer), is that the encoder may have a pyramid architecture
(i.e., time subsampling) which is very common in speech. i.e., depending on
the graph, we may reduce it by 4/8/16 multipliers. so we need to know this
ahead of time (the time factor reduction) and pass that in to the input op
that is generating the mask.
Where as w/ a op that does the masking for you where you only need to pass
in the sequence_length, every time we do hierarchical subsampling in the
graph, we can just do sequence_length_tensor = sequence_length_tensor / 2;
energies = attention_mask(energies, sequence_length);
However, I think this problem can be alleviated by tf.cond(), where we can
do something like tf.cond(sequence_length < encoder_idx, 0, -FLT_MAX),
something like that... however we need to do this T times where T is the
length of the encoder and encoder_idx \in range(0, T).
But I agree w/ you, I think (havent completely tested it out) you are right
this can be done almost completely w/ the existing python ops. Personally,
I'll still be using my op since I already wrote it and it works and it
should be slightly faster (and simplier graph). But we can close this PR if
you think this is adding too much code to maintain.


Change readme title to chimoney community projects

Assign me this @phyleria

Will you still be working on this @Olvine23 ?

Great,
Let me know once you're done.

Yes, it's done.
On Wed, Oct 18, 2023 at 3:38 PM Phyllis Atieno @.***>
wrote:

Great,
Let me know once you're done.
—
Reply to this email directly, view it on GitHub
https://github.com/Chimoney/chimoney-community-projects/issues/206#issuecomment-1768365388,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMWUKRIK2LFAJV7LVADKAADX77EVZAVCNFSM6AAAAAA6AENI6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRYGM3DKMZYHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

@phyleria assign me this


add option to hide files with 100% type coverage from output

Including the option --hide-complete will hide the files with 100% coverage from the output. This makes it much easier to see which files still have missing type declarations.
The files with 100% coverage are still included in the calculation of the total coverage percentage.

This would be great!

I think #3 is pretty much the same but not tied to 100% exactly but respects the defined minimum.

Can you pull request to 3.x instead? Please call this option --terse.


API 500 on new installation

Installed via HACS, adding integration returns the error details below. The same error also occured with the version of the Subaru module built into HA Core (which is why I tried the HACS version).
Logger: custom_components.subaru.config_flow
Source: custom_components/subaru/config_flow.py:59
Integration: Subaru (HACS) (documentation, issues)
First occurred: 1:07:39 PM (1 occurrences)
Last logged: 1:07:39 PM
Unable to communicate with Subaru API: HTTP 500: <coroutine object ClientResponse.text at 0x7f6d0317c0> RequestInfo(url=URL('https://mobileapi.prod.subarucs.com/g2v21/twoStepAuthContacts.json'), method='POST', headers=<CIMultiDictProxy('Host': 'mobileapi.prod.subarucs.com', 'User-Agent': 'HomeAssistant/2022.5.5 aiohttp/3.8.1 Python/3.9', 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Cookie': 'JSESSIONID=EAA829EFAC06239B329F4007C4F7E63E; X-Oracle-BMC-LBS-Route=4f65eea550dddceedeb77fa2e2997443fe398a6302222a4a6cf6a9a4be077efb53b7bfbe45671244', 'Content-Length': '0', 'Content-Type': 'application/octet-stream')>, real_url=URL('https://mobileapi.prod.subarucs.com/g2v21/twoStepAuthContacts.json'))
Snip from Full Logs:
/usr/local/lib/python3.9/site-packages/subarulink/connection.py:334: RuntimeWarning: coroutine 'ClientResponse.text' was never awaited
  raise SubaruException("HTTP %d: %s %s" % (resp.status, resp.text(), resp.request_info))
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
2022-05-25 13:07:39 ERROR (MainThread) [custom_components.subaru.config_flow] Unable to communicate with Subaru API: HTTP 500: <coroutine object ClientResponse.text at 0x7f6d0317c0> RequestInfo(url=URL('https://mobileapi.prod.subarucs.com/g2v21/twoStepAuthContacts.json'), method='POST', headers=<CIMultiDictProxy('Host': 'mobileapi.prod.subarucs.com', 'User-Agent': 'HomeAssistant/2022.5.5 aiohttp/3.8.1 Python/3.9', 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Cookie': 'JSESSIONID=EAA829EFAC06239B329F4007C4F7E63E; X-Oracle-BMC-LBS-Route=4f65eea550dddceedeb77fa2e2997443fe398a6302222a4a6cf6a9a4be077efb53b7bfbe45671244', 'Content-Length': '0', 'Content-Type': 'application/octet-stream')>, real_url=URL('https://mobileapi.prod.subarucs.com/g2v21/twoStepAuthContacts.json'))

I uninstalled/reinstalled the integration but have not been able to reproduce the error. HTTP 500 errors have typically been temporary outages on the Subaru side. Have you tried again with any success? Is 2FA with the official mobile app working?
Your logs do show that there is indeed an issue at connection.py:334, so thanks for sharing. But it isn't the root cause, since the HTTP 500 had already occurred, and this is just an error in the exception reporting, which unfortunately removes the useful message content (if any was provided).

Finally got around to re-trying and was able to authenticate and integrate my forester. I'm assuming whatever this issue was back in May was related to Subaru's site and not the integration itself. Thanks!