Using Bayesian optimization to fit the parameters of a GP

Hi,
I wonder whether it would be possible to use a GPy in order  to fit a GP (quad GP  ?) to the negative log likehood of another GP (x = hyper parameters and y =nlml) and then try to find the hyper parameters by doing BO.
The closest approach I have found was Bayesian quadrature, but if I understood it correctly it tries to model the whole posterior rather than nlml . In addition in this case, should one try to minimize the more conventional acquistions (say EI and UCB) or perhaps try to minimize uncertainty instead  (say the variance of the quad GP)  ?
Thanks very much,

Thanks for your interest! I think you gonna reach more people if you write to the email list, and have a look at GPyOpt Also located in the SheffieldML organisation on github. The email is in the readme on GPy's github.

On 10 Nov 2016, at 18:04, achapchap<EMAIL_ADDRESS>wrote:
Hi,
I wonder whether it would be possible to use a GPy in order to fit a GP (quad GP ?) to the negative log likehood of another GP (x = hyper parameters and y =nlml) and then try to find the hyper parameters by doing BO.
The closest approach I have found was Bayesian quadrature, but if I understood it correctly it tries to model the whole posterior rather than nlml . In addition in this case, should one try to minimize the more conventional acquistions (say EI and UCB) or perhaps try to minimize uncertainty instead (say the variance of the quad GP) ?
Thanks very much,
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

From what I understand, you essentially want to attempt global optimisation
of the hyper-parameters using bayesian optimisation (BO).
This would certainly be possible, I had considered doing it before. There
is a really great BO package that is built on top of GPy called GPyOpt.
Essentially you want to just use a GPy model (the nll) as your function.
There is an example notebook here:
https://github.com/SheffieldML/GPyOpt/blob/master/manual/GPyOpt_GPy.ipynb
Cheers,
Alan
On 10 November 2016 at 18:04, achapchap<EMAIL_ADDRESS>wrote:

Hi,
I wonder whether it would be possible to use a GPy in order to fit a GP
(quad GP ?) to the negative log likehood of another GP (x = hyper
parameters and y =nlml) and then try to find the hyper parameters by doing
BO.
The closest approach I have found was Bayesian quadrature, but if I
understood it correctly it tries to model the whole posterior rather than
nlml . In addition in this case, should one try to minimize the more
conventional acquistions (say EI and UCB) or perhaps try to minimize
uncertainty instead (say the variance of the quad GP) ?
Thanks very much,
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/SheffieldML/GPy/issues/457, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABIrsUXx3pmevY_nwRVBsZJT8EzeAjXZks5q81yfgaJpZM4Ku9iW
.

HI!
Thanks for the interest! Yes, the notebook that Alan mentioned is an
example of how you can do that. Is not quite documented but you can have an
idea of how to do it having a look to the code. Just let us know if you
have any questions. Perhaps the best is if you open an issue in GPyOpt (
https://github.com/SheffieldML/GPyOpt/issues) if you want to discuss this
point further.
Cheers!
Javier
On 10 November 2016 at 19:09, Alan Saul<EMAIL_ADDRESS>wrote:

From what I understand, you essentially want to attempt global optimisation
of the hyper-parameters using bayesian optimisation (BO).
This would certainly be possible, I had considered doing it before. There
is a really great BO package that is built on top of GPy called GPyOpt.
Essentially you want to just use a GPy model (the nll) as your function.
There is an example notebook here:
https://github.com/SheffieldML/GPyOpt/blob/master/manual/GPyOpt_GPy.ipynb
Cheers,
Alan
On 10 November 2016 at 18:04, achapchap<EMAIL_ADDRESS>wrote:

Hi,
I wonder whether it would be possible to use a GPy in order to fit a GP
(quad GP ?) to the negative log likehood of another GP (x = hyper
parameters and y =nlml) and then try to find the hyper parameters by
doing
BO.
The closest approach I have found was Bayesian quadrature, but if I
understood it correctly it tries to model the whole posterior rather than
nlml . In addition in this case, should one try to minimize the more
conventional acquistions (say EI and UCB) or perhaps try to minimize
uncertainty instead (say the variance of the quad GP) ?
Thanks very much,
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/SheffieldML/GPy/issues/457, or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABIrsUXx3pmevY_
nwRVBsZJT8EzeAjXZks5q81yfgaJpZM4Ku9iW>
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/SheffieldML/GPy/issues/457#issuecomment-259778875,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGiS8wS5P3Vb0sb2sH94whGGFL3TifHAks5q82vSgaJpZM4Ku9iW
.

Thanks very much, very helpful comments.
The main reason I am considering doing this is to try to address the variance of negative log likehood as well and hopefully try to avoid regions with high variance. Intuitively I am temped to think that regions where log likehood has a large variance are prone to overfitting. Do you think this could make sense ?
Still regarding the GPy, I noticed that the Gabor kernel does not seem to be implemented. However, it seems to be possible to construct it by taking the product of the periodic kernel (cos(r) ) and the exponential kernel. Is that the case or should I try to construct the Gabor kernel following the GPy documentation ?

You should give it a go and see if you get problems by doing the combination. If you run into stability problems you might consider implementing a more stable version directly yourself.


Clarification on Timestamps and End-to-end runtimes

I'm trying to figure out when the "timestamp" gets written for any given test.  Is it when the test starts, or when the test finishes?  (Is it possible to get both?)
I'm also trying to figure out end-to-end runtime...  is it inclusive of setting up the environment (creating networks, spawning vm's, etc)?  In cases where packages are being installed to run the benchmark, does the end-to-end runtime include this time?
I'm trying to figure out how to isolate (and benchmark) the FABRIC CONTROLLER of the cloud providers to see what kind of overhead each has....  apart from cluster_boot, there really isn't anything that measures the overhead of orchestration in the benchmarks...
If you can control variables like OS, Image, Packages, etc... then the end-to-end runtime is useful for understanding the differences in fabric controller overhead.

The timestamp is generated at the end of the benchmark run, before resource cleanup happens.  Including the start of the benchmark wouldn't be difficult, but isn't implemented presently.

I'm also trying to figure out end-to-end runtime... is it inclusive of setting up the environment (creating networks, spawning vm's, etc)? In cases where packages are being installed to run the benchmark, does the end-to-end runtime include this time?

Yes, the End to End Runtime reported includes resource provisioning (VMs, disks, etc), benchmark runtime, and resource teardown time.

Thanks for this.  So in order for me to control all the variables, I'll need to create a VM that I can keep consistent between the various cloud providers (fun) :-)

Should I close this issue and open a separate one specifically for the request to implement "start time stamp" for the benchmarks?

Yes, please do. Thanks!


(refactor) Add get/set and plugin for Java target & Remove Corda target

Signed-off-by: Jerome Simeon<EMAIL_ADDRESS>This PR:

adds getters/setters to the Java target (using options when calling the fields visitor)
makes Java fields private
adds a simple kind of plugin for the Java visitor to let users parameterize code-generation
removes the Corda target which can now be implemented using the provided plugin

A variant of the Corda target is expected to be provided in the Accord Project Cicero library.
Issue/User story
This PR avoids duplication between alternative Java-based target formats (e.g., Corda and Java).
Automated Tests
Testing is provided for default Java plugin (empty) and abstract plugin (raising errors).

thanks!


Chromium docs 5229192

Updated link per feedback in Issue #868 .

Docs Build status updates of commit 42e4521:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-edge/devtools-guide-chromium/network/reference.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

#sign-off and merged.


[FEATURE] anthropic claude models

thanks for this great project
I would like to propose the inclusion of the anthropic claude models, it is a necessary function to do solutions with this kind of models
https://js.langchain.com/docs/modules/model_io/models/chat/integrations/anthropic
https://docs.anthropic.com/claude/reference/complete_post

hi, @JCASTANO if you have access to Claude, we can try to implement it, but then we will need your help to test it, as our contributors still don't have access to it.

@JCASTANO Claude support was added via AWS Bedrock support. Not sure if it is good enough for your use case.

@JCASTANO feel free to reopen if still needed


CIS Feature: Enforcing SELinux

Background:
PR #771 added the first iteration of the new CIS feature to Garden Linux. However, this feature is still declared as WIP and needs to be adjusted to fulfill all requirements of #647.
Task:
This is more or less a reminder task. To fulfill the CIS benchmarks we need to switch the current SELinux status from Permissive to Enforcing. A related check script is already including within the corresponding unit test and will fail until this has been changed.

As discussed we'll change this just for cis from permissive mode to enforcing.

This is enabled and works as desired. However, we need to adjust some policies in ä993.

<IP_ADDRESS>_enable_selinux    [ OK ] SELinux is configured in syslinux bootloader
<IP_ADDRESS>_enable_selinux    [ OK ] Bootloader SYSLINUX found
<IP_ADDRESS>_enable_selinux    [ OK ] Check Passed
<IP_ADDRESS>_enforce_or_compla [INFO] Working on <IP_ADDRESS>_enforce_or_complain_apparmor
<IP_ADDRESS>_enforce_or_compla [INFO] [DESCRIPTION] Enforce or complain AppArmor profiles.
<IP_ADDRESS>_enforce_or_compla [INFO] <IP_ADDRESS>_enforce_or_complain_apparmor is disabled, ignoring
<IP_ADDRESS>_enforcing_apparmo [INFO] Working on <IP_ADDRESS>_enforcing_apparmor
<IP_ADDRESS>_enforcing_apparmo [INFO] [DESCRIPTION] Enforce Apparmor profiles.
<IP_ADDRESS>_enforcing_apparmo [INFO] <IP_ADDRESS>_enforcing_apparmor is disabled, ignoring
<IP_ADDRESS>_enforcing_selinux [INFO] Working on <IP_ADDRESS>_enforcing_selinux
<IP_ADDRESS>_enforcing_selinux [INFO] [DESCRIPTION] Ensure SELinux is enforcing.
<IP_ADDRESS>_enforcing_selinux [INFO] Checking Configuration
<IP_ADDRESS>_enforcing_selinux [INFO] Performing audit
<IP_ADDRESS>_enforcing_selinux [ OK ] SELinux is active.
<IP_ADDRESS>_enforcing_selinux [ OK ] Check Passed


Tutorials: minor typo/import fixes

Couple of typos fixed and cleaned unused imports

This change is

Thanks for the PR! Much appreciated! Please refer to the contributing guide. We are aligning internally on how to make contributing to Metaflow easy and transparent.


[🧪] Make the E2E test run in every new commits to the main branch

Now the CI can only run using workflow dispatch which is need a manual click.
Add this block:
on:
  workflow_dispatch:
  push:
    branches: 'master'

To the workflow file at https://github.com/depapp/IndoGitHubers/blob/main/.github/workflows/e2e.yml#L3-L4

Hi, mas! I want to work on this issue, may I? 🙏


Add Jellyfin Support

Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
Works like plex but for Jellyfin
Describe alternatives you've considered
None
Additional context
None

Has there been any progress on this?


added shoutout command that returns a tallycoin invoice

I added a new command that returns a tallycoin lightning invoice. Users can specify an amount and a memo, that will be displayed on tallyco.in. As tallyco.in allows memos up to 140 characters, memos longer than that will be shortened (users wont get notified).
Invoices are generated by tallycoin via their API . There is currently no error handling for API failure.
It would be nice if we found a way to inform users about character limit and the podcast specific 21.000 sat shoutout threshold.

I added chat.type as a condition to the shoutout function. If users use /shoutout in a group they will be told to use it in a private chat with the bot instead. I though this might be a good idea to avoid confusion in bigger groups.
I'll modify the return to show qr code + invoice string as soon as i figure out how to do that.

I figured it out and added the function. "/shoutout  " will now request a lightning invoice from tallyco.in API and return it as QR code + string.

I figured it out and added the function. "/shoutout " will now request a lightning invoice from tallyco.in API and return it as QR code + string.
Still havn't figured out how to commit as github user though...

I think you have to set the email in your commit to the same email you use for github.

Thanks, that worked. Commits are now linked to the correct user :)
I moved TALLYDATA into the getInvoice function, which should resolve the cuncurrency issue. Also i have changed the createQR function to include chat_id in the file name. The image-files will be removed once they have been sent.
I looked at the telegram convesation handler. However i was not able to figure out how to save values until the end of the conversation. So when the amount is specified in the first part of the convesation, it is easy to pass that value to the second part, but i wasn't able to retrieve this value at the end of the conversation to request an invoice. While this would be very nice to have, i am not sure if this is worth the hassle (unless there is an easy fix for this).

I think i figured it out. However, as my testing envoirment is a sinmgle file, I am not sure if conversations states are imported correctly into bot.py.
/shoutout (only in private chats) will start a conversation with the bot. First the user has to specify a donation amount, then the donation memo. The bot will return a lightninhg invoice + qr code.

I have implemented the fixes suggested in your commit/PR with one exception.
In https://github.com/Egge7/einundzwanzigbot/pull/1 you suggested to return an error, whenever users try to donate less than 21k satoshis. I don't think this is necessary. With my current commit, the bot will point out, that only shoutouts above 21k will be read out in the next podcast episode, however it will still accept donations below 21k.

I've added some more fixes, I think this feature is ready to be merged now.

Thanks for the review! I definitely have to change up my workflow, to avoid making mistakes in the future.
I think this looks great, however I am not quite sure if memos that are longer than 140 characters are handled the right way now.

I have added another function to the shoutout conversation. I think this feature is pretty much completed now.
checkPayment will check the payment status of the generated invoice every 20 seconds for 10 minutes. Once the payment ist confirmed, users will receive a "Thank you"-message.

Understood! I didn't think of this. I reverted the checkPayment commit. Thanks for the detailed feedback!


Updating maintainer

Added new maintainer and moved previous as authors!

@clalancette should this be on humble branch instead?


Can't read Form data from HttpContext

Thanks for this great repo and article!
I've hit an issue implementing this into my project though and not sure where the issue is stemming from (worth noting I'm using RC2 version of Core: 1.0.0-rc2-3002702 ).
I've set code up exactly as in your Git Repo but have hit problems in the middleware code and not entirely sure what is causing the problem.
So with Content-Type: application/x-www-form-urlencoded
I post to /api/token and all good. The code gets inside private async Task GenerateToken(HttpContext context) but then blows up.
context.Request.Form["username"] throws an exception and returns HTTP Error 502.3 - Bad Gateway.
Form = '((Microsoft.AspNetCore.Http.Internal.DefaultHttpRequest)((Microsoft.AspNetCore.Http.DefaultHttpContext)context).Request).Form' threw an exception of type 'System.MissingMethodException'
Changed Content-Type: multipart/form-data, but this too throws different exception:
Form = '((Microsoft.AspNetCore.Http.Internal.DefaultHttpRequest)((Microsoft.AspNetCore.Http.DefaultHttpContext)context).Request).Form' threw an exception of type 'System.IO.InvalidDataException'
So there would appear to be something fundamentally wrong with either the set up of my WebApi or it's a bug in RC2.
Worth noting the WebAPi is working just fine with 'application/json', PUT, POST, GET, DELETE verbs etc all work fine.
Can anyone help?

Closing this.
My issue was caused by a bug in RC2 - after full upgrade to Core 1.0.0 Preview 2 SDK it all worked.
Shocking...

Hey @sicollins - sorry I couldn't reply to this earlier. It's been a busy week!
Glad you got your problem fixed. I've seen a few other issues from the RC2->RTM transition so that's not too surprising.
Cheers!


Kube detection in test suite is flawed

Currently: https://github.com/dagger/dagger/blob/main/pkg/alpha.dagger.io/helpers.bash#L77 assumes that if ~/.kube/config exists, then there must be a running kube.
I was bitten by this (leftover file, no running kube, test suite failing in a very confusing way).
Is there a better way to verify that the cluster exist rather than this?
Maybe something as simple as:
k=$(grep "<IP_ADDRESS>" ~/.kube/config); curl "${k#*server: }"; echo $?

Is this still relevant in 0.2?

Probably not?
https://github.com/dagger/dagger/blob/main/docs/v0.1/learn/tests/helpers.bash


Clarify grammar regarding let binding

Changed

its value is supposed to and will never change

to

its value will never change

:+1:

Thanks very much @cyung!


Deduplicate OpenStack auth functions

What this PR does / why we need it:
It unifies all the OpenStack auth options and deduplicates code.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
#640
Special notes for your reviewer:
This PR is based on #733. I'll rebase this PR once #733 is merged.
Release note:

Unify openstack authenticate options

@kayrus this looks great.
A couple of questions, if this is ready do you feel ok squashing the commit? I'm not sure we need the complete history and the changeset is fairly intuitive.
On the docs update, did the configuration api change? We would definitely want to alert the userbase to the change for the 1.16 release if that's the case.
Want to get some more eyes on it before approving, but /lgtm
/assign @adisky

/ok-to-test

@hogepodge

if this is ready do you feel ok squashing the commit

commits can be squashed automatically in github web UI.

On the docs update, did the configuration api change?

I kept the full backward compatibility, but additionally added new auth options to components. Unfortunately new options are not documented. I assume this must be a new PR with the docs refactoring.

@kayrus could you open a doc PR in parallel, it will be easier for reviewers

@adisky updated the docs, where OS auth options were mentioned.

I'm ok if you just merge this PR and close #733

@adisky any concerns or comments on this PR?

@kayrus give me sometime to review, very busy this week, would to review it tomarrow

@adisky do you have any updates on this PR? Concerns, objections?

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.15

Is this PR "only" blocked by flaky tests or does it need more work?

@jborg we need to be sure for this PR to pass the conformance tests, tests were flaky due to some other reasons, Tests are passing now, you need to rebase the PR to latest master, please rebase  after https://github.com/kubernetes/cloud-provider-openstack/pull/753 this one gets merged.

/lgtm
@lingxiankong @gman0 @tsmetana @tombarron PTAL
/priority-important-soon

I was asked to squash this PR. If someone wants to see the detailed history, he can take a look on the https://github.com/kayrus/cloud-provider-openstack/tree/step-by-step. It has identical code, but not squashed.

openstack-cloud-controller-manager test ok

octavia-ingress-controller ok

/lgtm
/approve


Audit our project management system

[ ] Make sure all issues and PRs are properly labeled
[ ] Ensure everything is searchable in the project
[x] Clean up the public roadmap for release
[ ] #4385

FYI Tristan confirmed in another issue we can reassign his tickets.


Add code to jump to anchor link when hash changes

Additions

When hash changes, also call code to jump to anchor links

Reviews
@virginiacc @cfarm

@amymok how can I test these? is there a link i can click or a URL

nevermind, found this! https://github.com/cfpb/owning-a-home/pull/496
looks good :+1:

oops - the loan options collapse browser tests are failing now, can you take a look?


PowerToys Preview/Window doesn't open since the last two updates

Microsoft PowerToys version
0.69.1
Installation method
GitHub, PowerToys auto-update
Running as admin
None
Area(s) with issue?
General, PowerToys Run, Settings, System tray interaction
Steps to reproduce
PowerToysReport_2023-04-14-18-57-51.zip
✔ Expected Behavior
No response
❌ Actual Behavior
All the features seem to be working, except powertoys run which fails to open just like the settings window
Other Software
No response

This might be caused by some older files not being uninstalled correctly.
Would you mind:
1 - Uninstall PowerToys.
2 - See if the PowerToys install directory (default: C:\Program Files\PowerToys) is still there. If it is, delete it manually.
3 - Install PowerToys.
Please let us know if this works for you.
/needinfo

This might be caused by some older files not being uninstalled correctly. Would you mind: 1 - Uninstall PowerToys. 2 - See if the PowerToys install directory (default: C:\Program Files\PowerToys) is still there. If it is, delete it manually. 3 - Install PowerToys.
Please let us know if this works for you. /needinfo

I did this, and it fixed it for me. Thanks.

This might be caused by some older files not being uninstalled correctly.
Would you mind:
1 - Uninstall PowerToys.
2 - See if the PowerToys install directory (default: C:\Program Files\PowerToys) is still there. If it is, delete it manually.
3 - Install PowerToys.
Please let us know if this works for you.
/needinfo

Worked for me as well, thanks!

Same thing was happening to me on Windows 11 latest (22H2 build 22621.1555) for a couple of PowerToys versions.
I can confirm this works, and even simpler solution.

Go to Add or remove programs
Uninstall PowerToys (preview)
Install PowerToys downloaded from here
All PowerToys settings preserved, and its Settings open as expected.

This has been shipped for 0.70.0. Please updated to the latest release. Thank you.


Map existing e2e tests to APIs they cover

In analyzing some of the existing e2e tests, specifically POD related APIs, it's become evident that the e2e tests are not one to one with the APIs, as most of the well written tests (emulating user stories) hit more than one API.
To assist with Identifying e2e tests to upgrade to Conformance tests, we need find a way to correlate our raw audit log entries to the tests that created them.
Some options to explore:

[ ] Inject a parameter into the http requests identifying the e2e test that ends up in the audit-log entry
[x] Dual time-series data syncing (record timestamps for start/finish for each test -- less exciting, but this got us across the line for our first dataset)

Open to other options, some type of injection is the cleanest.
I'll start by exploring what injection options are available.
/cc @fedebongio @AishSundar @wangzhen127

Message sent to relevant groups for feedback:
https://groups.google.com/forum/#!topic/kubernetes-sig-testing/m57_-AB0muM

/cc

/cc @mayankg

Inject a parameter into the http requests identifying the e2e test that ends up in the audit-log entry

Another alternative to this could be requesting a specific endpoint when the e2e test begins and ends
ie /apis/?mark=test-start&file=test/e2e/apimachinery/garbage_collector.go&message="should delete pods created by rc when not orphaning"`

Maybe we set the UserAgent for each api to include the current line-number / e2e test.
https://github.com/kubernetes/client-go/blob/d9cf97751096b28cbb73b091f61226e8ddd4ab42/rest/config.go#L86-L87
thanks Morgan Harold Bauer

Maybe we could just wrap the e2e test script and parse the .It() stdouts and timestamp them then compare to the audit log entry timestamps

Another option might when we detect a new test starting in the stdout, we curl out to an endpoint that will generate an audit-log entry.
With both stdout monitoring approaches we need to be aware of timing / interpolating the e2e test boundaries.

@AishSundar Preliminary results using timestamps obtained from wrapping the e2e test script.
See https://github.com/cncf/apisnoop/tree/master/dev/e2e-audit-correlate to see how these were obtained.
The only step not in the scripts is running the following on the master of the test cluster before the e2e tests are started:
nohup tail -f -n0 /var/log/kube-apiserver-audit.log 2>/dev/null &
Results (14MB text file)
20180605_e2e-test-endpoint-hits.txt.gz

Looks like there is interest in logging user-agent with audit-logging and setting user agent per-e2e-test. I'm going to create tickets for those.

[ ] Set user-agent per test in e2e framework
[ ] Add user-agent to audit-logging

@rohfle this is awesome progress ! Thanks for such a quick turnaround ! A couple of ques aorund the results

Was the outputfile (20180605_e2e-test-endpoint-hits.txt.gz) obtained by running all e2e tests our just the conformance suite?
https://github.com/cncf/apisnoop/tree/master/dev/e2e-audit-correlate does not have much info on how to trigger the script post a test run. Would it be possible to qucikly out a README together?
when you say "preliminary" results, are you working on making more improvements to this approach and/or focusing on another approach to corroborate results?

I ran all (or nearly all?) the e2e tests, not just conformance (it took many many hours and the audit log was around 1GB)
I can put a readme together to explains things
There are a couple of assumptions that I made to reduce complexity. I am using the console output from kubetest to determine when tests are run. I can detail these in the readme also.

The assumptions are mainly:

The latency between kubetest writing to stdout and my program reading and detecting a new test is negligible. I measured 300us using the simulate-e2e.py script.
The time between the GCE master and the staging server I was running of were synchronised.
The mark -------------- denotes a new test being run and this mark is found nowhere else in the e2e logs.

@rohfle thanks for the explanation and listing the assumptions. For now this is more than what we have and we can definitely use this data to narrow down our search to answer "which test case exercised the API". Looking forward to the README
Coming back to my one other previous ques, @hh are you actively working on "logging user-agent with audit-logging and setting user agent per-e2e-test"? Until we get data out of that we will use Rohan's report to guide test case promotion to conformance

@AishSundar The README is live - https://github.com/cncf/apisnoop/tree/master/dev/e2e-audit-correlate. I've also included everything except the audit log (due to its size and potentially secret contents)


from somefile import *

from somefile import *  is failing, probably due to not defining the __all__ variable in the target files.
This bug effects import statements Metrics.py and RepoCentricMeasurements.py.
RepoCentricMeasurements.py, UserCentricMeasurements.py, and plots.py all are missing the __all__ variable and other files attempt to import them using the wildcard import.
This bug may depend on the version of python used.  I tested using python 3.5.0, but documentation indicates that all recent versions of python are similarly effected.

Hello,
Would you mind posting the exact error that is thrown out?
Best,
Josh M

This bug should now be fixed. Python3 does not allow importing within functions. Plots.py was calling an import in one of the methods.

Here's an example. The same error appears everywhere there is an import *, including from itertools import * in plots.py line 186
Traceback (most recent call last):
File "evaluation/Metrics.py", line 11, in
from RepoCentricMeasurements import *
File "/gpfs/home/j/m/jmbollen/Karst/github_simulation/evaluation/RepoCentricMeasurements.py", line 6, in
import plots
File "/gpfs/home/j/m/jmbollen/Karst/github_simulation/evaluation/plots.py", line 169
def plot_distribution_of_events(data,weekday,loc=False):
SyntaxError: import * only allowed at module level

Thanks for the prompt response.


java.lang.AbstractMethodError  when adding " builder.addPlugin(new CrawlOverview())"

Hi,
I am evaluating different crawlers for Java, and I gave a try to Crawljax. However, when I try to use its overview plugin I get the following exception:
java.lang.AbstractMethodError: com.crawljax.plugins.crawloverview.CrawlOverview.preStateCrawling(Lcom/crawljax/core/CrawlerContext;Lcom/google/common/collect/ImmutableList;Lcom/crawljax/core/state/StateVertex;)V
at com.crawljax.core.plugin.Plugins.runPreStateCrawlingPlugins(Plugins.java:286)
at com.crawljax.core.Crawler.crawlIndex(Crawler.java:457)
at com.crawljax.core.CrawlTaskConsumer.crawlIndex(CrawlTaskConsumer.java:85)
at com.crawljax.core.CrawlController.call(CrawlController.java:66)
at com.crawljax.core.CrawljaxRunner.call(CrawljaxRunner.java:37)

I am using Java 8 on Windows 7. From Maven, I have:
    <dependency>
        <groupId>com.crawljax</groupId>
        <artifactId>crawljax-core</artifactId>
        <version>3.6</version>
    </dependency>
    <dependency>
        <groupId>com.crawljax.plugins</groupId>
        <artifactId>crawloverview</artifactId>
        <version>1.2</version>
    </dependency>

I am running Crawljax with:
CrawljaxConfiguration.CrawljaxConfigurationBuilder builder = CrawljaxConfiguration.builderFor(getBaseUrl());
builder.setMaximumRunTime(MAX_MINUTES, TimeUnit.MINUTES);
builder.setMaximumDepth(50);
builder.setMaximumStates(2000);
    builder.setBrowserConfig(new BrowserConfiguration(EmbeddedBrowser.BrowserType.CHROME, 1));
    builder.crawlRules().dontClick("a").withAttribute("class", "image-link");

    builder.crawlRules().waitAfterReloadUrl(1000, TimeUnit.MILLISECONDS);
    builder.crawlRules().waitAfterEvent(200, TimeUnit.MILLISECONDS);

    builder.addPlugin(new CrawlOverview());  //this does crash

    InputSpecification input = new InputSpecification();
    //be sure we can login
    input.field("username1").setValue(name);
    input.field("password1").setValue(password);
    builder.crawlRules().setInputSpec(input);

    CrawljaxRunner crawljax = new CrawljaxRunner(builder.build());
    try {
        crawljax.call();
    } finally {
        crawljax.stop();
    }

36 expects Java 7.

If you need to run it with Java 8 on Windows, try building the code from the master branch using Maven:
 mvn package -DskipTests

(We have noticed that some test cases fail on Windows due to environmental settings, hence -DskipTests, but the build should work fine)

Hi,
thanks for the quick response. I am using 3.6 as it seems the latest stable release from Maven Central:
http://mvnrepository.com/artifact/com.crawljax/crawljax-core
however, it looks like it is 1 year old. Is there any new release planned for the future?
Compiling from source is doable, but quite cumbersome, as I would need to install it on many different machines (eg colleagues' desktops and remote Jenkins/Bamboo). It can be installed on a corparate Nexus, but I would need to look into it. However, something to consider in the future if the other crawlers I am evaluating do not work well :)
thanks
Andrea

No new release planned for anytime soon. We are however working on a cloud-based version, which  removes the need to build/install/etc.

I see. That is unfortunate. Anyway, thanks for the clarification.

Hi,
When I tried the above and still tried to set the jdk to 7, the same problem exists.
thanks,
Santy


Mute SLACK items such as "reacts @"  and "unreacts @" from being sent to an IRC client

Slack sends information used in their clients & web interface, but that info is not useful in IRC.  This information should be muted and not forwarded to the IRC client.  The user name associated with these messages is unique - in Textual it begins with a dot and begins with an asterisk in HexChat.
Examples of these items in IRC (Textual client) are shown here - https://postimg.cc/image/jhv3ie19j/
The same items shown in Slacks web client - https://postimg.cc/image/ydtmq3upj/

Hey there @Hack5190. I added the react/unreact messages intentionally. They are not directly from the Slack API. They will hopefully be more useful with #38. But also I will make a note to include an option to disable them entirely (#34).

Understood, thanks.  FYI: I added a patch to lib/ircd.js in my fork as a temporary fix.

Added in 20d7bc6


Swap out openshift client for kubernetes client

All the functionality provided by the openshift client library has been upstreamed to the kubernetes client library, so it's no longer necessary for both of these.
This also switches community.kubernetes to kubernetes.core as the former has been deprecated for some time now. This change only affects the testing and is a drop in replacement.

Looks good to me too! Thank you so much for this PR!!


Error Encountered When Trying To Import azure.cognitiveservices.speech

I am going through this exercise on Microsoft site (https://docs.microsoft.com/en-gb/learn/modules/transcribe-speech-input-text/5-exercise-convert-speech-from-audio-file?pivots=python)
Using:

Visual Studio Code
Python version - Python 3.7.4 (also have Python 3.8.5 installed)
MacOS

Error Encountered:
Run Python in the terminal
/usr/local/bin/python3 /Users/comp/Python/SpeechToTextPython/speechtotext.py
The default interactive shell is now zsh.
To update your account to use zsh, please run chsh -s /bin/zsh.
For more details, please visit https://support.apple.com/kb/HT208050.
(base) $ /usr/local/bin/python3 /Users/userid/Python/SpeechToTextPython/speechtotext.py
Traceback (most recent call last):
File "/Users/userid/Python/SpeechToTextPython/speechtotext.py", line 1, in
import azure.cognitiveservices.speech as speechsdk
ModuleNotFoundError: No module named 'azure'
To Reproduce
Steps to reproduce the behavior:
As above.
Expected behavior
As stated in the exercise.
Version of the Cognitive Services Speech SDK
Requirement already up-to-date: azure-cognitiveservices-speech in /Users/userid/opt/anaconda3/lib/python3.7/site-packages (1.13.0)
Platform, Operating System, and Programming Language

OS: MacOS
Programming language: Python
Browser: Chrome

Really appreciate help here.

Hi @terencebeh, it looks like the Python interpreter you are using (/usr/local/bin/python3, which may be from Homebrew) is not the one where the speech sdk is installed (judging from this path /Users/userid/opt/anaconda3/lib/python3.7/site-packages).  Please try to either explicitly use the anaconda interpreter for the sample, or install the Speech SDK package with the pip in /usr/local/bin/pip3.

@terencebeh Was your issue resolved by the suggestions? Please update.

Closed as answered. Please create a new issue if you need further support on any specific topics.


Feat/elasticache

Adds metadata caching via redis cache and AWS elasticache

Added metadata caching via redis cache and AWS elasticache.
Use fakeredis for cache in tests.
Remove starlette-cramjam CompressionMiddleware.
Address more cases of protocol/engine combinations in reader.py#get_filesystem.
Moved cftime and pandas requirements from Dockerfile to pyproject.toml.

The deployment is accessible here: https://feature-titiler-xarray.delta-backend.com/map and https://feature-titiler-xarray.delta-backend.com/api.html
performance results compared against dev are reviewable here: https://github.com/developmentseed/tile-benchmarking/blob/dcbeb00189837d443014541f0133082342749e53/03-e2e/compare-dev-feature.ipynb

I'm going to merge this so it doesn't get stale but happy to incorporate any feedback to come in later.


Add signature fields to compressed tx form

As we concluded in the call Monday, we want to add a signature field to the transaction fields of the compressed form.
We'll use Transfer as example
Before
Currently, the struct Tx.Transfer looks like
    struct Transfer {
        uint256 senderID;
        uint256 receiverID;
        uint256 amount;
        uint256 nonce;
    }

And it will be compressed to [sender_state_id<4>|receiver_state_id<4>|amount<4>|nonce<4>]
This means for ECDSA version we had to commit signatures separately to submitBatch, and we need to merklize/hashchain the signatures together and add a field in the Batch.
After
We add the signature field in Tx.Transfer
    struct Transfer {
        uint256 senderID;
        uint256 receiverID;
        uint256 amount;
        uint256 nonce;
+     bytes signature;
    }

And it will be compressed to [sender_state_id<4>|receiver_state_id<4>|amount<4>|nonce<4>|signature<64>]
So the signature will be committed with the transaction itself. No extra merkle/hashchain/field in Batch.

Being Implemented in #111

Fixed in #111 and #114


Update HelloWorld.java

Description

Types of changes

[x] Bug fix or improving code quality
[ ] New language
[ ] New feature

Checklist:

[ ] My code follows the code style of this project.
[ ] I have updated the documentation accordingly.

Better to use print along with a backslash


Convert sen1floods11 Dataset

Connects https://github.com/azavea/noaa-flood-mapping/issues/22

Permanent Water

Imagery

Labels
Sentinel-2 Water
Imagery

Labels


Navigating to an element without a README description does not remove the old README description of the previous element

Steps to reproduce:

Go to an element page, for example https://beta.webcomponents.org/element/LasaleFamine/polymer-twitter-timeline asdf
Load an element without any description

In the search bar enter lazyload
Click lazyload-image

Expected behavior
Message that there is no README.md (which you see when you directly go to https://beta.webcomponents.org/element/1000ch/lazyload-image)
Actual behavior
The old description of polymer-twitter-timeline is shown

Probably this line should be changed to readme === null || readme === undefined || !data


"External component has thrown an exception." for SudokuGrover running 4x4-classic

Running SudokuGroverSample with argument 4x4-classic.  Classic solution runs well and quick.  Tried solving this with quantum,  consistently get:
System.Runtime.InteropServices.SEHException
HResult=0x80004005
Message=External component has thrown an exception.
Source=Microsoft.Quantum.Simulators
StackTrace:
at Microsoft.Quantum.Simulation.Simulators.QuantumSimulator.AllocateOne(UInt32 id, UInt32 qubit_id)
at Microsoft.Quantum.Simulation.Simulators.QuantumSimulator.QSimQubitManager.Allocate(Boolean usedOnlyForBorrowing)
at Microsoft.Quantum.Simulation.Common.QubitManager.Allocate(Int64 numToAllocate)
at Microsoft.Quantum.Simulation.Common.SimulatorBase.Allocate.Apply(Int64 count)
at Microsoft.Quantum.Samples.ColoringGroverWithConstraints.ApplyVertexColoringOracle.<get___Body_>b__31_0(ValueTuple6 __in__) in C:\Users\lfenster\OneDrive - Microsoft\Repos\Quantum\Quantum\samples\algorithms\sudoku-grover\ColoringGroverWithConstraints.qs:line 146    at Microsoft.Quantum.Simulation.Core.OperationPartial3.<get___Body__>b__22_0(P a)
at Microsoft.Quantum.Simulation.Core.Operation`2.Apply(I a)_
To Reproduce
Steps to reproduce the behavior:

Go to SudokuGroverSample project.
After code for running puzzle4 with classic, add code to solve this same puzzle with Quantum

int[,] puzzle4Q = {
                    { 0,2,0,4 },
                    { 3,0,0,2 },
                    { 0,3,0,1 },
                    { 4,0,2,0 } };
                Console.WriteLine("Solving using quantum computing.");
                ShowGrid(puzzle4Q);
                resultFound = sudokuQuantum.QuantumSolve(puzzle4Q, sim).Result;
                VerifyAndShowResult(resultFound, puzzle4Q, answer4);

Make sure Project properties have 4x4-classic set as Application Arguments
Run in debug
See error

Expected behavior
Solved quantumly with same result as classic
Additional context
This may be a memory issue.  According to VS, over 24gigs of memory usage at time of exception.  Snapshot below:

Yes, this definitely is a memory issue. The sudoku puzzle in this example has 8 missing digits, with 2 bits per digit the algorithm will need 16 qubits to represent the search space alone. With the extra qubits allocated during the execution to check the constraints, one per constraint, the number of qubits used in the oracle implementation is over 50, which is a lot more than the simulator can handle. The hardcoded examples used with the quantum solution have fewer missing digits and fewer constraints, so they don't run in a similar memory issue.
It would make sense to update this example to add some checks on the total number of qubits that will be used, and to fail early with a cleaner error message when the number of qubits is over 30. @cgranade, what do you think?

Thanks @tcNickolas.  this is what i suspected too.  I wonder if there is an additional way to address some of these kinds of situations.  It occurs to me that the number of starting constraints + number of possible values = size (e.g., 4 for a 4x4, 9 for a 9x9 puzzle) * empty vertices.  So, in the case I outlined above, there are 8 vertices which equate to 32 values all in all; 22 starting number constraints are known...leaving a total of ten possible values left for the 8 vertices (6 have 1 possible value and 2 have 2 each).  In the case where starting constraints is greater than 1/2 * empty vertices * size, could the puzzle be solved with possible values instead of starting constraints?  In this case, it would use 10 instead of 22..
I've played around with this and I think it would need something similar to the ConstrainByEdgeAndStartingColors function but for PossibleValues instead of constraints.  Instead of checking if the numbers are the same, it would check to see if the value is one of the possible values; like an == (1 OR 3).  I think it would replace these lines of code
  for ((cell, value), conflictQubit) in Zipped(startingColorConstraints, startingColorConflictQubits) {
            // Check that cell does not clash with starting colors.
            ControlledOnInt(value, X)(
                colorsRegister[cell * bitsPerColor .. (cell + 1) * bitsPerColor - 1],
                conflictQubit
            );

But I am still coming up to speed with Quantum and not sure what the code would look like (or if its even possible) to change the conflictQubit based on whether the value is equal to a set of possible values.
thanks again for your help

I finally had the time to dig into this question, and this optimization alone, while reducing the number of qubits necessary to evaluate the same constraints, doesn't bring it down enough to solve your puzzle. I had to do something cleverer, incorporate this type of constraints into the search space construction (prep of the initial state and the reflection about the mean step) so as to avoid checking them as constraints at all. I even wrote a blog post about this!
@msoeken, @cgranade - does it make sense to bring those optimizations into our official sample, so that it could solve larger and more interesting puzzles?

yes, i tried it out and you are correct  - it doesnt bring it down enough.  I have other ideas (intersections of rows, columns, blocks) but got pulled into other work and havent been able to look at this in a bit.  i plan to try some other thoughts when i get time again

You couldn't have known about the blog post, I've published it less than 12 hours ago :-)


Set scope highlight by ibl.setup_buffer

Problem
rainbow-delimiters could be disabled for specific buffers, it would be better if indent-blankine's scope highlight is buffer-scoped so that we can only apply rainbow-delimiters' highlights to the buffers it enabled, and keep IblScope in the buffer rainbow-delimiters does not enabled.
Expected behavior
scope highlight is buffer-scoped and could be set by ibl.setup_buffer

While it would be possible with namespaced highlight groups, this would be quite complicated to implement, I think.
I'm not against merging a PR, but I don't think I will work on this myself.


cant export exr files!

Original ticket:
https://sourceforge.net/p/djv/support-requests/12/

I've decided to limit the scope of DJV and not include export functionality.


i18n(de): translate components/tabs

Description
Translation of components/tabs.mdx into German language.

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

de
components/tabs.mdx
Localization added, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

By the way: Hugh thanks to you for reviewing all my PRs, we are close to having 100% for Starlight German translation again!
I'm hyped because it kinda feels like a 100% speedrun xD

Well, huge thanks to you for all the hard work and it's great to see German rising up the charts again. I'm not the fastest to review German and some PRs may require some help/validation from Chris, but it's getting there :rocket:

Yeah, no hurry, some breaks are also very important, especially at these times... I mean after midnight. At least for me it's better to take it slowly instead of rushing through it and making more mistakes on my way...


[Masthead (Storage)]: dds-masthead using data endpoint from "Storage" translation renders EN version on soft refresh in FF

Description
When a translated version of the Storage page, which uses the Global Masthead "Storage", the links of the masthead appear in English instead of the desired language, but only on Firefox. Chrome and Safari work as expected.
Examples:
https://www.ibm.com/es-es/storage
https://www.ibm.com/fr-fr/storage
https://www.ibm.com/jp-ja/storage
Occasionally, on first load, they render correctly on Firefox, but a soft refresh or navigate away and back renders them in English.

Proposed Workaround solution

For (https://www.ibm.com/jp-ja/storage), currently, the API call for the translation file is trying to query for usja.json which doesn’t exist within the Storage translation repo (https://github.ibm.com/webstandards/ibm-dotcom-translations/tree/master/translations/storage-masthead).
A workaround for this issue that we can look into is to duplicate jpja.json for usja.json.

Component(s) impacted

dds-masthead
Masthead translation API

Browser
Firefox
Carbon for IBM.com version
v.1.35.0
Severity
Severity 3 = The problem is visible or noticeable to users but does not impede the usability or functionality. Affects minor functionality, has a workaround.
Application/website
Drupal
Package
@carbon/ibmdotcom-web-components, @carbon/ibmdotcom-services
CodeSandbox example
https://www.ibm.com/es-es/storage
Steps to reproduce the issue (if applicable)
No response
Release date (if applicable)
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I checked the current issues for duplicate issues

@proeung Does this address this issue (bullet #2)?


Error decoding pickle: too many values to unpack

I have previously used the plaintext protocol to feed data into carbon. However since I send a lot of data every minute this has proven ineffective since it occupies too many ports. So I decided to try to use the pickle protocol instead. However when I try to send the data with the pickle protocol I get the following errors in /var/log/carbon/listener.log:
09/11/2017 14:41:19 :: Error decoding pickle: too many values to unpack
09/11/2017 14:41:19 :: MetricPickleReceiver connection with <my other server's ip> lost: local variable 'value' referenced before assignment

I have looked at the issue #421. However, since the lines regarding CARBONLINK_HOSTS are out commented in my local_settings.py (and therefore not present in the listing of my local_settings.py below) I do not think that is the issue in my case.
Any help or tips is appreciated!
Below you can see my carbon.conf, local_settings.py and the script used to send the data to my carbon/graphite server.
carbon.conf
STORAGE_DIR    = /var/lib/graphite/
LOCAL_DATA_DIR = /var/lib/graphite/whisper/
CONF_DIR       = /etc/carbon/
LOG_DIR        = /var/log/carbon/
PID_DIR        = /var/run/

ENABLE_LOGROTATION = False

USER = _graphite

MAX_CACHE_SIZE = inf

MAX_UPDATES_PER_SECOND = 500

MAX_CREATES_PER_MINUTE = 50

LINE_RECEIVER_INTERFACE = <IP_ADDRESS>
LINE_RECEIVER_PORT = 2003

ENABLE_UDP_LISTENER = False
UDP_RECEIVER_INTERFACE = <IP_ADDRESS>
UDP_RECEIVER_PORT = 2003

PICKLE_RECEIVER_INTERFACE = <IP_ADDRESS>
PICKLE_RECEIVER_PORT = 2004

LOG_LISTENER_CONNECTIONS = False

USE_INSECURE_UNPICKLER = False

CACHE_QUERY_INTERFACE = <IP_ADDRESS>
CACHE_QUERY_PORT = 7002

USE_FLOW_CONTROL = True

LOG_UPDATES = False
LOG_CACHE_HITS = False
LOG_CACHE_QUEUE_SORTS = True

CACHE_WRITE_STRATEGY = sorted

WHISPER_AUTOFLUSH = False

WHISPER_FALLOCATE_CREATE = True

[relay]
LINE_RECEIVER_INTERFACE = <IP_ADDRESS>
LINE_RECEIVER_PORT = 2013
PICKLE_RECEIVER_INTERFACE = <IP_ADDRESS>
PICKLE_RECEIVER_PORT = 2014

LOG_LISTENER_CONNECTIONS = False

RELAY_METHOD = rules

REPLICATION_FACTOR = 1

DESTINATIONS = <IP_ADDRESS>:2004

MAX_DATAPOINTS_PER_MESSAGE = 500
MAX_QUEUE_SIZE = 10000

QUEUE_LOW_WATERMARK_PCT = 0.8

USE_FLOW_CONTROL = True

[aggregator]
LINE_RECEIVER_INTERFACE = <IP_ADDRESS>
LINE_RECEIVER_PORT = 2023

PICKLE_RECEIVER_INTERFACE = <IP_ADDRESS>
PICKLE_RECEIVER_PORT = 2024

LOG_LISTENER_CONNECTIONS = True

FORWARD_ALL = False

DESTINATIONS = <IP_ADDRESS>:2004

REPLICATION_FACTOR = 1

MAX_QUEUE_SIZE = 10000

USE_FLOW_CONTROL = True

MAX_DATAPOINTS_PER_MESSAGE = 500

MAX_AGGREGATION_INTERVALS = 5

local_settings.py
SECRET_KEY = <my secret key>
TIME_ZONE = 'Europe/Stockholm'

LOG_RENDERING_PERFORMANCE = True
LOG_CACHE_PERFORMANCE = True
LOG_METRIC_ACCESS = True

CONF_DIR = '/etc/graphite'
STORAGE_DIR = '/var/lib/graphite/whisper'
CONTENT_DIR = '/usr/share/graphite-web/static'

WHISPER_DIR = '/var/lib/graphite/whisper'
LOG_DIR = '/var/log/graphite'
INDEX_FILE = '/var/lib/graphite/search_index'

DATABASES = {
    'default': {
        #'NAME': '/var/lib/graphite/graphite.db',
        #'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'graphite',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'graphite',
        'PASSWORD': '<my password>',
        'HOST': '<IP_ADDRESS>',
        'PORT': '3306',
        'STORAGE_ENGINE': 'INNODB'
    }
}

script used on server to send data
where sys.argv is data on the following format:
[ (buses.bus.501.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)), (buses.bus.313.is_up,<PHONE_NUMBER>, 1)) ]

#!/usr/bin/python

import sys
import pickle
import struct
import socket
from time import sleep

sys.argv.pop(0)
print 'Argument List:', str(sys.argv)
payload = pickle.dumps(sys.argv, protocol=2)
header = struct.pack("!L", len(payload))
message = header + payload

print str(message)

sock = socket.socket()
sock.connect( ("<my carbon/graphite server's ip>", 2004) )
sock.send(message)
sleep(5)
sock.close()

Hi @deniszh!
The reason is that when using the plaintext protocol I end up with the following (output from netstat -anp on the server which sends data to my carbon/graphite server):
tcp        0      0 <IP_ADDRESS>:58536        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:32833        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:58526        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:33476        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:60308        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:55685        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:56596        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:55811        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:59064        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:57906        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:58097        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:55726        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:58051        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:33779        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:59611        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:58452        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:58653        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:60054        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:58986        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:57537        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:56769        <my carbon/graphite server's ip>:2003      TIME_WAIT   -
tcp        0      0 <IP_ADDRESS>:58414        <my carbon/graphite server's ip>:2003      TIME_WAIT   -

And this is just a very small portion of it.
when using the plaintext protocol I use the following code:
echo buses.bus.321.is_up 0 date +%s | nc -q0 <my carbon/graphite server's ip> 2003
if you have any idea how to solve this issue instead I would gladly abandon the pickle protocol ... :)

echo buses.bus.321.is_up 0date +%s| nc -q0 <my carbon/graphite server's ip> 2003
opens a new TCP socket (connection) for every metric. Instead open the socket once and send all metrics at once. See example-client.py as to see how to do this from Python.

Not sure which issue do you mean, @AmyNgithub
Many connections to graphite server? Does it cause issues?
Then open a single connection and push metrics there. Of course, you need to write some python code for it, but you need to write some python code for pickle sender too. For example, check this link - https://coreygoldberg.blogspot.nl/2012/04/python-getting-data-into-graphite-code.html
Performance-wise pickle is not faster than line proto too.
Or you can try to enable net.ipv4.tcp_tw_reuse sysctl - sysctl -w net.ipv4.tcp_tw_reuse=1
There's quite big but excellent article about TIME_WAIT issue - https://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux

Ah, please check @piotr1212 answer - we have a code example in repo too. :)

sys.argv is going to be a list of strings, not a list of tuples.  Can you paste the actual output of your script?

Ah, and we have open issue for having some bulk import tool - https://github.com/graphite-project/carbon/issues/612

Thank you @piotr1212! Now I got a working solution! Also thank you @deniszh for putting me on the path to stick with the plaintext protocol :)


Add item help

This is part of my quest to add in-game help to everything.
This PR adds help texts for the items and tools (except spawn eggs and mob spawner), which will be used by the Help modpack: https://forum.minetest.net/viewtopic.php?f=9&t=15912&p=240152
This improves the mod's self-documentation.
doc_items becomes an optional dependency.
Some item names (description) are changed accordingly. “Nametag” becomes “Name Tag” to fix spelling.
The description of net and magic lasso are changed to just “Net” and “Magic Lasso” if doc_items is detected because the usage help is moved to the help system.
Important! Please proofread all the help texts I have written and check if they describe the items accurately. If I made a mistake or I forgot to mention something, please complain before merging. If everything is correct, please confirm this.
You can test this PR by installing the Help modpack yourselves, start the game, type in /help_reveal, then open the help via /helpform and navigate to the items which this mod registers.

Hmmm.
I can't remember why this has been closed. Care to explain?


Add config parameter to change per-container stop timeout during daemon shutdown

This fix tries to add a StopTimeout in container's Config so that a per-container stop timeout could be specified during daemon shutdown.
The StopTimeout is defined as *int so that unspecified value could be correctly identified as using the default value (10s).
Note: Both 0 and -1 could not be used as the default value because 0 means stop immediately and -1 means wait forever.
This fix is related to docker:
https://github.com/docker/docker/pull/22566
Signed-off-by: Yong Tang<EMAIL_ADDRESS>

LGTM 🐮

LGTM

"Make the zero-value useful"
Could we please stop adding needless pointer values everywhere? This is getting tiresome. 1ns is just as good as zero and doesn't cause all kinds of havoc.


R Version in Singularity Image

Describe the bug
I'm trying to get xcpEngine running with a singularity image for the first time and it's telling me that my version of R is wrong. It looks like it's using 3.3 from within the image but our server has version 3.6, which is also seems to be finding. Any help would be appreciated!
Error message
Paste your error message between the backticks
Error message
Checking general dependencies
····································································
· Version AFNI           AFNI_19.1.18
· Version ANTs           <IP_ADDRESS>
· Version FSL            fsl-5.0.10
· Version C3D            1.1.0
· Version JQ             jq-1.5
· Version XCP Engine     xcpEngine-v1.0
· R version              3.3.3
· R scripting front-end  3.3.3
Failed with error:  ‘cannot read workspace version 3 written by R 3.6.0; need R 3.5.0 or newer’
· [Dependencies check failed: RNifti]
· [Please add RNifti to your R installation]
· · RNifti version
· · optparse version
· · pracma version
· · signal version
[Aborting]

yes you need  to include -e or  --cleanenv  with singularity  to clean environment before running container. If it doesnt work, you can temporarily delete .Rprofile in your home directory


Add LatencyTracker to Session

Motivation
Required e.g. by #505, it would enable updates of all entities that depend on latencies, including LatencyAwarePolicy.
Proposed solution
We could create a struct with at least minimum data required for latency tracking:
struct LatencyInfo {
    node: Arc<Node>,
    latency: Duration,
}

And then embed a tokio::sync::broadcast in Session:
pub struct Session {
    ...
    latency_tracker: tokio::sync::broadcast::Sender<LatencyInfo>,
}

...enable subscribing for it:
impl Session {
    ...
    pub fn subscribe_for_latency_tracking(&self) -> tokio::sync::broadcast::Receiver<LatencyInfo> {
        self.latency_tracker.subscribe()
    }
}

... and then the latency tracking info would be broadcast from here:
session.rs:1211:execute_query():
last_error = match query_result {
  Ok(response) => {
      trace!(parent: &span, "Query succeeded");
      let elapsed = query_start.elapsed();
      self.latency_tracker.send(LatencyInfo{
          node,
          latency: elapsed,
      });
      let _ = self
          .metrics
          .log_query_latency(elapsed.as_millis() as u64);
      return Some(Ok(response));
  }
  ...
};

The LatencyAwarePolicy would be created using the subscribed broadcast::Receiver as an argument.
What do you think about such a solution?
@psarna @piodul @cvybhu

How about keeping an (optional) latency tracker within each Node instance? We keep a very limited number of these in memory (since clusters tend to have only a few nodes), and then we won't need any mechanism for broadcasting and subscribing, we could just update the latencies in-place, we have access to node in the code snippet you quoted.

How about keeping an (optional) latency tracker within each Node instance? We keep a very limited number of these in memory (since clusters tend to have only a few nodes), and then we won't need any mechanism for broadcasting and subscribing, we could just update the latencies in-place, we have access to node in the code snippet you quoted.

What do you mean by a "latency tracker within each Node instance"? The LatencyAwarePolicy requires to be notified on each query completion in order to update internal statistics, and if we just store the last latency info within Node, it won't be able to trigger the policy update accordingly - that's why I would stick to a broadcast.
Moreover, we would probably prefer to give the user ability to implement arbitrary logic based on subscribed latency tracking.

@psarna Let's resume the discussion, as I'm back working on this issue.

@piodul @cvybhu I would appreciate your opinions too.

I was sure I posted a response a few days ago, but maybe I didn't hit the Comment button... so let me start over:
Notifying LatencyAwarePolicy is one way, but LatencyAwarePolicy could also ask nodes for their fresh latency information right before making a decision. Then, nodes update their latency info locally when taking part in query processing, and the information is only gathered once needed - i.e. when LatencyAwarePolicy needs to make a decision. I think that this way may make sense, because it delays aggregating computation results until they're needed, and avoids keeping another bunch of atomically refcounted pointers to Node instances. Let's assume we have a token-aware latency-aware policy. Token awareness would reduce the candidates to 3 nodes, and out of these 3 nodes, the latency-aware policy will choose a subset with predicted best latency by examining the nodes' latency information right before making the choice.

I'm not sure if we can just take the latencies of user queries and use them for detecting local DCs and stuff like that.
Some queries might take a long time because of the query itself, not the network latency.
Maybe we could spawn a latency tracker worker for each node, which would perform a simple query every 60s. This could even be a query that fails to parse, just something to ping the database.
Actually doesn't the code for keeping connections alive do something like this?
Then each node would have some atomic float representing the latency which could be used by latency aware policies.
I agree that the broadcast approach looks overcomplicated, does the policy really need to know how each query went to make reasonable choices? Couldn't it just be provided the latency by a third party?

@wprzytula could you do a small r&d research and look how it's implemented in Java, and perhaps gocql? I generally agree that latency depends heavily on the query itself, but on the other hand workloads are usually rather homogenous wrt. query type. a 60s ping may be counterproductive, because I imagine that people who use LatencyAware policy expect it to react reasonably fast to changes - e.g. when a node starts showing symptoms of overload.

@wprzytula could you do a small r&d research and look how it's implemented in Java, and perhaps gocql?

The cpp driver might be worth looking into, too.

First conclusion: Java driver's way to latency-aware routing is similar to the solution I proposed (and that's how it came into my mind). On init, LatencyAwarePolicy register a LatencyTracker in the cluster, and it's a rather lightweight store of an exponential moving average (refer to Tracker and HostLatencyTracker in LatencyAwarePolicy.java). The Tracker is updated each time the query completes (successfully or with an error) by RequestHandler, and when a query plan is computed, each node requires a separate read from a AtomicReference (which seems to be a typical Java-devs manoeuvre...).

I was sure I posted a response a few days ago, but maybe I didn't hit the Comment button... so let me start over:
Notifying LatencyAwarePolicy is one way, but LatencyAwarePolicy could also ask nodes for their fresh latency information right before making a decision. Then, nodes update their latency info locally when taking part in query processing, and the information is only gathered once needed - i.e. when LatencyAwarePolicy needs to make a decision. I think that this way may make sense, because it delays aggregating computation results until they're needed, and avoids keeping another bunch of atomically refcounted pointers to Node instances. Let's assume we have a token-aware latency-aware policy. Token awareness would reduce the candidates to 3 nodes, and out of these 3 nodes, the latency-aware policy will choose a subset with predicted best latency by examining the nodes' latency information right before making the choice.

But what kind of latency information do you suggest storing inside Node ? An exponential moving average? Last n latencies?

If Java uses a lightweight exponential moving average, perhaps we can assume it works well enough for them and do the same. Each node could keep that information internally, it would get updated on every query, and LatencyAware policy would ask for that information and use it when deciding which replicas to pick - in particular, if it's wrapped in TokenAware policy, it would only do so for nodes that own the particular data.

How would we decide that such per-node latency information shall be collected? Type-casts with Any trait to the particular LatencyAwarePolicy seems to be a big no-no, so perhaps we could add a new method to RetryPolicy trait: requires_latency_measurements() -> bool, which would set up session accordingly for measuring latencies or not.

Sounds good to me. The alternative is to collect unconditionally (and then expose the information via metrics), but that may be a little too costly in terms of performance, and it's hard to guess without  benchmarks

As such information (whether a policy requires latency stats collection) is constant per policy, I believe it could be done better in Rust using enums. One option to do so would be to make RetryPolicy an enum of two variants: WithLatencyMeasurements and WithoutLatencyMeasurements, each containing a Box, where TRAIT={RetryPolicyWithLatencyMeasurements, RetryPolicyWithoutLatencyMeasurements}. Although beauty of such solution is debatable, it would make better utilization of Rust type system instead of moving responsibility to runtime checks. WDYT?

I'd go with a runtime check, under the assumption that most users won't be interested in latencyaware routing, so it's better to keep the code prettier for the common case. @piodul, @cvybhu?

I agree with @psarna.

I'd go with a runtime check, under the assumption that most users won't be interested in latencyaware routing, so it's better to keep the code prettier for the common case. @piodul, @cvybhu?

Runtime check sounds good. What if in the future we were to introduce other metrics for nodes like error rate, requests/s etc.
We would have to exponentially many enum variants.

I guess collecting the latency of actual queries instead of network might have its own benefits. When a node becomes overloaded its latency would go up and we could limit the amount of queries it gets.


Process replay segment regen TODOs

These improvements are important to make sure test_processes actually tests realistic cases.
https://github.com/commaai/openpilot/blob/master/selfdrive/test/process_replay/regen.py

added to #20025


Test: Lightbulb Widget in the Gutter

ref https://github.com/microsoft/vscode/issues/214711

[ ] anyOS @ulugbekna
[ ] anyOS @bamurtaugh

Complexity: 2
Create Issue

Background
There has been a long standing papercut where the lightbulb widget will often spawn over code in the editor and notebook. We mitigate this by trying a best case for each scenario.
Steps

Find some code that does not have indent.
Click or highlight around and see where the lightbulb renders.
Make sure clicking on the lightbulb in the gutter and in the editor actually do something.

example:

Things to verify

When breakpoints are in the gutter, the lightbulb will not spawn there.
When text is blocking above and below, lightbulb will spawn in the gutter
Also try in cases without indents - the widget will still spawn in the best case scenario based on available space.

I guess this is a known issue but a lightbulb still appears on code when there's a breakpoint


Fix memory leak on js! block

Fix https://github.com/koute/stdweb/issues/148
It seems that refcount for JS reference is incremented before serialized to wasm buffer[1], so refcount will be incresed twice if Reference::from_raw_unchecked is used on deserialize.
[1] https://github.com/koute/stdweb/blob/85f7aa484f3be9808b992b9258bd85fa588e05b1/src/webcore/runtime.js#L303-L311

Thanks for the PR!
Hmm... it'd be great to have a unit test for this so that we can avoid reintroducing something like this by mistake in the future. Would you be up to writing one?

As mentioned in the issue, this breaks stable ids, here's code that breaks after this PR.
#[macro_use]
extern crate stdweb;

#[test]
fn test() {
    js! { Module.test = {}; }
    let a = js! { return Module.test; }.as_reference().unwrap().as_raw();
    let b = js! { return Module.test; }.as_reference().unwrap().as_raw();
    assert_eq!(a, b);
}

This can be fixed though by not deleting from ref_to_id_map.

@koute I added testcases. A testcases relies on runtime details, like Module.STDWEB_PRIVATE.id_to_refcount_map. Is it okay?
@kuviman I changed the patch to retain reference ids.

Yeah, depending on the internals in tests is fine.
Also, you might want to get rid of those .swp files you've accidentially git added. (: (You can add *.swp to .gitignore if you want.)

@koute Thanks for the review. I updated the patch, and also rebased the PR to master branch.

LGTM! Although I can't merge this as there is still .runtime.js.swp file added. (:
So please remove that and it'll be good to go!

@koute I missed that one, sorry :(.

No problem!
Thanks! (:


Improve paradigm button styles

Now it looks like this when it's loading:

Codecov Report

Merging #468 into master will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #468      +/-   ##
==========================================
+ Coverage   86.67%   86.70%   +0.03%
==========================================
  Files          68       68
  Lines        2701     2701
==========================================
+ Hits         2341     2342       +1
+ Misses        360      359       -1

Impacted Files
Coverage Δ

CreeDictionary/API/models.py
88.05% <0.00%> (-0.32%)
:arrow_down:

CreeDictionary/utils/fst_analysis_parser.py
94.36% <0.00%> (+1.40%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ac1e69...609fa26. Read the comment docs.


Telnet: bad address 'tcp-echo-cloud-svc.cloudzone'

What happened:
The edge container failed to resolve the domain name, and executing nslookup on the edge node also failed to resolve it. I have reviewed this document（ https://zhuanlan.zhihu.com/p/585749690 ）, but still cannot resolve it. Do you have any suggestions for a solution?
edgenode

cloudnode pod

What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

EdgeMesh version: 1.13.2

Kubernetes version (use kubectl version):  v1.20

KubeEdge version(e.g. cloudcore --version and edgecore --version): v1.13.0

Cloud nodes Environment:

Hardware configuration (e.g. lscpu):
OS (e.g. cat /etc/os-release):
Kernel (e.g. uname -a):
Go version (e.g. go version):
Others:

Edge nodes Environment:

edgecore version (e.g. edgecore --version):
Hardware configuration (e.g. lscpu):
OS (e.g. cat /etc/os-release):
Kernel (e.g. uname -a):
Go version (e.g. go version):
Others:

As long as a domain name can be resolved normally, I understand that the domain name resolution module of edgemesh is normal. Edgemesh's domain name resolution needs to obtain metadata from edgecore's metaserver module, and the metaserver module needs to communicate with cloudcore's syncController module to obtain metadata, such as services and endpoints, etc., edgemesh needs to consume these metadata to provide normal domain name resolution function. If these metadata cannot be sent to the edge node normally, the domain name resolution of edgemesh will be affected.

As long as a domain name can be resolved normally, I understand that the domain name resolution module of edgemesh is normal. Edgemesh's domain name resolution needs to obtain metadata from edgecore's metaserver module, and the metaserver module needs to communicate with cloudcore's syncController module to obtain metadata, such as services and endpoints, etc., edgemesh needs to consume these metadata to provide normal domain name resolution function. If these metadata cannot be sent to the edge node normally, the domain name resolution of edgemesh will be affected.

Thanks, after restarting the edgecore and regenerating the edgemesh agent on the edge, it can now work.


chore(deps): update dependency stylelint to v13.7.2

This PR contains the following updates:

Package
Type
Update
Change

stylelint (source)
devDependencies
patch
13.7.1 -> 13.7.2

Release Notes

stylelint/stylelint
v13.7.2
Compare Source

Fixed: regression for disable commands and adjacent double-slash comments (#4950).
Fixed: use of full file path without converting it to glob (#4931).

Renovate configuration
:date: Schedule: At any time (no schedule defined).
:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.
:recycle: Rebasing: Whenever PR becomes conflicted, or if you modify the PR title to begin with "rebase!".
:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

Available now for Enterprise: Renovate Pro with real-time webhook handling and priority job queue.

:tada: This PR is included in version 6.0.0 :tada:
The release is available on:

v6.0.0
GitHub release

Your semantic-release bot :package::rocket:


NIFI-12491 Added Starting Row Schema Strategy to ExcelReader for 1.x branch

Summary
This feature has been added for the 2.x branch with #9064. This PR is to have it added to the 1.x branch. Per the comments in NIFI-11167  and NIFI-13304 it would seem this feature along with SplitExcel processor will help customers who had used ConvertExcelToCSV in 1.x be able to make a smoother transition from 1.x to  2.x.
NIFI-12491
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[x] Apache NiFi Jira issue created

Pull Request Tracking

[x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[x] Pull Request based on current revision of the main branch
[x] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[x] Build completed using mvn clean install -P contrib-check

[x] JDK ~21~  8

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

@exceptionfactory I cannot seem to restart the MacOs  build (as the current failure has nothing to do with my changes). I  do not see an icon on the side of the build on this page nor on this page which would allow me to restart. Please advise. Thanks!

@exceptionfactory I noticed I was able to restart (i.e. the icon was available) after all the jobs were done. If you changed something thank you!

@exceptionfactory
The MacOS build seems to consistently fail on the test below which I do not think has anything to do with my changes. I have already run this build three times. Please advise.
Error:  Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 25.52 s <<< FAILURE! -- in org.apache.nifi.processors.standard.TestListFile
Error:  org.apache.nifi.processors.standard.TestListFile.testFilterAge -- Time elapsed: 6.549 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <age2.txt> but was: <age1.txt>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1145)
	at org.apache.nifi.processors.standard.TestListFile.testFilterAge(TestListFile.java:330)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at java.util.ArrayList.forEach(ArrayList.java:1259)
	at java.util.ArrayList.forEach(ArrayList.java:1259)

@exceptionfactory The MacOS build seems to consistently fail on the test below which I do not think has anything to do with my changes. I have already run this build three times. Please advise.
Error:  Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 25.52 s <<< FAILURE! -- in org.apache.nifi.processors.standard.TestListFile
Error:  org.apache.nifi.processors.standard.TestListFile.testFilterAge -- Time elapsed: 6.549 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <age2.txt> but was: <age1.txt>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1145)
	at org.apache.nifi.processors.standard.TestListFile.testFilterAge(TestListFile.java:330)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at java.util.ArrayList.forEach(ArrayList.java:1259)
	at java.util.ArrayList.forEach(ArrayList.java:1259)

This is an unstable test, which we adjusted on the main branch, so not a concern for this pull request.


Chat completions support

I'm not entirely sure if this is useful, but the recent instruct templates feature gave me the idea for this new feature. Currently it's enabled with chat mode, but it might be optimal to make a separate checkbox for it.

What's the difference to normal completions? Does the back-end handle adding the correct instruct template?

What's the difference to normal completions? Does the back-end handle adding the correct instruct template?

That's true, but the main point is allowing the use of endpoints that only support chat completions, such as the endpoint for most OpenAI models. However, I guess the OpenAI endpoint isn't great for this because it doesn't support pre-filling.


Text cursor in date input missing on Firefox with jquery versions above 1.11.0 and 2.1.0

Don't know if this is a jquery bug or a webshim bug but when I open http://jsfiddle.net/trixta/VNuct/ in Firefox and use jQuery(edge) instead of jquery-1.11.0 and click on a date input, I can type, but I don't get a text cursor and I can not select any text.

@katzenmayer
Thx, I think I have fixed it in the master branch. I will check later. But you can try your own.

This also should be fixed with 1.15.6-RC1.

Would it be possible to know what was causing the problem? I'm on webshim 1.16 & jQuery v1.12.4 and having the same problem.


make netty direct buffer memory metric

boolean directBufferPreferred = io.netty.util.internal.PlatformDependent.directBufferPreferred();
long usedDirectMemory = io.netty.util.internal.PlatformDependent.usedDirectMemory();
long maxDirectMemory = io.netty.util.internal.PlatformDependent.maxDirectMemory();

cf) if below option is true, directBufferPreferred value is false.
-Dio.netty.noPreferDirect=true

hi @yangbongsoo
It is impossible to develop this function under the present situation.
Each data in Inspector page are based on the data that most of the agent have.
Data that this function will show is depended on Netty.
Therefore, Pinpoint needs to develop additional features to support this function.
I will give you the instructions you need to create it, after creating a new feature.
thanks :)

@yangbongsoo
Please record the data that needs to be collected periodically as follows.
I am not sure how to make it ideal for implementation.
So, I let records save with permisson.
I have defined two AllowedSources for you.
    private static final AllowedSource<LongCounter> NETTY_USED_DIRECT_MEMORY = new AllowedSource<LongCounter>("custom/netty/usedDirectMemory", LongCounter.class);
    private static final AllowedSource<LongCounter> NETTY_MAX_DIRECT_MEMORY = new AllowedSource<LongCounter>("custom/netty/maxDirectMemory ", LongCounter.class);

Registration
Add CustomMetricRegistry customMetricMonitorRegistry in  constructor of Interceptor.
Ex)

before

    public StandardHostValveInvokeInterceptor(TraceContext traceContext, MethodDescriptor descriptor, RequestRecorderFactory<HttpServletRequest> requestRecorderFactory) {

after

    public StandardHostValveInvokeInterceptor(TraceContext traceContext, MethodDescriptor descriptor, RequestRecorderFactory<HttpServletRequest> requestRecorderFactory, CustomMetricRegistry customMetricMonitorRegistry) {

Implemetation
Include a method that periodically returns the value that needs to be extracted by implementing the LongCounter interface.
Ex)

private UsedDirectMemoryCounter implements LongCounter  {

        private final String name;

        public UsedDirectMemoryCounter(String name) {
            this.name = name;
        }

        @Override
        public long getValue() {
		    // to make your own code.
        }

        @Override
        public String getName() {
            return name;
        }

}

If you send Pull-Request, I will check it.
Still, the collector and the web are being implemented, so even if you add them, the desired data may not be immediately visible.
thanks :)


[PHYSICAL_DESIGN] Label generate statements.

A few of the generate statements do not have labels on their begins, which will show up with synthesis dependent keywords. See the BSG SystemVerilog coding guidelines.
https://github.com/black-parrot/pre-alpha-release/blob/3e97c60fe6b7129871d98436f8cfb147c8695dc0/bp_fe/src/v/bp_fe_icache.v#L1

@tommydcjung I think this issue also exists in some of the dcache code.


Updated demo to use MSAL 3.1.0

currently the demo is using version 2.30.0. there have been some major changes to the MSAL-browser library in version 3.1.0

Can you expand on the changes?

Yes sorry. should have included link to migration. I am going to hold off doing any PRs until I hear back about on the bug I filed with them. Possible they have broken the logoutPopup.
https://github.com/AzureAD/microsoft-authentication-library-for-js/issues/6508

Hey, first of all @davestewart thank you so much!
I'm somewhat a beginner and finally managed to implement this in my app.
I added the pull request to add support to 3.1 and fix this issue.
I don't use Typescript so maybe I'll do a PR later to add a 'no-typescript' folder.
Now @MattMinke I have tried with both latest versions of @azure/msal-browser.
I can confirm that before the change it did't work with 3.1 and as you probably observed it did work after:
npm uninstall @azure/msal-browser
npm install @azure/msal-browser@2


step through federation

Sometimes a dataset is displayed on (federated to) an open data portal (www.chattadata.org), but the dataset actually technically lives on another instance (internal.chattadata.org).  We should step through the federation and attempt to download the source metadata and data, only downloading the federated version if the source is not available.
An example:

https://www.chattadata.org/api/views/nvdi-c4tt.json
https://internal.chattadata.org/api/views/nvdi-c4tt.json

We can use the existance of a "sourceDomainCName" key on the metadata response, to tell if it's federated.
We should probably have a CLI flag for this like
socrata-dump --prefer-source

Encountered a complication, "sourceDomainCName" only appears when you have authenticated


Is there a timeline for the planned merge into quickjs?

Some projects considering the compatibility of quickjs-ng will face the implementation of the function after the quikcjs-ng merged into quickjs, due to quickjs-ng added a lot of functionality and header files have been changed a lot, need to do a lot of work in order to be compatible, so whether there is a clear merge into quickjs schedule?
If there is a short term plan (maybe half a year?), I'd like to see it. then the adaptation work will be worthwhile.

We don't have a precise timeline, but there is willingness from both sides. You might have seen @chqrlie contributing to this repo, he is one of the original authors.

Hello,
My plan is to converge by fall '24, so stay tuned.  In any case, the APIs should be identical by then if not sooner.
Best
Chqrlie.

We haven't talked about that yet.

Ah got it, thanks for the quick response! Will monitor this repository for future developments and updates then.

Is it possible to create a todo list so that community members can also participate? Although it may not be possible to submit code directly, a large number of third-party libraries based on qjs need to be updated. Changes to qjs-ng header files and integrated patches may affect the upstream. I believe community members will want to provide some help

Not a bad idea!
We have tried, hard, not to change APIs just because, so unless you are using new APIs they should be source compatible.
I'll try to start a list of the differences.

It sounds like this effort is just getting underway (which is great). I'd like to encourage discussion about the versioning scheme that this will land on. I noticed this repo uses semantic versioning with project releases while the bellard mirror uses a calendar versioning approach with simple downloads.
This can impact community-driven tooling that starts to surface around this (like version managers).
My $0.02: I'd encourage the use of semver with project releases to align with how most other engines/runtimes handle releases.

@coreybutler I agree semantic versioning is preferable. Will work on converging that soon.

See https://github.com/quickjs-ng/quickjs/discussions/258#discussioncomment-11276265 for the latest on qjs vs. qjs-ng


Add init db script

This PR add an init_db script that can be called to set up the database either on an empty state or to incrementally update it from a previous state (e.g. when we add a new table/column etc.)
The script + all other required db information is packaged in the docker container we create on every build so that it can be called as an init container from within our kubernetes setting.
Test Plan
Run /usr/database/init_db.sh postgresql://localhost both on a cold and warm database.

Given @anxolin's suggestion we can look into flyway right away to make future migrations easier.

Re-requesting review as this PR changed quite a bit.

I don't think anything changed.

I forgot to push 🙈

Regarding @anxolin feedback (not related to this PR so maybe better to have on an Issue?)

The FKs are missing, like from trade to its order

I believe the reason for no imposing a FK is that we could potentially see a trade for which we didn't record the order data (in case of data loss, as trades come from on-chain). Right now we still want to store those in the database.

Ideally we would save already the denormalized short id

Can you elaborate on that? Shouldn't the database be usually be normalized if possible?

and I would consider to move those "block_number".

I don't understand what you mean by this. Where do you want to move block_numbers to?

@fleupold I'll remove the feedback from this PR since it's unrelated. I mentioned it since it was the first PR I see regarding the database structure and some things came to mind. I didn't review the schema in detail, those were just a few things that I notice.
Happy to open a discussion issue if that the prefered way.

I would make sure there is a tx hash

@anxolin and @fleupold - I came across the same issue yesterday when I realized that log_index was not transaction_index and concluded that txHash would be important to keep in the trades table. It was my plan for today to make a separate PR for this. However, I think it would be nice to merge the existing Trade API as is before hand.
Once we have txHash, we will have a notion of batchTx per trade. I will get started on this right away if it wasn't intended to include here.

Ok, this was quite a battle, but it seems to be working now (tmate for the win). The crux was that flydb connects via TCP/IP even if we only want to connect to localhost and by default the access mode in postgres for network connections is md5 (which requires to actually know the user's password). psql connects via Unix sockets and it's auth mode is peer (which means it just checks if the user is the current user without caring for a password).
I fixed this by manually setting a password for our test user.
I also added caching of the downloaded file to reduce the time we need to build.


Fix typo in the readme

In my mind this makes more sense.

Thanks for the contribution @rosner ! Cherry-picked into v0.5.0. Closing.

Build fails due to incompatibility between Caddy and golang version, see #158 . This is going to be fixed in v0.5.0 release.

Closing the PR since @rosner hasn't signed CLA yet and 0.5.0 is getting ready for release.


oC10 / oCIS parallel / migration deployment broke because of user authentication

Describe the bug
Users can't login to oCIS in the parallel deployment example since #2528, because the proxy no supports only lookups by username
https://github.com/owncloud/ocis/blob/1ee4198d531191cb6622e345baf093b30e066eed/proxy/pkg/user/backend/cs3.go#L35-L37
but the example deplyoment uses the userid
https://github.com/owncloud/ocis/blob/1ee4198d531191cb6622e345baf093b30e066eed/deployments/examples/oc10_ocis_parallel/docker-compose.yml#L71-L72
Steps to reproduce
Steps to reproduce the behavior:

start the parallel deployment https://owncloud.dev/ocis/deployment/oc10_ocis_parallel/
switch user to oCIS

Expected behavior
User logs in fine
Actual behavior
User can not login

With https://github.com/cs3org/reva/pull/2176, the machine auth accepts an optional claim


Update templates.md

Dummy change to test webhook

🚀 Deployed on https://6696ff35d78a0746695cc17e--cozy-fox-0defec.netlify.app


Training process for multi-GPUs

Hi, I am trying to run training/evaluation with 4 A100s.
However, after some experiments I noticed that the training speed was same compared with process trained with a single GPU.
Am I missing something?

Hello, Jaehoon. I encounter the same problem. I conjecture this is because your Tensorflow package is not installed correctly. I recommend that you should follow the tips provided by https://www.tensorflow.org/install/pip step by step. This maybe helps you solve the problem.

However, after solving the above issue, also as a 4*A100 user, I meet the CUDA out of the Memory issue. Do you encounter this issue for the code in this repository?

Take a look at this. https://github.com/yang-song/score_sde_pytorch/issues/14#issuecomment-1075887846
I solved CUDA memory issue with it.

Ok, thank you very much!

Hi, Jaehoon! Does your training speed on 4A100 improve? After re-checking my experiment, I found it is still quite slow: for each GPU, the utilization rate is around 50%. Do you found another trick to accelerate the training speed or can the author @yang-song provide some advice?


Unable to access login page

So I've run the following commands
sudo docker-compose -f versioneye-base.yml up -d
./versioneye-update
docker exec -it tasks /app/init_enterprise.sh

Navigated to http://localhost:9090, and clicked Sign In but I get redirected to an error page. The page address is http://localhost:9090/signin?utf8=%E2%9C%93 and the error message is The page you were looking for doesn't exist.
Is there a step that I'm missing?

Hi @JamieMagee Sorry for the late response. Is this ticket still current?

Yeah, I still get the same error message.

Hi @JamieMagee. I just tried it by myself. My docker ps -a returns this output:
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                                 NAMES
215387e4bbdf        versioneye/rails_api:2.23.18      "/bin/sh -c 'bundl..."   3 minutes ago       Up 3 minutes        <IP_ADDRESS>:9090->9090/tcp                rails_api
0ef984009ce0        versioneye/crawlj:2.4.27          "/bin/sh -c /mnt/c..."   3 minutes ago       Up 3 minutes                                              crawlj
7bd706ddc9c1        versioneye/crawlr:1.22.0          "/bin/sh -c '/usr/..."   3 minutes ago       Up 3 minutes                                              crawlr
9d012dfc7e3d        versioneye/tasks:2.11.25          "/bin/sh -c '/usr/..."   3 minutes ago       Up 3 minutes                                              tasks
44fe93d197b3        versioneye/crawlj_worker:2.4.13   "/usr/bin/supervisord"   3 minutes ago       Up 3 minutes                                              versioneye_crawlj_worker_1
076b1bf30a58        versioneye/rails_app:5.16.4       "/bin/sh -c 'bundl..."   3 minutes ago       Up 3 minutes        <IP_ADDRESS>:8080->8080/tcp                rails_app
313c699a6538        reiz/rabbitmq:3.4.2               "/bin/sh -c rabbit..."   2 days ago          Up 2 days           <IP_ADDRESS>:5672->5672/tcp                rabbitmq
9cddc8adb5e6        reiz/memcached:1.4.14             "/bin/sh -c '/usr/..."   2 days ago          Up 2 days           <IP_ADDRESS>:11211->11211/tcp              memcached
b27b0da0c18c        versioneye/mongodb:3.4            "/bin/sh -c /start..."   2 days ago          Up 2 days           <IP_ADDRESS>:27017->27017/tcp, 28017/tcp   mongodb
49a7c9112460        reiz/elasticsearch:0.9.1-1        "/bin/sh -c '/elas..."   2 days ago          Up 9 seconds        <IP_ADDRESS>:9200->9200/tcp, 9300/tcp      elasticsearch

I can access the web page on http://localhost:8080. After entering an API key I can login with admin/admin without problems. Please make sure that you have the same docker images and try again.
You tried somehow port 9090. That's the port there the API is running. You can not login there. I would recommend to put an NGinx before that which is proxying and running on port 80. That's how the Vagrant image is build up. Did you try the Vagrant image?

I close this because there is no response anymore. If this problem still exists then please re open this ticket.


Photorealistic Style Transfer

Reference

I want to resolve this issue #60. Please assign this to me.

I want to resolve this issue #60. Please assign this to me.

I want to resolve this issue #60. Please assign this to me.

Assigned for 5 days

I want to resolve this issue #60. Please assign this to me.

Assigned for 5 days

Is it okay???
https://github.com/sagnik1511/Style-Transfer-using-VGG-verydeep-19
please review this.
after that I'm going to create the PR.

Is it okay???
https://github.com/sagnik1511/Style-Transfer-using-VGG-verydeep-19
please review this.
after that I'm going to create the PR.