PolarizedMono12Packed

Hi everyone,
I am working on a grabber using Aravis in linux.
I have a DZK camera set as PolarizedMono12Packed, I am reading the payload directly and try to reconstruct the image of the 4 polarization angles (90,45,135,0). However I am not able to have a "clean" image of each polirization image. I would appreciate a feed back.
Here is the context and assumptions:

Compiler: GCC (any)
C++ 17.
Aravis -> version 0.8, but I guess my issue will happen with any version has it is only used to interface with the camera.
OpenCV any version (since 2.0), I am just using it to store the data.
I am able to set and get informations from the camera (pixel, format, sensor size, payload size, ...).
I am able to read the buffers properly.
The device is started when I try to read the buffer.
I do not receive any assertions from aravis or OpenCV
The camera I am using is a DZK 33GX250

Assuming that I am reading the payload from the last available buffer and return it as an OpenCV's Mat container, which dimensions are: rows <- payload_size, cols <- 1, type <- CV_8UC1, depth <- CV_8U.
As explained in the reference manual, when the camera is set in PolarizedMono12Packed the information of 2 pixels is stored in 3 bytes. In other word 24 bits store 8 values (2 for each polarization angle), which means 3 bits per value.
In order to make this easier I  used the following structure:
`
struct attribute((packed)) polarized_t
{
uchar angle90_1:3;
uchar angle45_1:3;
uchar angle135_1:3;
uchar angle0_1:3;
uchar angle90_2:3;
uchar angle45_2:3;
uchar angle135_2:3;
uchar angle0_2:3;

};
I then try to extract the information using the following function:
std::tuple<cv::Mat, cv::Mat, cv::Mat, cv::Mat> get_polarizations(const cv::Mat& input)
{
int payload = input.rows;

const polarized_t* packed = reinterpret_cast<const test_t*>(input.ptr());

std::cout<<"Distance check: "<<std::distance(reinterpret_cast<const uchar*>(packed), reinterpret_cast<const uchar*>(packed + 1))<<std::endl;

cv::Mat1b tmp = cv::Mat1b::zeros((payload * 2) / 3, 4);

for(size_t i=0; i<tmp.rows; i+=2, ++packed)
{
    tmp(i, 0) = packed->angle90_1;
    tmp(i, 1) = packed->angle45_1;
    tmp(i, 2) = packed->angle135_1;
    tmp(i, 3) = packed->angle0_1;

    tmp(i + 1, 0) = packed->angle90_2;
    tmp(i + 1, 1) = packed->angle45_2;
    tmp(i + 1, 2) = packed->angle135_2;
    tmp(i + 1, 3) = packed->angle0_1;
}

std::vector<cv::Mat> angles;

split(tmp.reshape(4), angles);

return std::make_tuple(angles.at(3), angles.at(1), angles.at(0), angles.at(2)); // 0, 45, 90, 135

}
`
However this approach only produce results such as the following:
Polarization 0:

Polarization 45:

Polarization 90:

Polarization 135:

Raw data:

Hello
I am sorry for my late answer.
From my understanding, you do not unpack the 12bit format correctly, but you simply uses 2 bytes per pixel. I suggest to use
"Polarized Mono 8" first and separate the incoming images with
void CreatePolarImages(int width, int height, _MyData *pData)
{
   if (!pData->mat_created)
   {
		int h = height / 2;
		int w = width / 2;
		for (int i = 0; i < 4; i++)
		{
			pData->grayImage[i].create( h, w, CV_8UC1);
			pData->colorImage[i].create(h , w, CV_8UC4);
			char WindowName[256];
			sprintf(WindowName, "Polarization %d", i + 1);
			cv::namedWindow(WindowName);
		}
		pData->mat_created = true;
	}
}

/////////////////////////////////////////////////////////////////////////
//
void AcquisitionCallback(ArvStream *pStream, void *pVoidData)
{
	_MyData *pMyData = (_MyData *)pVoidData;

	ArvBuffer *pBuffer = arv_stream_try_pop_buffer(pStream);

	if (pBuffer != NULL)
	{
		ArvBufferStatus Status = arv_buffer_get_status(pBuffer);
		if (Status == ARV_BUFFER_STATUS_SUCCESS)
		{
			pMyData->ImageReceived = true;

			int height = arv_buffer_get_image_height( pBuffer);
			int width = arv_buffer_get_image_width( pBuffer);

			pMyData->displayImage.create( height, width,
										   ArvPixelFormat2OpenCVType( pBuffer )
										   );

			size_t buffersize;
			const void* bytes = arv_buffer_get_data(pBuffer, &buffersize);
			memcpy( pMyData->displayImage.data, bytes, buffersize );

			if( strcmp(pMyData->displayWindowName,"") != 0)
			{
				cv::imshow(pMyData->displayWindowName, pMyData->displayImage);
				cv::waitKey(1);
			}
			CreatePolarImages(width,height, pMyData);

			unsigned char* sourcePixel = (unsigned char*)bytes;
			unsigned char* destPixel[4];

			for (int i = 0; i < 4; i++)
			{
				destPixel[i] = pMyData->grayImage[i].data;
			}

			for (int y = 0; y < height; y += 2)
			{
				for (int x = 0; x < width; x += 2)
				{
					*destPixel[1]++ = *sourcePixel++;
					*destPixel[0]++ = *sourcePixel++;
				}
				for (int x = 0; x < width; x += 2)
				{
					*destPixel[2]++ = *sourcePixel++;
					*destPixel[3]++ = *sourcePixel++;
				}
			}

			for (int i = 0; i < 4; i++)
			{
				cv::cvtColor(pMyData->grayImage[i], pMyData->colorImage[i], cv::COLOR_BayerRG2BGR);
			}

			ShowPolarImages(pMyData);

			arv_stream_push_buffer(pStream, pBuffer);
		}
		else
		{
            if( Status == ARV_BUFFER_STATUS_TIMEOUT)
            {
                pMyData->Packettimeouts++;
                //printf("Paket timeouts %d\n", pMyData->Packettimeouts);
            }
            else
			{
                printf("%s\n",getErrormessage(Status).c_str());
            }
            arv_stream_push_buffer(pStream, pBuffer);
			pMyData->error = true;
		}
	}
	else
	{
		printf("Buffer is null\n");
	}
}

If this works, you may change to 16 bits using the "Polarized Mono 16" format.
Stefan

Hi Stefan,
Thank you for your answer.

More than half of the code in your example is missing. I was lucky to find the code zipped in an other issue.
When compile with aravis-0.8 and OpenCV 4.6, the executable stay idle, and do nothing.

That said I was able to test, and adapt your code to work with PolarizedMono8 and PolarizedMono16.
Nonetheless I still want to use to PolarizedMono12Packed, what should I do to retrieve the information?
Thanks in advance.
Sincerely,
Julien Fleuret

Hello
I saw you also asked my US colleagues, who asked me too. However, I added the sample for Mono 8 at https://github.com/TheImagingSource/Linux-tiscamera-Programming-Samples/tree/master/cpp/Extract-four-images-from-polarization-camera. There should nothing missing.
12bit packed means, that 12bits make a pixel. Then immediately the next 12bits follow in the second byte of the first pixel. So you need to extract this stream.
src bits  01 02 03 04 05 06 07 08     09 10 11 12 13 14 15 16     17 18 19 20 21 22 24 25    26 27 28  ...
pixel       1  1  1    1   1   1   1   1        1  1    1   1   2   2  2   2       2    2   2   2   2   2   2   2      3   3    3 ...

You have 3 bytes for two pixels and split them accordingly.  The idea could be:
p1 = (WORD*)*data & 0x7FF8;
p1 = p1 >> 4;
pdata++;
p2 = (WORD*)*data & 0x0FFF;

data is a BYTE* pointer to the image data.
But I did not try that on my own.
Best regards
Stefan

Hello Julien
I updated the above mentioned sample. It handles the mono 12bit packed and mono 16 bit video formats now. Unpacking of 12bits to 16bits is not that easy as I thought.
Stefan


Run Functional Tests not finding ChromeDriver only on some machines?

I have a problem with running functional tests. We are moving from another product to VSTS for our integration testing. As such, I am setting up a cluster of a number of VM's for running web based tests. They all are supposed to run the same tests in distributed fashion, to make things faster. Tests are C# written and selenium based.
For now, there are 4 VM's which I did reinstall this morning side by side (i.e. same step on all vm's, then same step again on all VM's).
This is a base VM, domain user with local admin rights, chrome installed.
The test package includes a nuget determined chromedriver.exe so that we do not ahve to manually update the test machines for new chromedriver versions.
Again, just to emphasize, this is for the same test run. I.e. the same test data packet is distributed to the same machines (machien names in a variable) using the Windows Machine File Copy task.
The functional tests do not run on 2 of the machines. 2 of the 4 machines properly start the tests,, chromedriver starts, opens chrome, all works nice.
THe other 2 machines fail to locate the chromedriver and I am out of my guts on why. I find error messages that read:
Assembly Initialization method Tests.Integ.XXX.AssemblySetup.SetUp threw exception. OpenQA.Selenium.DriverServiceNotFoundException: OpenQA.Selenium.DriverServiceNotFoundException: The file C:\Users\TestAgentUser\AppData\Local\Temp\TR_92acb\Deploy_TestAgentUser 2017-06-30 03_35_49\Out\chromedriver.exe does not exist. The driver can be downloaded at http://chromedriver.storage.googleapis.com/index.html. Aborting test execution.
I validated that the driver is in the proper place (in this case D:\Testfiles), at the same place as the other two machines. It seesms that for some reason two of the 4 VM's decide to look in the wrong place, and I am at a loss here. ANyone an idea? Any log I can check?

Can you please share the complete build logs with us? It would be helpful if you can do a build with system.debug = true.

@dpksinghal Build or test logs? Because I run the tests from a release deployment, not a build. Want me to build with system.debug=true and then deploy? Please advice.

No you can do the same in the release. Please set this variable and run the release once. After that please share the logs with me.

@NetTecture Were you able to solve this problem? If not can you please share the release logs (with variable system.debug=true).
Please also share DtaExecutionHost.exe.log from C:\Dtalogs folder for both passing and failing VM.
Thanks.

@vimegh I have not solved it.
I am collecting DTAExecutionHost.exe.log later today.
Is there an email to send them to? I am not comfortable sharing environment specific with the internet.

@NetTecture You can email logs to us at<EMAIL_ADDRESS>Thanks.

@NetTecture Did you get a chance to capture the logs?

@NetTecture We are closing this for now. If you have already solved the problem, please share the resolution. If you are still seeing the issue, please feel free to reopen, and do send us the release logs (with variable system.debug=true) and DtaExecutionHost.exe.log from C:\Dtalogs folder in the VM for both passing and failing one. Thanks.

Reopenng. Just collecting the log files.

@vimegh Logs submitted. Please reopen.

@vimegh Logs not submitable. The email you indicated errors out. Error:
The groups only accept messages from people in their organization or on their allowed senders list, and your email address isn't on the lists.

We have received the mail today. Reopening the issue.

@NetTecture The errors indicate that the chromedriver is missing in the tests deployment folder. I don't see a runsettings file used to enabled deployment. Could you please share details on how are you getting this file to be part of the test deployment folder. Is it specified as a deployment item on specific test classes ?

@cltshivash Yes, as I said in the original post. The chromedriver is set as part of the deployed files. This also works for SOME machines, intermittently (I.e. it is random after a restart whether it will work or not until the next restart). There is no runsettings - things work without (at least where the chromedriver is found). The chromedriver is just part of the deployed files (similar to the dll's that are distributed with the output of the test project). It is properly deployed to D:\Testfiles (which is where all the relevant files reside) and then the agent copy operation just decides to sometimes ignore it, it seems.

@NetTecture Can you please check if chromedriver.exe is specified as a deploymentitem ([DeploymentItem("chromedriver.exe")]) in your test code ? it is possible that it is specified in one of the test classes and not in all of them.

@cltshivash It is not. As in: not specificed in ANY SINGLE TEST. Is it necesssary? It seems not to be documented anywhere. If it is - why does it work on some machines? (which change after every reset). Here is the code I am using to load the chromedriver:
    [AssemblyInitialize]
    public static void SetUp(TestContext context) {
        var baseDir = Path.GetDirectoryName((typeof(AssemblySetup)).Assembly.Location);
        _webDriver = new ChromeDriver(baseDir);
    }

It works on all setups we call in TeamCity and it works on some of the machines in VSTS - as I already pointed out repeatedly, the machines change on reboots, so SOMETHING is off there. I hate random stuff thrown around like that.

@NetTecture The chromedriver.exe will be copied over to the test run directory based on the below

If there's a test that has a compile time dependency on it
If it has been specified as a deployment item in the test class or in the testsettings.

Without looking at the code it is hard to comment on point no.1. Kindly add "chromedriver.exe" as a deploymentitem in the test code and verify if it helps. I tried locally and i am able to get past the error.

Please reach us if the problem persists. closing this issue. Thanks for your support


MSID: 99062 Version: 1 DOI: 10.1101/2<IP_ADDRESS>5219

MSID: 99062
Version: 1
Preprint DOI: https://doi.org/10.1101/2<IP_ADDRESS>5219
Step 1. Awaiting reviews
Editorial to post reviews via hypothesis
Useful links:

DocMap: https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=99062
New model tracking: https://docs.google.com/spreadsheets/d/1_fHaoOy7hjyocptKtVJRijeNpUY4hBS7Ck_aVmx6ZJk/
Reviews on sciety: https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>5219

For trouble shooting (e.g. no Docmaps available):

DocMap issue addressing: https://miro.com/app/board/uXjVNCwK6EI=/
Explore DataHub DocMaps API: https://lookerstudio.google.com/reporting/4c2f0368-babb-4beb-b5b3-497e7e7b0f08/page/ejphD
Unmatched submissions and preprints: https://lookerstudio.google.com/u/0/reporting/9f86204f-3bf7-477c-9b18-5c5ef141bf69/page/p_gxi57ha93c
Unmatched manuscripts spreadsheet: https://docs.google.com/spreadsheets/d/15QcK8w-ssB7109RQEDtFpJPZ0J5HTGxoHa_2TtpMBbg/edit#gid=1336081641

Step 2. Preview reviewed preprint
Production QC content ahead of publication
Instructions:

QC preview: https://prod--epp.elifesciences.org/previews/99062v1
Update ticket with any problems (add blocked label)
When QC OK, add QC OK label to ticket and add publication date and time to https://docs.google.com/spreadsheets/d/1amAlKvdLcaDp5W8Z8g77NmkwbMF5n_u89ArSqPMO8jg
Move card to next column
(At end of the day post link in #enhanced-preprint and ask for PDF to be generated)

Useful links:

Preprint DOI :  https://doi.org/10.1101/2<IP_ADDRESS>5219
Confirm reviews returned by EPP: https://prod--epp.elifesciences.org/api/reviewed-preprints/99062/v1/reviews
To update the MECA path in the docmap: https://docs.google.com/spreadsheets/d/1mctCQuNFBjSn97Lihy7_vBO6z7-N-oqyLv4clyi6zHg

Step 3: Awaiting search reindex
This step adds the reviewed preprint to the homepage: https://elifesciences.org
The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.
Useful links:

Jenkins pipeline to reindex search can be triggered sooner or monitored here: https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 4: Published! PDF requested
Waiting for PDF to be generated
Useful links:

PDF tracking: https://docs.google.com/spreadsheets/d/106_XeDjmuBae7gexOTNzg60lapeqjl2aRn9DzupGyS8/

Step 5: Introduce PDF to data folder and git repo
Upload PDF to relevent folder in git repo https://github.com/elifesciences/enhanced-preprints-data/
Step 6: Done!

[ ] Kettle is on!

warn: [journal-ref-text-content] This journal reference (id c32) has untagged textual content -  Suppl 2, ii54-58.. Is it tagged correctly?
warn: [mixed-citation-other-publication-flag] mixed-citation in reference (id=c67) has a publication-type='other'. Is that correct?

Thanks @rkcook312 all sorted


feat: show resource utilization in DevTools plugin

Hey, I just made a Pull Request!
Shows resource utilization on devtools page.

:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[x] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

With the caveat that it'll of course show the info for only the one host out of potentially many, that the request happened to go to.

@freben yep. But the same applies to all the other info like OS, config, and dependencies.

FYI - The E2E failures are related to this: https://github.com/remix-run/react-router/issues/10566#issuecomment-1593277043

Waiting for getting back from vacations 😅

Same here, it's vacation time! 😎

Uh oh. @drodil did you mean to let this get closed?

@efenner-cambia no. I've been on vacation so my activity has been zero for the past month 😎 Could someone re-open?

Remove stale, waiting for review.

@freben @Rugvip with @jhaals being away on vacation can you give this a review please?

Nope, still not stale.

Sweet! Thanks @drodil for this and your patience waiting for it to get merged 🚀
Thanks fort he merge @jhaals 👍


Testing node and more refactoring

Created a testing node to automate testing and logging, closes #19.
added new response variables to start_plan srv for logging purposes.
refactored code to move planning requests and collision geometries updating to Planner class as was originally desired in #16, this will help changing to online to planning for #23.
Added grasp attempt verification.

update:

fixed logging issue in testing node
arm reset is forced
cleaned up more unused methods in Manipulator class


Not showing error.vue

I need to show error.vue if the response.status is 403. Right not its showing Nuxt.js Error with a pink color. Any way to do this?

I'm not sure to understand your question @shyamchandranmec
Can you create a reproduction by forking https://glitch.com/edit/#!/nuxt-hello-world to show what you mean?
I will re-open the issue then.


Assess whether Python solution should be introduced for file templating

As per documentation comments made in https://github.com/seqeralabs/cx-field-tools-installer/pull/44, the Terraform templatefile & Ansible solution is starting to show signs of strain as more business logic is showing up in this solution.
We can still work around the problems for the time being but it may be worth evaluating whether we should move the responsibility of template file generation to a proper Python script (a la .githooks configuration checker) to gain access to a real programming language which could simplify the implementation. TBD.

The Terraform external provider may fit the bill.
This was just used to externalize some nest if statements to a Python script which, IMO, was much more suitable to the task some a complicated multi-line nested set of ternary checks in a local variable instead.


Two factor auth?

can't find anything about it, is it supported in any form?

Yes, it's possible.
https://github.com/seishun/node-steam/tree/master/lib/handlers/user#logonlogondetails
https://github.com/SteamRE/SteamKit/blob/master/Resources/Protobufs/steamclient/steammessages_clientserver.proto#L90
I have it implemented here.

Assuming @scholtzm is correct, then it's already supported. You're welcome to add information about it to the wiki.

@seishun Forgot about the wiki. Added. :+1:

Trying to login with two factor, after successful authorization trying one again without code, steam saying I need code. Sentry file doesn't work for two factor or I'm doing something work?

"Use the wiki, Luke."


project end point

A project web-service end point is needed. Since project information will go beyond simple counts, this should be part of a root-level end point and not just contained as a fragment of the cluster end point. A similar approach to that for PSMs should be taken.

Implementation is done, but needs testing

done


os_free should used rather than free

os_free's behavior may be different from free, so if the buffer is allocated using os_malloc, it should be freed with os_free rather than free.

@aohwang thanks for this


Add custom namespace support

nice plugin!
just modified it a bit for my use case. And thought might helpful for others also.

Cool, thanks @zmalik !


Cannot scrape targets specified by mDNS name

What did you do?
I am trying to scrape a target that runs node_exporter by its mDNS name (published through Avahi) in my local network (see configuration below).
What did you expect to see?
The same behavior as specifying the IP address in targets (which works as expected).
What did you see instead? Under which circumstances?
State of the endpoint is DOWN in http://ptometheus.local:9090/targets, while ping works as expected.
Environment

System information:

# uname -srm
Linux 4.4.21-69-default x86_64

Prometheus version:

Latest released:
# /opt/prometheus/prometheus -version
prometheus, version 1.5.2 (branch: master, revision: bd1182d29f462c39544f94cc822830e1c64cf55b)
  build user:       root@a8af9200f95d
  build date:       20170210-14:41:22
  go version:       go1.7.5

Also reproduced on self-built master:
# /opt/prometheus/prometheus -version
prometheus, version 1.5.2 (branch: master, revision: 29f05680a29afe9aa6cf119563e99723a41765bd)
  build user:       silvio@lenovo
  build date:       20170328-11:31:33
  go version:       go1.7.3

Prometheus configuration file:

global:
  scrape_interval: 5s
scrape_configs:
  - job_name: 'avahi'
    static_configs:
      - targets: ['avahi.local:9100']

Slightly related, I recently implemented mDNS service discovery.
It looks for things advertising _prometheus-http._tcp (and ditto HTTPS) over zeroconf/mDNS and regularly dumps what it found. You won't need the zeroconf hostnames, but you will need to publish your service for consumption by prometheus.

We support DNS, with would require specific mDNS support to be added.

@msiebuhr What were the main reasons to close the PR? I'd actually love to see this in Prometheus, with the influx of people using Prometheus for IoT use cases etc.

It drags in quite a few large dependencies (low-level networking to get mDNS working), and I was told in private conversation (perhaps @brian-brazil ; cannot remember) that new service discovery mechanism preferably should be built outside core prometheus.
But yes, I generally find it weird (from a hobbyist/IoT point-of-view) that Prometheus comes with complex discovery mechanisms for some really big systems and everyone else must implement some custom cron/daemon thing poking at file_sd.

@msiebuhr I fully agree with this. I don't know why proprietary, vendor specific SD mechanisms for things like triton or azure are in scope while the most widely deployed open mechanism isn't.
I know that @SuperQ would also like to see mDNS support. So if we can find lazy consensus here, I'd probably call in a official vote on this.

We continue to have a moratorium on new SD mechanisms, and once the example file_sd code is in the plan is to close all existing feature requests and point to that instead.
SDs take a lot of work to maintain, and history shows that there's little interest in doing so.

I don't agree with this plan.

There's no sign of an end to the conditions that gave us the moratorium, and I'd rather clearly set expectations with the various requestors of SDs - some of whom have been waiting for 2+ years - than continue to give them false hope. Between the file_sd example and setting expectations I'm hoping that there'll be a surge in the development&maintenance of these by 3rd parties, rather than the current stream of +1s on issues which are ignored.
This structure gives us the option of pulling in some of the SDs into Prometheus in the future when things are healthier, but we continue to be unable to accept new SDs or even maintain what we already have.

This bug—which is specifically about resolving .local domain names (that come from somewhere else, such as static_configs), not using mDNS for service discovery—seems to have been fixed by https://github.com/golang/go/issues/16739. It should probably be closed, and a separate issue opened for a possible future SD mechanism.

Yes, we also ended the moratorium on new service discovery, so a feature design issue would be appropraite.

This bug—which is specifically about resolving .local domain names (that come from somewhere else, such as static_configs), not using mDNS for service discovery—seems to have been fixed by golang/go#16739. It should probably be closed

According to my understanding this is not the case - the Go issue above is not really about implementing mDNS resolution in the pure Go resolver.
Reproducer (main.go):
package main

import "fmt"
import "net"

func main() {
	name, err := net.LookupHost("prometheus.local")
	if err == nil {
		fmt.Println(name[0])
	} else {
		fmt.Println(err.Error())
	}
}

On the commandline:
> go version
go version go1.12.17 linux/amd64

> GODEBUG=netdns=cgo go run ./resolve.go
<IP_ADDRESS>

> GODEBUG=netdns=go go run ./resolve.go
lookup diskstation.local on <IP_ADDRESS>:53: no such host

So to me the original description of this issue is still valid.

This bug—which is specifically about resolving .local domain names (that come from somewhere else, such as static_configs), not using mDNS for service discovery—seems to have been fixed by golang/go#16739. It should probably be closed

According to my understanding this is not the case - the Go issue above is not really about implementing mDNS resolution in the pure Go resolver.
Reproducer (main.go):
package main

import "fmt"
import "net"

func main() {
	name, err := net.LookupHost("prometheus.local")
	if err == nil {
		fmt.Println(name[0])
	} else {
		fmt.Println(err.Error())
	}
}

On the commandline:
> go version
go version go1.12.17 linux/amd64

> GODEBUG=netdns=cgo go run ./resolve.go
<IP_ADDRESS>

> GODEBUG=netdns=go go run ./resolve.go
lookup diskstation.local on <IP_ADDRESS>:53: no such host

So to me the original description of this issue is still valid.

There's something odd with Prometheus... pure Go as per above comment seems to work fine. From go documentation, it should be using cgo resolver:

...and when the name being looked up ends in .local or is an mDNS name.

So, it should be working for .local targets.
I even tried running my Prometehus with GODEBUG=netdns=cgo, but scraping still fails:
Get "http://brown.local:9100/metrics": dial tcp: lookup brown.local on <IP_ADDRESS>:53: no such host

The error tells me Prometheus is talking to the DNS server from my resolv.conf, and NOT using what's at nsswitch.conf, as it'd be expected if it was using cgo resolver. getent hosts brown.local works just fine though.
My guess is that Prometheus is using some custom code for name resolution, that does not leverage Go's "just work" approach for mDNS for .local domains.

@fornellas see https://github.com/prometheus/prometheus/pull/2538. The pure Go resolver is forced at build time, so GODEBUG=netdns=cgo does not have effect.

Can I poke on this issue? It was opened on March/2017 and the last comment was December/2021 and while there's been some interesting discussion, it has not led to any resolving of the situation-at-hand.
I would like to point out that in this day-n-age, one would expect an MDNS based hostname (RFC6763  etc) to be accepted by ALL software running on a modern-day operating system. If the host(1) command shows resolution of a hostname, then similar resolution should be provided. Sadly, most of the discussion here is all about GoLang's purity vs actually solving @moio's initial (five and half year old) issue.
While I know the right thing for me to do it to buck up and send in a PR; sadly, I'm a C & Python kinda person and GoLang isn't my forte. But I am someone that's used Prometheus for a long time and was shocked when I spun Prometheus up at home to do some testing and found it would not cooperate across my two simply configured hosts within my .local DNS range. I was basically shocked (until I read this, then I was further shocked).
Summary: If all other system s/w can resolve a host in the .local domain, so should Prometheus.

I welcome contributions regarding this , maybe with a new service discovery provider, mdns, based on https://github.com/hashicorp/mdns

There's also an mDNS plugin for CoreDNS. https://coredns.io/explugins/mdns/

Hi All, I ran into this and wanted to follow-up to help any others who stumble across this issue.
There's great discussion here, in PRs, and in the official docs. The solution @msiebuhr came up with is super cool. Alas, none of my hosts announce their services available and I'm hoping to just centrally configure everything. Their approach of using another service to handle MDNS resolution and to write the result to a file is really neat and easy to reason about, which is appealing.
That got me pondering their approach and reading the docs. It turns out file-based service discovery is super flexible and shockingly easy to use. I ended up hacking together a quick solution to get support for MDNS *.local hostnames and I figured I should share to help any others who want it:
How the solution works:

List the hosts and tags you want to scrape in a json file. I went with files named /etc/prometheus/lan-blackbox-hosts.json and /etc/prometheus/lan-node-hosts.json.
Configure a systemd timer service that runs a python script. It reads in those files and writes out the resolved ip (or ip:port) in place of the hostnames (or hostname:port), and preserves the tags you provide with the input.
Configure prometheus to scrape the targets in those output json files, rather than manually enumerating the scrape targets inline in prometheus.yml.
Sit back and relax, it's all on autopilot now!

Operating assumptions
(feel free to regex replace these to your preference)

Prometheus configs live in /etc/prometheus/
Prometheus is storing it's runtime-mutable data in /var/lib/prometheus/
You'd like to run prometheus-related services under a service user named prometheus

Add one via sudo useradd --system prometheus
You'll want to chown any folders that user will write to (i.e., sudo chown -R prometheus /var/lib/prometheus/)

Hack together support for MDNS using Custom Service Discovery
Sources and inspiration:
https://prometheus.io/docs/guides/file-sd/
https://github.com/prometheus/prometheus/pull/1903
https://github.com/msiebuhr/prometheus-mdns-sd
https://prometheus.io/blog/2015/06/01/advanced-service-discovery/#custom-service-discovery
https://prometheus.io/blog/2015/08/17/service-discovery-with-etcd/
sudo mkdir /opt/mdns-resolve/
sudo vim /opt/mdns-resolve/mdns-resolve.py
import argparse
import json
import socket
import sys
import urllib.parse

# Get the input and output files as command line arguments
parser = argparse.ArgumentParser()
parser.add_argument('in_file', help='The input JSON file')
parser.add_argument('out_file', help='The output JSON file')
args = parser.parse_args()

# Load the input data into a holder object
with open(args.in_file, 'r') as input_data:
    lan_description = json.load(input_data)

# Resolve the host names to IP addresses
for lan_hosts in lan_description:
    resolved_ips = [];
    for host_port in lan_hosts['hosts']:
        parsed_url = urllib.parse.urlsplit('//' + host_port);
        try:
            ip = socket.gethostbyname(parsed_url.hostname);
            if parsed_url.port:
                ip = ip + ":" + str(parsed_url.port);
            resolved_ips.append(ip);
        except socket.gaierror:
            sys.stderr.write("Failed to resolve hostname: {host}");
    del lan_hosts['hosts']
    lan_hosts['targets'] = resolved_ips;

# Write the updated list to the output JSON file
with open(args.out_file, 'w') as f:
    json.dump(lan_description, f, indent=4)

sudo vim /etc/prometheus/lan-blackbox-hosts.json
[
    {
        "hosts": ["some_lan_host1.local"],
        "labels": {
            "label1": "label_value1",
            "label2": "label_value2"
        }
    },
    {
        "hosts": ["some_lan_host2.local"],
        "labels": {
            "label1": "label_value1",
            "label2": "label_value2"
        }
    }
]

sudo vim /etc/prometheus/lan-node-hosts.json
[
    {
        "hosts": ["some_lan_host.local:9100"],
        "labels": {
            "label1": "label_value1",
            "label2": "label_value2"
        }
    }
]

sudo vim /etc/systemd/system/prometheus-mdns-resolve.service
[Unit]
Description=Resolves MDNS Hosts for Prometheus
After=network-online.target

[Service]
User=prometheus
Type=oneshot
ExecStart=\
python3 /opt/mdns-resolve/mdns-resolve.py \
/etc/prometheus/lan-blackbox-hosts.json /var/lib/prometheus/lan-blackbox-targets.json && \
python3 /opt/mdns-resolve/mdns-resolve.py \
/etc/prometheus/lan-node-hosts.json /var/lib/prometheus/lan-node-targets.json

sudo vim /etc/systemd/system/prometheus-mdns-resolve.timer
[Unit]
Description=Refreshes resolved MDNS hostnames for Prometheus every 5m
After=network-online.target

[Timer]
OnBootSec=0min
OnCalendar=*:0/5

[Install]
WantedBy=timers.target

sudo systemctl daemon-reload && \
sudo systemctl enable prometheus-mdns-resolve.timer && \
sudo systemctl start prometheus-mdns-resolve.timer && \
sudo systemctl status prometheus-mdns-resolve.timer

Seeing what's up:

sudo systemctl status prometheus-mdns-resolve.timer
sudo systemctl status prometheus-mdns-resolve.service
sudo journalctl -u prometheus-mdns-resolve.service

Consume in prometheus by replacing relevant static_configs: entries with something like:
    file_sd_configs:
      - files:
        - /var/lib/prometheus/lan-node-targets.json
        refresh_interval: 5m

or
    file_sd_configs:
      - files:
        - /var/lib/prometheus/lan-blackbox-targets.json
        refresh_interval: 5m

etc

Hope this helps any folks who stumble across this issue! Happy holidays!

https://github.com/brutella/dnssd seems to be a lightweight, well tested library for mDNS service handling.
Also +1 on the IoT usecase.

Hello from the bug scrub: this seems like a legit use case perhaps better suited for working through https://prometheus.io/docs/prometheus/latest/configuration/configuration/#http_sd_config or https://prometheus.io/docs/guides/file-sd/. Contributions welcome.

If anyone interested, I've implemented HTTP SD:
https://github.com/fornellas/prometheus-mdns-http-sd
"works for me", hopefully for others as well :-D

7 years and no solution?


Missing Closing Tag in Example

Missing closing tag for <TreeView.ItemTemplate> in the XAML Codeblock in 'Examples'

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9904e8f2-4d99-d0f6-9d86-637fbf280de6
Version Independent ID: f8931355-6c5f-188e-d2aa-00f6475b2865
Content: TreeView.ItemTemplate Property (Windows.UI.Xaml.Controls) - Windows UWP applications
Content Source: winrt-api-build/xml/Windows.UI.Xaml.Controls/TreeView.xml
Product: uwp
Technology: winui
GitHub Login: @jwmsft
Microsoft Alias: jimwalk

Thanks for letting us know about this. The sample is fixed.


Update to steps for enablement

Need to update the steps for enablement. Currently, we have $RM but don't show customers who/what this is, so added a step to fix this.

@jambirk - for your review.

@nenonix @JohanFreelancer9 Copy editing is needed for this PR. Thanks.


Feature: support gherkin localization

The problem
When you want to use gherkin synthax in a other language than english it fails
A solution
Allow the parser (I know it's kind a lot of work) to play with the language like defined there https://cucumber.io/docs/gherkin/languages/

Localization should work. Could you provide an example code and error message?

Arf, I miss this example. Sorry.
It's working well. Thanks to show it.
I would suggest to add it in the documentation.

Added to docs: https://vitalets.github.io/playwright-bdd/#/writing-features/i18n
And a config field language that was also missing: https://vitalets.github.io/playwright-bdd/#/configuration/options?id=language
thank you!


wvunpack should ignore qmode from WavpackGetMode() for WAV family output

As the title says.
When a big endian caf file is encoded with -r option set, wvunpack decodes it into a big endian wav file by default.

Thanks, good catch!
There are actually several things along those lines that I need to look into. Without even trying, I'm afraid that using the --skip and --until options might produce some wrong results with other formats/endians too.
And I really need to figure out how I want the -r (raw) option in wvunpack to work. I think it should always output the native (original) endian, but then I should have alternate versions like --raw-be and --raw-le to force a specific format.
But yeah, WAVs should always be LE...  :-)

I decided that to really fix this (and related) issues correctly I need to store the source file format in the file, along with the original extension, and create an array of available output formats that wvunpack can use.
So now, if you pack a big-endian caf file with -r and unpack it, you'll get a big-endian caf file (with a freshly generated generic caf header). If you want a wav file instead, the -w option will do it, and it will not be big-endian (obviously). I think I covered all the bases here, but feel free to point out if I missed anything.
Thanks again!


moving past last line crashes

if you press down arrow on the last line of a code cell, it should move to the next cell.
instead i get this stack trace:
Uncaught TypeError: scope.bkNotebook.getFocusable(...).focus is not a function
moveFocusDown @ core.js:503
goDownOrMoveFocusDown @ core.js:491
...

same for up arrow

@scottdraves , the same is for up arrow, when you in the first line of a cell and it should move you to the previous cell. Getting 't.focus is not a function'.


Airdrop Bonus discussion

Come for your investment guides and u your Gmail Airdrop Bonus activities. We help you get your wallet activated for AIRDROP BONUS


Installing 2.6.3 and missing database?

Hi,
I have done a fresh installation to windows server 2012r2 and the geoportal application does not start in Tomcat. There was no instruction to set up database for 2.6.3 version or have I missed something? In previous version 1.2.9 there are instructions to set up a database. Is that database valid for this version 2.6.3?
Thanks!

hi, you have not missed anything. Geoportal 2.6.3 does not use a relational database. It does use elasticsearch. Installation instructions for 2.6.3 are here.

Thanks, this is my first time installing this Geoportal. I did not realize that elasticsearch is used to store metadata. Will try Kaizen when installation is success. Nice to know! Should have read more about these components. But elasticsearch is installed.
And also my server is not connected to internet. I have followed installation instructions and done configuring for disconnected environments.
When I look at the logs there is errors about springframework xml schema definitions. Because I am not connected to internet, I cannot read those. Is there a workaround for this? Could this be the problem with my installation?

you would need to locally host those online references, such as XML schemas, the JS API, basemaps in the map facet and viewer, references to external JS libraries or CSS, etc.


KAS-3635: Remove duplicate codelists

Updates mu-authorization config to remove unused types & adds migrations to move all public static types to the public graph, so we no longer duplicate them over all graphs.
Related PRs:

[ ] https://github.com/kanselarij-vlaanderen/yggdrasil/pull/47

Will bump Yggdrasil with a tagged version in #332. Running an older version for now doesn't break this PR.


Readme document release process for release candidates

Do not merge yet!

publish.sh contains a bug when the release is performed from the master branch.

Coverage remained the same at 78.598% when pulling 6c30af3664725fd9cd089c0074be6fb7edb161a3 on pavolloffay:release-readme-fix-RC into 27327e91ba30b2280ed86afe9f65583c5c3db95f on opentracing:v0.31.0.

@carlosalberto could you please review?

Coverage remained the same at 78.598% when pulling b4c500a2d0d9a7a4153a4f121db8a4507bc5ffa0 on pavolloffay:release-readme-fix-RC into 27327e91ba30b2280ed86afe9f65583c5c3db95f on opentracing:v0.31.0.

@pavolloffay Looks great! Now we will be able to release just fine from master (when the time comes) :)

@carlosalberto  can I merge? Or feel free to merge


Provide drg commands to set passwords and gateways.

The release tracking issue:

[ ] Specify API
[ ] Allow setting a password and username/password
[ ] Allow setting a gateway
…

duplicate of https://github.com/drogue-iot/drg/issues/41 ?

Tracking issue … for the project board.


Docker run failed

After run docker container it is fail
/usr/bin/env: 'bash\r': No such file or directory
"[37;40mnpm ERR! Missing script: "start:prod
npm ERR!
npm ERR! Did you mean this?
npm ERR!     npm run start:prod # run the "start:prod" package script
npm ERR!
npm ERR! To see a list of scripts, run:
npm ERR!   npm run
npm ERR!

Windows 10

@vladimirpetukhov got it. Thanks!


Message history is bound to a split, not channel

Describe the bug
Message history is bound to a split, not channel meaning that each split of same channel has different message history (accessable by pressing up arrow on input box)
Chatterino version
Chatterino Nightly 2.1.5 (commit 6111985b7, 26.12.2019)
Operating system
windows 10

This is intended


how are express locals made available in public/js scripts?

How can express response locals to be read in client code?
public/js/blah.js
$(document).ready(function() {
   // read variables set here
});

...here I load the script in the page, using the connect-assets shorthand
views/some-view.jade
extends layout
   !=js('blah')
  block content
  ...

Why not adding your code in main.js ?
Or you can add  this line in application.js :
//= require blah.js

Oh, I just omitted that necessary step from my shown setup...
connect-assets seems to detach public/js from access to local variables in
the express.js response object, which are, of course, available in
hackathon-starters jade templates.
The point of the issue I'm raising is to confirm that there is not some
obvious thing I'm missing, that allows public/js scripts access to the
express response object.
On Thu, May 7, 2015 at 11:10 AM, Fabien Champigny<EMAIL_ADDRESS>wrote:

Why not adding your code in main.js ?
Or you can add this line in application.js :
//= require blah.js
—
Reply to this email directly or view it on GitHub
https://github.com/sahat/hackathon-starter/issues/270#issuecomment-99902783
.

@gpolyn You are correct, when you reference an external script, you cannot use local variables like you do in Jade templates and that has nothing to do with connect-assets. You have 2 options that I can think of:

Inline JS code inside Jade templates. That's what I had to use when I was first starting out with Node.js and web development in general. https://github.com/sahat/csc322/blob/master/views/profile.jade
From Jade, add data- attribute to your HTML elements with variable content, then read it from blah.js, e.g:

 var example =  $('.container').data('example');

There could be other ways, but these are the only two ways that I know of for what you are trying to achieve.

@sahat thank your for confirming (and thank you for this resource!)
With respect to your comment...
Inline JS code inside Jade templates. That's what I had to use when I was first starting out with Node.js and web development in general.
...if you're implying that more mature approaches rely only on client-server http interface to exchange data, not sure I agree.


Add all assets

This issue has been closed figuratively for the time being -- a new and more in-depth issue report has been opened.

This issue has been closed figuratively for the time being -- a new and more in-depth issue report has been opened.

This is an official comment made by a contributor from an alternative account.


Possible wrong discount scenario

When looking at the discount example given on 5000 CU is says 20% discount, I am pretty sure that it is only 10% :)
/Brian

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 3873dadd-9f2b-6ad7-6124-ba0c234f7c72
Version Independent ID: ba43428e-f38a-8a8c-1751-4f31d94e07c8
Content: Optimize Microsoft Defender for Cloud costs with a prepurchase plan - Microsoft Defender for Cloud
Content Source: articles/defender-for-cloud/prepurchase-plan.md
Service: defender-for-cloud
GitHub Login: @dcurwin
Microsoft Alias: dacurwin

@Brobtown this is being addressed and the documentation will be updated soon

#please-close


encoding/json: unmarshaler doesn't find UnmarshalJSON on embedded anonymous struct

What version of Go are you using (go version)?

$ go version
go version go1.15.9 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.15"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.15/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build393930481=/tmp/go-build -gno-record-gcc-switches"

What did you do?
https://play.golang.org/p/Xlxih58rSdC
Outer1 unmarshalled and Outer2 failed.
type Outer1 struct {
	Inner inner
}
type inner struct{ Mixin }

type Outer2 struct {
	Inner struct{ Mixin }
}

type Mixin string

func (r * Mixin) UnmarshalJSON(data []byte) error {
	return json.Unmarshal(data, (*string)(r))
}

What did you expect to see?
Outer2 unmarshalled as Outer1
What did you see instead?
panic: json: cannot unmarshal string into Go struct field Outer2.Inner of type struct { main.Mixin }

At first glance this seems like a bug, thanks. There have been many bugs in the past involving struct embedding, but after a quick search I don't think this is a duplicate. Want to try working on a fix with a test?
cc @dsnet

It's because json.indirect() only tries to get a field address with a named type. See https://cs.opensource.google/go/go/+/refs/tags/go1.16.4:src/encoding/json/decode.go;l=443.
The type of Outer2.Inner is a struct without a name, so json.indirect() doesn't try its pointer and of course it's not compatible with json.UnmarshalJSONinterface.
I think at the entrance of json.indirect(), checking v.CanAddr() is enough. It needs not to check if v is a named type.

@googollee, thank you for your analysis. I agree with your assessment of why this isn't triggering. It seems that this is a bug that's existed for 10 years :)
We could try fixing this for the Go1.18 release cycle, but I'm always wary of how fixing bugs breaks a number of users who already depend on the current behavior.

We could always merge this early in the 1.18 cycle, and see if any users complain. I would hope that noone is depending on the existing inconsistent behavior - I've been wrong on that assumption before, but not always :)

Removing the check of named structs looses the limitation. I think it should be fine with existing codes.
I made a change with a unit test.


fix: move types condition to the front

I moved types condition to the front. package.json#exports are order-sensitive - they are always matched from the top to the bottom. When a match is found then it should be used and no further matching should occur.
Right now, the current setup works in TypeScript but it's considered a bug and it should not be relied upon, see the thread and the comment here. For that reason, I would like to fix all popular packages that misconfigured their exports this way so the bug can be fixed in TypeScript.
⚠ this PR focuses solely on fixing "🐛 Used fallback condition" problem but the "🚭 Unexpected ESM syntax" remains here. You can check the reported errors here

Oops yes thanks for reminding me


Excerpt files to .pbf instead of .osm, if possible

Currently, we source .pbf format OpenStreetMap data (a compressed storage format) but export to .osm when extracting the portion corresponding to a buffered study region.   The .osm file can be significantly larger in file size than the source .pbf, even if the geographical area of the pbf is considerably larger; also osm files can be quite large (e.g. 50mb to 500mb).
These seem to be good reasons for re-outputting excerpts to .pbf, instead of converting to .osm.
osmconvert can output pbf
osm2pgsql can read pbf
The lines that would require changing are here:
https://github.com/global-healthy-liveable-cities/global-indicators/blob/6d5c2f010c755f91356d948054e8cbf7fb6e9102/process/pre_process/02_create_osm_resources.py#L49
https://github.com/global-healthy-liveable-cities/global-indicators/blob/6d5c2f010c755f91356d948054e8cbf7fb6e9102/process/pre_process/02_create_osm_resources.py#L62
The only reason that I can see for not doing this is if the additional compression/uncompression required added appreciably to processing time at either of those two stages.
But its worth looking into, particularly if we seek to store a small file size example study region in our code repository (#189)

Actually, the one place that would require a change of file extension from .osm to .pbf is in project setup here:
https://github.com/global-healthy-liveable-cities/global-indicators/blob/6d5c2f010c755f91356d948054e8cbf7fb6e9102/process/pre_process/_project_setup.py#L174
The change in file name would cascade to other scripts.

Incidental to the above, there's a redundant/outdated code line below this that is not used elsewhere and can be deleted:
https://github.com/global-healthy-liveable-cities/global-indicators/blob/6d5c2f010c755f91356d948054e8cbf7fb6e9102/process/pre_process/_project_setup.py#L177

So I just did a quick comparison of the two methods for a small test region in Manchester and then for Greater Manchester, just for 10 iterations each of the respective commands across the two output filetypes.  This used the following code launched from the python console from the preprocess directory.  The default study region was changed manually in project setup script, first as the test_sm_manchester region (about 75 sqkm), then as Greater Manchester (1276 sqkm; about 16 times larger).
import timeit
import subprocess as sp
from _project_setup import *
locale_poly = f"poly_{db}.poly"
feature = f'PG:"dbname={db} host={db_host} port={db_port} user={db_user} password={db_pwd}" {buffered_urban_study_region}'
command = f'python ogr2poly.py {feature} -f "db"'
sp.call(command, shell=True)
command = f"mv {locale_poly} {locale_dir}/{locale_poly}"
sp.call(command, shell=True)
for osm_region in [
	'manchester_osm_20221015.osm',
	'manchester_osm_20221015.pbf',
]:
    print(f'\n{osm_region}')
    command = f'osmconvert "{osm_data}" -B="{locale_dir}/{locale_poly}" -o="{locale_dir}/{osm_region}"'
    print("- osmconvert")
    timeit.timeit('sp.call(command, shell=True)', number=10, globals = globals())
    command = f"osm2pgsql -U {db_user} -l -d {db} --host {db_host} --port {db_port} {locale_dir}/{osm_region} --hstore --prefix {osm_prefix}"
    print("- osm2pgsql")
    timeit.timeit('sp.call(command, shell=True)', number=10, globals = globals())

Here are the results (with some logging output from osm2pgsql removed, as it was verbose; wasn't sure of the flag to hide that for now)
test_sm_manchester_osm_20221015.osm
- osmconvert
17.518547102999946
- osm2pgsql
[removed noisy output!]
79.53437023100014

test_sm_manchester_osm_20221015.pbf
- osmconvert
12.472907305999797
- osm2pgsql
[removed noisy output!]
72.97080051800003

Those times are for 10 iterations; divide by 10 for the average single run time (pretty quick for this small area, which used Greater Manchester's pbf as an input with a bounding box polygon).
So, in terms of speed, pbf actually wins --- at least for a relatively small study region.  Compared to using .pbf format, it was about 1.4 times as long to excerpt the OSM data, and 1.1 times as long to extract .osm data.  Overall, using the .osm format took about 13% longer.
In addition the file size advantage is quite considerable: the osm file (as currently used) is 101 MB; the pbf file is about 5.1MB; that's a huge difference (.pbf is about 20 times smaller)!
The question is, do these benefits scale as study region becomes larger?  Its possible that compress/uncompression may impact performance -- so we'll check for full manchester too:
So, for Greater Manchester:
manchester_osm_20221015.osm
- osmconvert
35.17619475399988
- osm2pgsql
[removed noisy output!]
435.1993458479992

manchester_osm_20221015.pbf
- osmconvert
33.88227260399981
- osm2pgsql
[removed noisy output!]
402.88766323599884

Using the larger data excerpt for Greater Manchester, conversion took on average 1.03 times as long using .osm format for excerpt, while excerpting to Postgis took 1.08 times longer using .osm format.
So, some minor speed efficiencies in the favor of .pbf; however for file size: .osm is 616 MB, compared to 27.7MB using .pbf; again, that's (more than) 20 times smaller using .pbf!  So its a bit of a nobrainer to switch to using this format for the excerpts.
There are limitations to these tests --- I was using my computer (including QGIS, multiple browsers open in Chrome, Notepad++ and OneNote) while running them, so there's background noise.  But given there were 10 iterations, it presumably gives at least some chance for differences in individual runs somewhat to be glossed over.

fyi, the code is in this gist
https://gist.github.com/carlhiggs/bea408e2b24079cb7748a36282acdadb#file-test_osm_vs_pbf-py
should have coded the comparison calculations, but i was rushing to just get this done!


IOSXE/show wlan id <id_number> client stats

Description
Adds parser for show wlan id <id_number> client stats.
Motivation and Context
Takes raw output of show wlan id <id_number> client stats and coverts it to structured data.
Impact (If any)
None
Screenshots:

Checklist:

[X] I have updated the changelog.
[X] I have updated the documentation (If applicable).
[X] I have added tests to cover my changes (If applicable).
[X] All new and existing tests passed.
[X] All new code passed compilation.

IOSXE/show wlan id client stats #298


Revert reason: 'execution reverted'.

🐛 Bug Report for zksync2-java Java SDK
📝 Description
Hello, Sir. I have deployed a Contract to the era_node_test network. Using zysync2-java, I am able to call view methods within the Contract, but I am unable to invoke methods that consume gas.
🖥 Environment

Operating System & Version: [e.g., Ubuntu 22.04]
Other relevant environment details: zksync-java0.3.0

📋 Additional Context
`
17:28:02.420 [main] DEBUG org.web3j.protocol.http.HttpService - {"jsonrpc":"2.0","error":{"code":3,"message":"execution reverted","data":"0x"},"id":9}
17:28:02.420 [main] DEBUG org.web3j.protocol.http.HttpService - <-- END HTTP (87-byte body)
Exception in thread "main" org.web3j.protocol.exceptions.TransactionException: Transaction 0x4f22aee9d0e477914df3b4ab81a2b963c739d27e289df805fbf5bb7e2d3453ed has failed with status: 0x0. Gas used: 130610. Revert reason: 'execution reverted'.
at org.web3j.tx.Contract.executeTransaction(Contract.java:435)
at org.web3j.tx.Contract.executeTransaction(Contract.java:374)
at org.web3j.tx.Contract.executeTransaction(Contract.java:368)
at org.web3j.tx.Contract.executeTransaction(Contract.java:363)
at org.web3j.tx.Contract.lambda$executeRemoteCallTransaction$3(Contract.java:468)
at org.web3j.protocol.core.RemoteCall.send(RemoteCall.java:42)
at Main.main(Main.java:61)
`

Can you maybe share how you call that contract? I assume that this could be a solution for your problem, you should build call data using ZkFunctionEncoder
final Function function = new Function(
                FUNC_FINALIZEWITHDRAWAL,
                Arrays.<Type>asList(new Uint256(_chainId),
                        new Uint256(_l2BatchNumber),
                        new Uint256(_l2MessageIndex),
                        new org.web3j.abi.datatypes.generated.Uint16(_l2TxNumberInBatch),
                        new DynamicBytes(_message),
                        new DynamicArray<Bytes32>(
                                Bytes32.class,
                                org.web3j.abi.Utils.typeMap(_merkleProof, Bytes32.class))),
                Collections.<TypeReference<?>>emptyList());
        return ZkFunctionEncoder.encode(function);

Can you maybe share how you call that contract? I assume that this could be a solution for your problem, you should build call data using ZkFunctionEncoder
final Function function = new Function(
                FUNC_FINALIZEWITHDRAWAL,
                Arrays.<Type>asList(new Uint256(_chainId),
                        new Uint256(_l2BatchNumber),
                        new Uint256(_l2MessageIndex),
                        new org.web3j.abi.datatypes.generated.Uint16(_l2TxNumberInBatch),
                        new DynamicBytes(_message),
                        new DynamicArray<Bytes32>(
                                Bytes32.class,
                                org.web3j.abi.Utils.typeMap(_merkleProof, Bytes32.class))),
                Collections.<TypeReference<?>>emptyList());
        return ZkFunctionEncoder.encode(function);

Hello, Sir. I invoke the Contract through the following method.Thank you,sir.
        EthSigner signer = new PrivateKeyEthSigner(credentials, 260L);         ZkTransactionFeeProvider feeProvider = new DefaultTransactionFeeProvider(zksync,Token.ETH);         ZkSyncTransactionManager transactionManager = new ZkSyncTransactionManager(zksync, signer, feeProvider);         Test t = Test.load(contract_address, zksync, transactionManager, new DefaultGasProvider());         TransactionReceipt receipt = t.setA(BigInteger.ONE).send();         System.out.println(receipt.getRevertReason());

Can you please try to encode it using ZkFuntionEncoder and not one from the web3j

Sir, alright, I will give it a try immediately.

Can you please try to encode it using ZkFuntionEncoder and not one from the web3j

hi,sir I try as below ,but the issue still exists.
      List res = new ArrayList();         String contract_address="0x588758d8a0ad1162a6294f3c274753137e664ae0";         Function contractFunction = new Function(                 "setA",                  Arrays.<Type>asList(new Uint256(123)),                  Collections.<TypeReference<?>>emptyList());         String calldata = ZkFunctionEncoder.encode(contractFunction);         TransactionReceipt receipt2 = wallet.execute(contract_address, contractFunction).send();
the sol file is as below
      pragma solidity ^0.8.0; contract Test{     int a ;     function setA(int b) public {         int c = b;     }     function get() public view returns  (int) {         return a;     } }

Can you please try to encode it using ZkFuntionEncoder and not one from the web3j

Sir ,I can call function by npx zksync-cli contract write --rpc http://<IP_ADDRESS>:8011 , but it doesn't work with ZKSync 2-java sdk.

Here is simple example with storage contract you can try it. If you continue to have problems maybe you can share me your contract, abi and bin.
String bin = "0x0002000000000002000100000001035500000000030100190000006003300270000000130030019d0000008004000039000000400040043f00000013033001970000000102200190000000290000c13d000000040230008c000000420000413d000000000101043b000000e001100270000000150210009c000000310000613d000000160110009c000000420000c13d0000000001000416000000000101004b000000420000c13d000000040100008a00000000011000310000001702000041000000200310008c000000000300001900000000030240190000001701100197000000000401004b000000000200a019000000170110009c00000000010300190000000001026019000000000101004b000000420000c13d00000004010000390000000101100367000000000101043b000000000010041b0000000001000019000000490001042e0000000001000416000000000101004b000000420000c13d0000002001000039000001000010044300000120000004430000001401000041000000490001042e0000000001000416000000000101004b000000420000c13d000000040100008a00000000011000310000001702000041000000000301004b000000000300001900000000030240190000001701100197000000000401004b000000000200a019000000170110009c00000000010300190000000001026019000000000101004b000000440000613d00000000010000190000004a00010430000000000100041a000000800010043f0000001801000041000000490001042e0000004800000432000000490001042e0000004a00010430000000000000000000000000000000000000000000000000000000000000000000000000ffffffff0000000200000000000000000000000000000040000001000000000000000000000000000000000000000000000000000000000000000000000000006d4ce63c0000000000000000000000000000000000000000000000000000000060fe47b1800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000008000000000000000000000000000000000000000000000000000000000000000000000000000000000e69512c94890ba2a8ca933413f57834c632dc3539b4ce6d968c0185b902f8025";
TransactionReceipt result = wallet.deploy(Numeric.hexStringToByteArray(bin)).sendAsync().join();
System.out.println("Contract address: " + result.getContractAddress());

Function increment = new Function("set", Arrays.<Type>asList(new org.web3j.abi.datatypes.generated.Uint256(100)),
                Collections.<TypeReference<?>>emptyList());
wallet.execute(result.getContractAddress(), increment).sendAsync().join();

Here is simple example with storage contract you can try it. If you continue to have problems maybe you can share me your contract, abi and bin.
String bin = "0x0002000000000002000100000001035500000000030100190000006003300270000000130030019d0000008004000039000000400040043f00000013033001970000000102200190000000290000c13d000000040230008c000000420000413d000000000101043b000000e001100270000000150210009c000000310000613d000000160110009c000000420000c13d0000000001000416000000000101004b000000420000c13d000000040100008a00000000011000310000001702000041000000200310008c000000000300001900000000030240190000001701100197000000000401004b000000000200a019000000170110009c00000000010300190000000001026019000000000101004b000000420000c13d00000004010000390000000101100367000000000101043b000000000010041b0000000001000019000000490001042e0000000001000416000000000101004b000000420000c13d0000002001000039000001000010044300000120000004430000001401000041000000490001042e0000000001000416000000000101004b000000420000c13d000000040100008a00000000011000310000001702000041000000000301004b000000000300001900000000030240190000001701100197000000000401004b000000000200a019000000170110009c00000000010300190000000001026019000000000101004b000000440000613d00000000010000190000004a00010430000000000100041a000000800010043f0000001801000041000000490001042e0000004800000432000000490001042e0000004a00010430000000000000000000000000000000000000000000000000000000000000000000000000ffffffff0000000200000000000000000000000000000040000001000000000000000000000000000000000000000000000000000000000000000000000000006d4ce63c0000000000000000000000000000000000000000000000000000000060fe47b1800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000008000000000000000000000000000000000000000000000000000000000000000000000000000000000e69512c94890ba2a8ca933413f57834c632dc3539b4ce6d968c0185b902f8025";
TransactionReceipt result = wallet.deploy(Numeric.hexStringToByteArray(bin)).sendAsync().join();
System.out.println("Contract address: " + result.getContractAddress());

Function increment = new Function("set", Arrays.<Type>asList(new org.web3j.abi.datatypes.generated.Uint256(100)),
                Collections.<TypeReference<?>>emptyList());
wallet.execute(result.getContractAddress(), increment).sendAsync().join();

Thank you, sir, I feel fortunate to receive your help

No problem, I hope we can resolve this issue you are experiencing

No problem, I hope we can resolve this issue you are experiencing

sir,How can I go about obtaining the var  'bin' , for example using  solidity remix  , and the example you provided me is runnable.

No problem, I hope we can resolve this issue you are experiencing

sir, How would I compile the sol file to get the correct bin, I think the bin I got when I compiled sol might not be correct

No problem, I hope we can resolve this issue you are experiencing

sir, How would I compile the sol file to get the correct bin, I think the bin I got when I compiled sol might not be correct

I get the bin by  zksolc-linux-amd64-gnu-v1.5.8 --solc solc-linux-amd64-0.8.28-1.0.1 --bin Test.sol

No problem, I hope we can resolve this issue you are experiencing
Sir, could you please tell me how to compile the sol file to get its corresponding bin? Does bin refer to bytecode? Thank you, sir.

my sol file is as below
// SPDX-License-Identifier: MIT pragma solidity ^0.8.0; contract Test{     int a ;     function set(int b) public {         a = b;     }     function get() public view returns  (int) {         return a;     } }
bin is
0x0000008003000039000000400030043f0000000100200190000000180000c13d00000060021002700000000c02200197000000040020008c000000250000413d000000000301043b000000e0033002700000000e0030009c000000200000613d0000000f0030009c000000250000c13d0000000003000416000000000003004b000000250000c13d000000230020008c000000250000a13d0000000401100370000000000101043b000000000010041b00000000010000190000002c0001042e0000000001000416000000000001004b000000250000c13d0000002001000039000001000010044300000120000004430000000d010000410000002c0001042e0000000001000416000000000001004b000000250000c13d000000030020008c000000270000213d00000000010000190000002d00010430000000000100041a000000800010043f00000010010000410000002c0001042e0000002b000004320000002c0001042e0000002d000104300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000ffffffff0000000200000000000000000000000000000040000001000000000000000000000000000000000000000000000000000000000000000000000000006d4ce63c00000000000000000000000000000000000000000000000000000000e5c19b2d00000000000000000000000000000000000000200000008000000000000000000000000000000000000000000000000000000000000000000000000000000000a68fd5f5e675cd0b548395e3a018f8b8cc0d64b57f9339261ad91efb6af4bb45


Errors after running on Win10

Thanks for your help on this!
Using PowerShell Administrator Prompt.. Python3.10 downloaded from MSFT Store and I get these errors AFTER clearing the Database in BloodHound 4:
New Settings:
DB Url: bolt://localhost:7687
DB Username: neo4j
DB Password: BloodHound
Use encryption: False
Testing DB Connection
Database Connection Successful!
(Cmd) connect
Database Connection Successful!
(Cmd) generate
Starting data generation with nodes=500
Populating Standard Nodes
Traceback (most recent call last):
File ".\DBCreator_randNetAccess.py", line 902, in
MainMenu().cmdloop()
File ".\DBCreator_randNetAccess.py", line 70, in cmdloop
cmd.Cmd.cmdloop(self)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\cmd.py", line 138, in cmdloop
stop = self.onecmd(line)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\cmd.py", line 217, in onecmd
return func(arg)
File ".\DBCreator_randNetAccess.py", line 216, in do_generate
self.generate_data()
File ".\DBCreator_randNetAccess.py", line 304, in generate_data
"DEFAULT DOMAIN CONTROLLERS POLICY"), guid=ddp)
File "C:\Users\Administrator\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\neo4j\work\simple.py", line 218, in run
self._bookmarks, **kwparameters
File "C:\Users\Administrator\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\neo4j\work\result.py", line 114, in _run
self._attach()
File "C:\Users\Administrator\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\neo4j\work\result.py", line 203, in _attach
self._connection.fetch_message()
File "C:\Users\Administrator\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\neo4j\io_common.py", line 167, in inner
func(*args, **kwargs)
File "C:\Users\Administrator\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\neo4j\io_bolt4.py", line 309, in fetch_message
response.on_failure(summary_metadata or {})
File "C:\Users\Administrator\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\neo4j\io_common.py", line 225, in on_failure
raise Neo4jError.hydrate(**metadata)
neo4j.exceptions.ConstraintError: {code: Neo.ClientError.Schema.ConstraintValidationFailed} {message: Node(9) already exists with label Base and property objectid = 'adfce851-e0bb-4129-af67-4033983fa60a'}
PS C:\Users\Administrator\Downloads\BloodHound-Tools-main\BloodHound-Tools-main\DBCreator>

I'm not sure how to recreate this issue, but it does look like an error during one of the queries.
When clearing the data, I find it easiest with the following command: "MATCH (n) DETACH DELETE n".
If this doesn't work, try reinstalling the neo4j and rerun.
Please let me know if the issue persists.


Question regarding directional derivative computation

Hi,
While working with tcav code after reading the related paper " Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)" (https://arxiv.org/pdf/1711.11279.pdf), I had trouble understanding the computation of the directional derivative.
From my understanding of the paper, 'Conceptual sensitivity' is measured by the directional derivative of  logit values with respect to activations at the layer of interest. Thus for scoring you take into account the proportions of image with positive directional derivative.
On the other hand in the code, as shown in the snippet below, sensitivity score seems to be  the directional derivative of loss values with respect to activations at the layer of interest.
https://github.com/tensorflow/tcav/blob/c7f55108e26c66fef9d21754d4d697ed6337ddc3/tcav/model.py#L54-L55
As the loss is a function of the logits, the gradient we obtain in the code depends of the loss function we choose. In your case with cross entropy  (snippet below) using  the article's notation we have :

This leads to a change in magnitude ( which is not used is the score) but also in terms of sign.
https://github.com/tensorflow/tcav/blob/c7f55108e26c66fef9d21754d4d697ed6337ddc3/tcav/model.py#L173-L178
Though  it seems that in the code you compensate the sign change by counting the number of negative gradients instead (last snippet).
https://github.com/tensorflow/tcav/blob/c7f55108e26c66fef9d21754d4d697ed6337ddc3/tcav/tcav.py#L42-L60
Have I misunderstood either the paper or the code ? Or is there a specific reason to use loss instead of logit, as you still obtain the same score by changing sign ?
Thank you for your answer !

Hi again,
@jameswex would you have any updates/explanations regarding the matter ?
Thanks in advance

@BeenKim can you please take a look? Thanks.

Hi,
I also stumbled across this issue. The pytorch re-implementation by @mlomnitz does indeed contain a commit in which this was adressed: https://github.com/mlomnitz/tcav_pytorch/commit/c2b5b968cc0c8c4fab1390328c7771f98c579224
Maybe @mlomnitz remembers what his thought process was when changing it? I know it has been a while though...
Thanks in advance

@jameswex  Any update regarding the matter ? Thanks !

Sorry for this delayed response! We coded such that the two combination (loss and minus sign) would result in the same outcome. The code probably can be cleaned up to make this point clear.. sorry about that. Please reopen if your question remains!


Mouiad

Check those please
Implement Expense API #5
Implement Category API #4
And next time please inform me about the tasks :)

It's not assigned to YOU we said we will do this stuff together.
So I will close it now.


ARROW-5834: [Java] Apply new hash map in DictionaryEncoder

Related to ARROW-5834.
Apply new hash map in DictionaryEncoder to make it work.
Meanwhile provide benchmark for DictionaryEncoder:
DictionaryEncoderBenchmarks#testEncode:
Before: 5  430860.989 ± 14157.675  ns/op
After:   5  415703.943 ± 9258.049  ns/op

Codecov Report

Merging #4786 into master will increase coverage by 2.18%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #4786      +/-   ##
=========================================
+ Coverage   87.41%   89.6%   +2.18%
=========================================
  Files         996     661     -335
  Lines      139489   96300   -43189
  Branches     1418       0    -1418
=========================================
- Hits       121938   86291   -35647
+ Misses      17189   10009    -7180
+ Partials      362       0     -362

Impacted Files
Coverage Δ

cpp/src/plasma/common.cc
76.19% <0%> (-12.28%)
:arrow_down:

python/pyarrow/error.pxi
51.21% <0%> (-1.85%)
:arrow_down:

cpp/src/arrow/status-test.cc
98.3% <0%> (-1.7%)
:arrow_down:

cpp/src/arrow/status.h
95.5% <0%> (-0.38%)
:arrow_down:

cpp/src/arrow/python/serialize.cc
89.61% <0%> (-0.33%)
:arrow_down:

cpp/src/plasma/client.cc
97.1% <0%> (ø)
:arrow_up:

python/pyarrow/__init__.py
40.54% <0%> (ø)
:arrow_up:

cpp/src/plasma/common.h
100% <0%> (ø)
:arrow_up:

cpp/src/plasma/protocol.cc
93.76% <0%> (ø)
:arrow_up:

cpp/src/arrow/python/common.h
98.8% <0%> (ø)
:arrow_up:

... and 347 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ecbe116...2818cbe. Read the comment docs.

LGTM, @tianchen92 can you rebase, i think CI failures should go away.

LGTM, @tianchen92 can you rebase, i think CI failures should go away.

I just rebased yesterday, should I rebase again or just reopen this PR？

+1 LGTM, CI failure is due to rust (unrelated)


[docs] correct some links

This un-hard-codes a few links I found in the docs. Most of those would work fine in most setups but were hard coded to ~/.emacs.d/... so I changed them to ..

Ah, my mistake. Thanks for your help!


feature/FRO-1___adjust-frontend-skeleton-for-implementation

Merging into master

Branch tested and able to merge.


Port 80 or 443  section :  6.3. Configuring probe

A question regarding to Section :6.3. Configuring probe, is it port 80 or 443 ( the picture is showing port 443 )

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e1eac8a5-9217-7d91-d775-e062e7ea9c74
Version Independent ID: ee080620-cc4f-bf5f-2c55-ca06d8deca63
Content: Active Directory Federation Services in Azure
Content Source: WindowsServerDocs/identity/ad-fs/deployment/how-to-connect-fed-azure-adfs.md
Service: active-directory
GitHub Login: @billmath
Microsoft Alias: billmath

its http http://federation url/adfs/probe

Thanks for your dedication to our documentation. If we have been unable to review your issue in a timely manner, we sincerely apologize for the delayed response. We have captured the informtion you have provided for review. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close

#assign:IngridAtMicrosoft
gh issue comment 2200 -b #assign:IngridAtMicrosoft

#assign:IngridAtMicrosoft


Fix: TokenScript file for ENS' events should filter against the owner Ethereum address

Since we are only interested in events concerning the user's wallet.

@hboon that's not necessarily the case here. Since we get the token ids from opensea, it's simpler to derive the ensName from the token id (1:1) rather than the owner address (1:N)

@James-Sangalli  Hmm.. you are right. But it doesn't look like desirable behavior.
Assuming we get a balance of 4 from OpenSea.
If we do owner=${ownerAddress}, we get say 4 events, we might just use those 4 and present them in some order.
On the other hand, if get 5 events with owner=${ownerAddress} (because the user has transferred 1 to another address), how do we know which is the correct 4 to use?
But if we use label=${tokenId}, it's not ambiguous like you say correctly, but that means we need to fetch the same event 4 times, right?
Heads up for @colourful-land
@James-Sangalli would you close this if this is acceptable for now, since we will implement distinct at some point?


Align recipes for cocktails starting with A

I started reviewing all the recipes and trying to align them with each other, make sure they matched the recipe, and tried to avoid having recipes as a direct copy from another source and instead paraphrase. There are lots of cocktails so I've only made it through the ones starting with A so far :laughing:
There were some merge conflicts from other changes, I'm not a git expert so if there's something I'm supposed to do to make it merge nicer then let me know, but I think it's ok now.
Each update to a recipe is in a commit, with each commit description having more specific information about what was changed. Most are just some minor wording changes, but there are a small number with changes to ingredients.
Plus melon liqueur was listed as it's own replacement so I fixed that :laughing:

Thank you! 🚀


issue #49, update README

HI! I'm new to QuTiP and would like to contribute
Added instruction in README.md for formatting .md files with nbQA

Hi @LegionAtol,
thanks for the contribution :)
I left a short comment. When you adapt the changes we can merge your changes to the Readme.md!

Hi, is there something wrong?

Hi @LegionAtol, thanks for the contribution :) I left a short comment.
@christian512 It's not clear to me either where you left this comment

Looks good. Merging.

Thanks @LegionAtol!


 When this happens we just throw an exception. It should be possible to just retry again if this happens as it most likely failed due to a concurrency issue with storage.

Closing issue in favor of #5300. This is currently an issue for the worldstate download pipeline and will be handled at the feature level


MAINT: Refactor forward solution code

Refactor forward solution code to eventually make it easier to add channel types. This PR's goals:

[x] Do a basic refactoring using dict instead of lists
[x] Fix tests (I expect simulation to fail)
[x] Make it more readable
[x] Hopefully simplify the coils/comp-coils (ccoils) stuff so that these can be computed all at once

Okay tests pass locally and I find things much more readable now. @agramfort can you look and see if you agree?
The big structural changes are:

For forward channel types, use a dict-of-coil-type (e.g., sensors['meg']) approach instead of the existing list-of-length-2 (meg, eeg) approach
Instead of _prep_meg_channels returning a list of coils, compcoils, ... it

Returns a dict (e.g., sensors['meg']['defs'] rather than coils_list[0][0] being the coils)
Combines the coils and compcoils into a single list
Adds a post_picks that says what to pick after applying the compensator to get the non-ref channels


Codec derivation usage

I have defined the following abstraction for my API endpoints:
case class Endpoint[In, Out](route: String)(
  implicit
    val inEncoder: io.circe.Encoder[In],
    val inDecoder: io.circe.Decoder[In],
    val outEncoder: io.circe.Encoder[Out],
    val outDecoder: io.circe.Decoder[Out]
)

case class CorrectionView(original: String, correction: String)
val correctionView =
  Endpoint[Long, Option[CorrectionView]]("/correction/view")

With io.circe.generic.simple.auto._, all four instances can be derived automatically.
I would like to use the new Codec functionality to shorten the compilation time. I rewrote Endpoint as follows:
class Endpoint[In, Out](route: String)(
  implicit
    val inCodec: io.circe.Codec[In],
    val outCodec: io.circe.Codec[Out]
)

import io.circe.generic.simple.semiauto._
val correctionView =
  Endpoint[Long, Option[CorrectionView]]("/correction/view")(deriveCodec, deriveCodec)

According to the pull request #1151, the fully automatic codec derivation does not support Codecs. Therefore, I am using semiauto here and pass in deriveCodec explicitly.
I am wondering whether there is a better way to use Codecs. I did try the @JsonCodec annotation which now uses deriveCodec internally, but it does not make any Codec instances available in the scope.
Unfortunately, my rewritten Endpoint does not compile because deriveCodec does not work for primitives:
import io.circe.generic.simple.semiauto._

implicit val longCodec   = deriveCodec[Long]
implicit val stringCodec = deriveCodec[String]

This yields:
could not find implicit value for parameter codec: io.circe.generic.simple.codec.DerivedAsObjectCodec[String]
could not find implicit value for parameter codec: io.circe.generic.simple.codec.DerivedAsObjectCodec[Long]

Is this the expected behaviour?
I am using the latest Circe release with Scala 2.13.0. The scalac flag -language:experimental.macros is enabled. The build contains the following dependencies:
scalaDeps = [
  ["io.circe", "circe-core", "0.12.0-M4"],
  ["io.circe", "circe-generic-simple", "0.12.0-M4"],
  ["io.circe", "circe-parser", "0.12.0-M4"]
]

@tindzk Sorry for the delay in responding, but for now at least this qualification still holds:

Note that the intention is that Codec should only be used to make defining instances more convenient, and not as a constraint.

This isn't set in stone, but it simplifies a lot of things.

Closing for now, since this is documented and expected behavior.


.softValidation argument

To accelerate the process of creating cohorts in a pipeline I think we could have the option of ".softValidation" in our functions, so you can set it to TRUE until the last function in the pipeline. What do you think?

@nmercadeb I agree!

set it to TRUE internally and do not have the argument
export checkCohortRequirements and document it in the vignettes


Update-TTKCache misses at least Microsoft.Resources/tags

After running Update-TTKCache we still seem to be missing the latest version of Microsoft.Resources/tags (2024-03-01 is missing):
$ jq -a '."Microsoft.Resources/tags"' arm-ttk/arm-ttk/cache/AllAzureResources.cache.json
[
  {
    "aliases": null,
    "apiProfiles": null,
    "apiVersions": [
      "2023-07-01",
      "2022-09-01",
      "2019-10-01",
      "2019-05-01",
      "2019-04-01",
      "2019-03-01",
      "2018-11-01"
    ],
    "capabilities": "SupportsExtension",
    "defaultApiVersion": null,
    "locationMappings": null,
    "locations": [],
    "properties": null,
    "resourceType": "Microsoft.Resources/tags",
    "zoneMappings": null
  }
]

Other resources in that cache file have API versions even from 2024-08-XX-preview so something is up.
Using arm-ttk version 2024.03.28 - 0.25.
Let me know if you need anything else.

Deeper down the rabbit hole I go...
Latest versions from differing Microsoft sources (ttk post running Update-TTKCache):

Microsoft.Web/serverFarms:

ttk: 2023-12-01
doc: 2022-09-01
REST: 2023-12-01

Microsoft.Resources/tags

ttk: 2023-07-01
doc: 2024-03-01
REST: 2024-08-01

Microsoft.Compute/virtualMachines

ttk: 2024-07-01
doc: 2024-07-01
REST: 2024-07-01

From this it looks like:

REST I am assuming is 'truth' and reflects the latest version available for a resource type
Azure user documentation lags somewhat behind REST a little which is a bit unexpected, it would be helpful to release the docs before the API version and just state attempts to use an API version earlier than the date it represents is 'undefined' behaviour
TTK, uses the output of az provider list which lags a little behind REST which I suppose is to be expected as API versions are region specific; as an example even nearly a year later there is a warning in regards to interacting with IMDS

The problem here is TTK is beating up its userbase on in-actionable items, such as how can I fix Microsoft.Web/serverFarms when there is no documentation for that API version?
What is the actionable solution here for the user facing these error messages?


[turborepo] Heroku build failing on v1.7

What version of Turborepo are you using?
1.7.2
What package manager are you using / does the bug impact?
pnpm
What operating system are you using?
Linux
Describe the Bug
After updating to 1.7.2 I'm unable to build my app on Heroku (Heroku-22 stack).
thread 'main' panicked at 'Failed to execute turbo.: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', crates/turborepo/src/main.rs:50:10
Full stacktrace: https://pastebin.com/v67Z624V
Expected Behavior
Builds should pass like on v1.6.3 which I'm downgrading to for now.
To Reproduce
Attempt to build a node app with turbo v1.7.2 on Heroku
Reproduction Repo
No response

It looks like one of our executables isn't getting the correct permissions set. Are you doing anything out of the ordinary with how you're installing turbo?

Nope, Turbo is added as a project dependency and is installed along with the rest via PNPM. No tweaks were applied whatsoever. Everything installs fine, here's the complete build log. Rolling back to 1.6.3 helped immediately without any other changes to the environment.

Could you get me the build log with -vv passed to the turbo command? I'm guessing that the buildpack is doing something that we're not expecting and I'm hoping those logs will give a better picture of where the binaries are on disk.

Hello, I am encountering the same error.
Here are my logs with -vv option :
[DEBUG] turborepo_lib::shim: Global turbo version: 1.7.4
[DEBUG] turborepo_lib::shim: Local turbo version: 1.7.4
[DEBUG] turborepo_lib::shim: Running local turbo binary in /home/xxx/project/node_modules/.bin/turbo

[DEBUG] turborepo_lib::shim: supports_skip_infer_and_single_package true
thread 'main' panicked at 'Failed to execute turbo.: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', crates/turborepo/src/main.rs:50:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 ELIFECYCLE  Command failed with exit code 101.
 ELIFECYCLE  Command failed with exit code 1.
 ```

Dowgrade to<EMAIL_ADDRESS>works for me : pnpm i -D -w<EMAIL_ADDRESS>

Sorry for the delayed response Chris. I've just bumped to 1.7.4 and triggered a build - which passed. I've then retried on the same branch and it crashed again. Here's the new log: https://pastebin.com/ZBBWqHU2

Odd that it only happened the second time. Do you know if heroku does anything with file permissions of binaries in node_modules?

same issue with 1.8.2

I was having the same issue with 1.8.2, downgraded to 1.8.1 and it works again. 🤷

Using v1.6.3 for a month is my solution.
@chris-olszewski are you aware of an executable change Vercel did between v1.6.x and v1.7.x? I tried to look at the diff betwen those tags but they are too many things and I don't know well how turbo is structured :D
Thank you,

I'm experiencing the same issue with turbo 1.8.3. It runs fine locally, but fails when I run it in a Docker container. I'm using the node:18.14.2-alpine3.16 image.
In my case downgrading to v.1.6.3 doesn't solve the problem.

It turns out the error is caused by the missing libc6-compat library (depending on your distro the package might have a different name).
This isn't a problem with turbo, it's a consequence of using Rust. Its binaries depend on this lib.

This could possibly be fixed by #4123.

I've test ran 1.8.4-canary.6 and it indeed seems to be working. Will confirm after the stable version hits. Thanks!

I would appreciate if anybody who is experiencing this issue can update to 1.8.5 to see if that addresses their problems.
The change from 4123 is a bit disconnected from what is likely the root cause so I'm not 100% certain that it is a general fix.
I will note that, if this works for @pete-willard I will be closing this issue, locking it, and requesting that everybody else post new issues so we can differentiate the root cause.

I would appreciate if anybody who is experiencing this issue can update to 1.8.5 to see if that addresses their problems.
The change from 4123 is a bit disconnected from what is likely the root cause so I'm not 100% certain that it is a general fix.
I will note that, if this works for @pete-willard I will be closing this issue, locking it, and requesting that everybody else post new issues so we can differentiate the root cause.

just tested 1.8.5 seems to have fix this issue for me

Since @pete-willard reports that it worked on a canary & @emko notes that it works on latest I'm going to preemptively close, and lock after I hear back. 👍

Apologies for the delayed response, Nathan. We haven't had an issue since updating. Go ahead and lock.


trustedGrub build broken

Issue description
TrustedGrub fails to build.
make[2]: *** [Makefile:7543: libgrubmods_a-grub_script.yy.o] Error 1

Steps to reproduce
nix-build '<nixpkgs>' -A trustedGrub

Technical details

System: 17.03pre98682.f673243 (Gorilla)
Nix version: 1.11.4
Nixpkgs version: 17.03pre98682.f673243

It seems to be. Should this be worked around in the same way?

Not anymore:
$ nix-build ~/Projects/nixpkgs/ -A trustedGrub
[...]

Still doesn't compile, but I wonder if anyone still uses it.


padding between the Y axis and the first X axis tick

Expected behavior
padding between the Y axis and the first X axis tick is not working as expected in previous version it is working. In latest version it is moving y-axis also . I have requirement to have some space between Y-axis and x- axis first tick mark. But It is not working as expected.
Below is the example which is working . I want same behavior in latest version
https://jsfiddle.net/AbhishekAdusumilli/hknbd7c0/5/
Current behavior
padding between the Y axis and the first X axis tick is not working as expected in previous version it is working. In latest version it is moving y-axis also . I have requirement to have some space between Y-axis and x- axis first tick mark. But It is not working as expected.
Below is the example which is working . I want same behavior in latest version
https://jsfiddle.net/AbhishekAdusumilli/hknbd7c0/5/
Reproducible sample
https://jsfiddle.net/AbhishekAdusumilli/hknbd7c0/5/
Optional extra steps/info to reproduce
No response
Possible solution
No response
Context
No response
chart.js version
v3.7.0
Browser name and version
No response
Link to your project
No response

We are also facing a same issue and now had to downgrade the version. This limitation does not allow us to move to the latest version


Wiki lists non-standard element property in JS example

The example in Using Snackbars for notifications uses innerText to set the text of a notification:
  text.innerText = message;

innerText is a non-standard implementation of textContext that isn't (and won't be) supported in Firefox. I'd suggest replacing it with the standard textContent or, for better support, innerHTML.
Love MDL by the way so thanks for the amazing work :heart:

I hasn't realised the current way Snackbars/toast works makes the innerText usage redundant. Here's an updated version of the example function:
function createToast(message) {
  'use strict';
  var snackbar = document.createElement('div'),
  text = document.createElement('div'),
  button = document.createElement('button'),
  data = {
    message : message,
  };

  snackbar.classList.add('mdl-snackbar');
  snackbar.classList.add('mdl-js-snackbar');
  button.classList.add('mdl-snackbar__action');
  text.classList.add('mdl-snackbar__text');

  snackbar.appendChild(text);
  snackbar.appendChild(button);
  document.body.appendChild(snackbar);

  componentHandler.upgradeElement(snackbar);

  snackbar.MaterialSnackbar.showSnackbar(data);
}

Here's it working after running createToast('Testing Foo Bar') (it fades as it should):

Ah, whoops. I forgot to remove that wiki page. It is very outdated and referenced an old version of snackbar where I attempted to not provide JS internally but let developers do as they wanted in that regard.
I'll go ahead and remove the wiki page so it doesn't cause anymore confusion. They way to use snackbars is provided in the snackbar documentation. Sorry for having left the page up when it wasn't relevant. Thank you for reporting it to us.


added Wikimedia tiles

Hi,
I've added wikimedia tiles who looks more like GMaps.
Thanks

Another commit has changed the way providers are added. Please look at that. Another PR would be acceptable.

Seriously bro, after a 1 year reply I would certainly not push another one !


Convert Inputs to Blade Components

ONLY immediately reactive components should use Livewire. Everything else needs to be moved to Blade components.
Should we need Livewire to use an input or other blade component in Livewire, we can wrap the Blade components.

Everything is Blade/Streams right now..


update to pcidevices chart

changes to pcidevices chart to support additional webhook.

@ibrokethecloud Could help backport to release branch once pcidevices is ready and tagged. Thanks.

@bk201 already on it.


What is the difference between tags and categories?

You want to keep your site organized with Categories. But at the same time, for SEO purposes you want to assign  the article as many keywords as possible. So consider tags as keywords.
But I don't think it's that important on Github pages, which is why I never considered tags unless requested,  because as you know there are not actual permalinks for each categories & tags but #.
Anyway, if not for SEO, the blogger might want to guide the user with related tags so...it doesn't hurt...

Okay, thanks. I don’t think google takes keywords into account anymore, but I presume you can sort by tags?
Thanks!

My idea for my site is having the anime category, and then tagging the article with the anime name. Is this possible?
Also, can tags have spaces?
Thanks!

Yes, I think that's great. This way if you write about more episodes, the user can get to the posts of the other episodes of the show easily.
Structure is same as category. We know we fixed the space for category so it should work in a similar way.

Great, thanks for the great updates. Appreciate it!


Add Validation of CIMInstances in Test-Office365DSCParameterState

This Pull Request (PR) fixes the following issues
Fixes #186

This change is

Codecov Report

Merging #227 into Dev will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##              Dev     #227   +/-   ##
=======================================
  Coverage   89.38%   89.38%
=======================================
  Files          55       55
  Lines        4976     4976
  Branches        8        8
=======================================
  Hits         4448     4448
  Misses        520      520
  Partials        8        8

Impacted Files
Coverage Δ

...SearchResultSource/MSFT_SPOSearchResultSource.psm1
92.02% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b15eae...e801d95. Read the comment docs.

@ykuijs Bugging you for a code review once again :)

On it!


Update QAP User Guide for 1.0.8

[ ] The ability to exclude zeros from anatomical input files with the "exclude_zeros" flag.
[ ] Make sure QAP metric descriptions are up-to-date.
[ ] New Pipeline ID/name config field.
[ ] New wording and changes for some of the config fields (num participants at once, template_skull_for_anat -> template_head_for_anat, etc.).
[ ] New JSON output files.
[ ] Update new AMI information once it's created.

(More may be added as detail comes in.)

Best practice would be to refer to the 1.0.8 milestone and view the Closed Issues to see what has changed.


Add a NIST Validated CVE Scanner for use in disconnectecd networks

For environments without internet access, we would still like the ability to perform CVE scans of managed hosts.  Having a scanner that has been NIST Validated would also benefit for use in US Govt organizations.  NIST Validated products are here: https://csrc.nist.gov/projects/scap-validation-program/validated-products-and-modules.  OpenSCAP may serve as a good starting point.  Would like input from others on views of OpenSCAP vs other tools.  Pros/Cons, etc.  This is related to #412.

https://jira.corp.adobe.com/browse/CSTTEAM-44225

@dgmorrisjr it looks like vulners isn't able to do offline scanning unless you pay for their commercial product:
https://github.com/vulnersCom/api/issues/13#issuecomment-480479271
@basepi is CVE auditing of non-internet connected hosts in the roadmap still?
I have a few ideas and if I can get them working I will submit a PR.

It's not a priority for us at the moment, though we would like to support it. I figured we'd go with something openscap related, as @dgmorrisjr recommended. Though keeping it up to date will be a whole different thing unless we can consume someone else's service.

@dgmorrisjr have you used oscap before?

While I have it our platform. I’ve been using an older version of hubblestack that lets me use the downloaded files from vulnrs.com.   I’ve been thinking abt upgrading to use the latest Hubble and then build a module for openscap. Just haven’t had the time.

@dgmorrisjr my team has submitted a PR you might be interested in
https://github.com/hubblestack/hubble/pull/654
Long term we would like to be able to point it at a local copy (either cached locally or on the system) of the xml files for the CVE definitions.
This isn't NIST validated, but it does fill the need for air gapped environments.


Another try to fix mission progress crash

The filtering was done to early and meant that we would send mission progress with equal values in rapid succession. Presumably this triggered a crash in the backend.
The error in GRPC was: GRPC_CALL_ERROR_TOO_MANY_OPERATIONS.

Already built and uploaded to s3, so time to merge this as well :smile: .