[RIP-7] Fix some issue and Optimization code for multiple_disk_dir

What is the purpose of the change
Add Multiple Directories Storage Support for commit log.
Detailed info is in RIP-7
Brief changelog
(1).Check whether the disk is full and available before creating the file
(2).When deleting files, change the calculation method of disk remaining space
(3).Support for online write-off paths
Verifying this change
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

@vongosling @Jason918 @liuruiyiyang @duhenglucky @RongtongJin please help to review this pr together.Thanks

I am closing this as RIP-7 was merge with another PR #3357


Oauth Error after I enter my Github credentials

Trying to use Boxen to set up a new MBP and keep receiving the following error:
--> Something went terribly wrong.
    I was able to get your OAuth token, but was unable to use it.

Not really sure how to proceed. Any help would be much appreciated.

Hey @IAMRYO! Me too. :)

@IAMRYO Try deleting your Boxen personal access token in your GitHub settings. That seems to move it along, at least.

Bam! That did it! Thanks @seanknox

There's a better description of the workaround on this issue.


Fix README elasticsearch-model

I found an inconsistency method name, inside README of elasticsearch-model's Asynchronous Callbacks section.
You wrote Article Model file as below.
class Article
  include Elasticsearch::Model

  after_save    { Indexer.perform_async(:index,  self.id) }
  after_destroy { Indexer.perform_async(:delete, self.id) }
end

But, you wrote example implementation of the Indexer worker class as below.
class Indexer
  include Sidekiq::Worker
  sidekiq_options queue: 'elasticsearch', retry: false
  ［...]
  def perform(operation, record_id)
    ［...]
  end
end

I think we should fix Indexer worker class's method name from perform to perform_async.

@katsuhisa91 thanks!


darwin: Creation of some controls within a second window causes a crash.

When creating certain controls within a second window through a user callback, the application crashes.
Here an example, or you can play around with the modified hello-world on this branch.
#include <stdio.h>
#include <ui.h>

int onClosing(uiWindow *w, void *data)
{
	uiQuit();
	return 1;
}

int onClosingSub(uiWindow *w, void *data)
{
	return 1;
}

void buttonComboboxWindowOnClicked(uiButton *b, void *data)
{
	uiWindow *w;
	uiCombobox *child;
	w = uiNewWindow("Combobox", 300, 30, 0);
	uiWindowOnClosing(w, onClosingSub, NULL);
	child = uiNewCombobox();
	uiWindowSetChild(w, uiControl(child));
	uiControlShow(uiControl(w));
}

int main(void)
{
	uiInitOptions o = {0};
	const char *err;
	uiWindow *w;
	uiButton *b;

	err = uiInit(&o);
	if (err != NULL) {
		fprintf(stderr, "Error initializing libui-ng: %s\n", err);
		uiFreeInitError(err);
		return 1;
	}

	// Create a new window
	w = uiNewWindow("Hello World!", 300, 30, 0);
	uiWindowOnClosing(w, onClosing, NULL);

	b = uiNewButton("Combobox");
	uiButtonOnClicked(b, buttonComboboxWindowOnClicked, NULL);
	uiWindowSetChild(w, uiControl(b));

	/* Interestingly enough this works AND prevents the crash from happening
	 * when triggering the creating through the callback later on. */
	//buttonComboboxWindowOnClicked(uiButton *b, void *data)

	uiControlShow(uiControl(w));
	uiMain();
	uiUninit();
	return 0;
}

Crash log:
2022-12-04 00:29:41.337 hello-world[10896:501236] [libui] ../darwin/map.m:25:uiprivMapDestroy() POSSIBLE IMPLEMENTATION BUG; CONTACT ANDLABS:
attempt to destroy map with items inside
zsh: illegal hardware instruction  ./build/meson-out/hello-world

Controls I tested that exhibit that behavior:

uiCombobox showcase
uiSlider showcase

Probably many more? Code that does weird things with delegates and uiprivMap internally? Interestingly enough uiButton seems to be fine, although it uses uiprivMap for the delegates.

While triggered in a slightly different way, I stumbled across another way of crashing applications while working on a new control. This breaks uiButton, which, for whatever reason, is not affected by the first code path:

uiButton showcase in qa > uiSplit

This just reinforces my initial assessment: the weird singleton delegates must go.

I get another issue with the same crash information. Maybe related : #160

I get another issue with the same crash information. Maybe related : #160

Guess you were quicker. Yes definitely the same issue from what I can tell! I'll fix uiWindow next and mark you in the PR so you can check if that fixes things. Thanks for reporting!


Fontstack lacks punctuation characters

The fontstack is missing certain punctuation characters that commonly occur in various languages including English. These punctuation characters are omitted from the labels.
Loveland–Madeira Road (missing en dash):

Cindy’s Friendly Tavern (missing right single quote, aka apostrophe):

„Die 2“ (missing German quotation marks, which have no ASCII equivalent):

Storå;Højris Å, which we should be rendering as “Storå • Højris Å”:

The last example shows that we’ve effectively regressed #666 on linear features.

Per Slack, the Combining Diacritical Marks block is also missing. The General Punctuation, Combining Diacritical Marks, and Spacing Modifier Letters blocks are necessary for a lot of languages, even though none of them is strongly associated with a particular language.


Added more improvements about the I18n support

Hi @avoinea
I fix the error about "intl" reference no found, also test the addon using the command "make project", and run the backend and frontend by separate, I test the addon, adding a "Tabs block", and set it, for many example using the template options avalaible.

@avoinea Can you checkout this pull request, if all ok can you merge it? I hope this time that all QA checkouts pass

Hi @macagua
I went ahead for this release to merge this pull request https://github.com/eea/volto-tabs-block/pull/62
which I have checked and it worked bringing in pt-br and Spanish translation.
Can you please merge the changes from our develop branch in your fork and make the necessary merge
changes so that we only add any i18n  strings that are not already translated from that pull request?
Thank you for your effort.


FallbackResource fails for root level routing.

In the documentation on the Webserver Configuration page, the it is suggested to use FallbackResource as an alternative to the mod_rewrite rules.  This works for most pages, but for requests to '/', when '/' is a defined route, the webserver is unable to return appropriate data.  On Chrome, it gives an encoding error.  Debugging shows the response is being generated, but no browser is able to receive it.
Switching back to mod_rewrite fixed the problem completely.

Closing this as there is nothing we can do, except perhaps added more info in the docs. I will let someone do that as I haven't used Apache for years.


How to test writeUnboundedToBigQuery within JobTest

Scio CustomOutput doesn't work because it returns nothing and we need dead letters.
TransformOverride is a hack but how to easily test output?

Done by extending Scio API locally but it should be accepted by Scio maintainers, see:
https://github.com/spotify/scio/issues/4995


cmd/govim: add support for generated files changing

On the gopls side this is being tracked by https://github.com/golang/go/issues/31553
On Vim/govim side we are going to need to add some level of file watcher, because this is not something that Vim supports out of the box. Just to be explicit, the requirement here is that any file considered as "relevant" by gopls should effectively be monitored by the LSP client (govim). Vim "monitors" files that it has open in buffers but doesn't do anything automatically (i.e. the user can choose to ignore the fact the file on disk has changed underneath the buffer).
So I think the requirement here is:

gopls to tell govim the set of file globs it should watch
Vim to be in "control" of the subset of files matching those globs that are open in buffers, i.e. govim to use the existing didOpen/didChange notification for those files, not the file contents of disk
govim to watch what remains and notify via didChangeWatchedFiles

Also part of https://github.com/golang/go/issues/31086

@findleyr is kindly going to take this on!


Check if strings are anagram or not

Question asked in interview from intimetec, Jaipur

This test already exists in here: https://github.com/adityaarakeri/Interview-solved/blob/master/LeetCode/Anagram.py
could you please add some other test?

Dup of https://github.com/adityaarakeri/Interview-solved/blob/master/LeetCode/Anagram.py

Just a suggestion, there should be a file that contains the list of questions that have already been submitted in this repository. If you want I can create it


DOCS-1718: Publishes OSS 3.26.2

Product Version(s):

Issue:

Link to docs preview:

SME review:

[ ] An SME has approved this change.

DOCS review:

[ ] A member of the docs team has approved this change.

Additional information:

Merge checklist:

[ ] Deploy preview inspected wherever changes were made
[ ] Build completed successfully
[ ] Test have passed

FYI @mgleung


Update terminology

Document => Documentation

I mean, the text should be Documentation, short for Documentation site, rather than document, which refers to a file


Issues with "Resend Invite"

See https://opencollective.com/babel/expenses/132669
Issues:

I can see the "Resend" button, even though I'm not an admin (it's even there when logged out)
It silently errors (no toast)
Error message from API is incorrect (You don't have the permission to edit this expense.)

cc @kewitz

Context: detected while investigating https://opencollective.slack.com/archives/C6JTTA4SK/p1681217531812429

Also some responsiveness issues


Create UI to select cloud provider

At the moment we support only AWS but in the future I would like add support for other cloud services as well.
For this feature, I would like to create a form for selecting from a list of cloud service providers:

Amazon AWS (default)
Microsoft Azure
Google Cloud

Selecting the yet unsupported options should render a "popup" saying that these providers are not yet supported. Selecting the AWS option should just proceed - and in this case exit the application as no other functionality exists.

AT this point I think I should create separate projects for the various cloud providers. But even before that, I should also consider a separate project for the data models and base classes... Thinking time...


Adding minimum X, Y and Theta speed limits to the planner

When using TEB local planner with some robots that have a lower speed limit for the motors, TEB tends to get 'suck' close to the goal due to the robot not moving.
This PR aims to avoid this via a minimum speed limit

Hey there. I'm not a maintainer, and it is not my decision to make, but in my opinion, it would be better to

rebase your commits to have a coherent/cohesive commit history, and
remove all indentation-related changes to keep the PR focused on the actual changes and avoid conflicts.

Thanks for this PR. However, we have some concerns merging this since overwriting the optimal motor commands with some custom values can have some unwanted side effects on the closed-loop performance. This is against the basic concept of predictive control. If you want to deal with friction forces, you have to use a dynamic model. Unfortunately, the TEB optimization model does not support this at the moment. Another option would be to switch to a custom control law very closed to the goal ensuring proper convergence under those additional constraints. Have you experienced any issues?

Thanks for this PR. However, we have some concerns merging this since overwriting the optimal motor commands with some custom values can have some unwanted side effects on the closed-loop performance. This is against the basic concept of predictive control. If you want to deal with friction forces, you have to use a dynamic model. Unfortunately, the TEB optimization model does not support this at the moment. Another option would be to switch to a custom control law very closed to the goal ensuring proper convergence under those additional constraints. Have you experienced any issues?

We have discussed quite a lot about minimum velocities on this PR (I point to the final conclusions)
This kind of addition can be very problematic given given the general TEB architecture, so we will defer them by now, until someone with a deeper understanding takes a more rigorous look
All that said, thanks for the contribution, and please don't feel discouraged to do more in the future!


Precision bug in optax.softmax_cross_entropy_with_integer_labels

Loss values are quantized when logits are supplied as bfloat16 due to log_normalizers getting quantized after computation.
https://github.com/google-deepmind/optax/blob/6f1291108f1ec2af3ce1f0191296d5b2a56cefab/optax/losses/_classification.py#L265-L271
Example:
b, c = 8, 100
key = jax.random.key(42)
loss_fn = optax.softmax_cross_entropy_with_integer_labels

# bug
logits_b16 = jax.random.uniform(key, (b, c), jnp.bfloat16)
labels = jax.random.randint(key, (b,), 0, c)

print(loss_fn(logits_b16, labels))
# [4.75 5 4.59375 5.0625 4.4375 4.15625 4.3125 4.53125]

print(loss_fn(logits_b16.astype(jnp.float32), labels))
# [4.7477226 4.9903674 4.592032  5.0644608 4.4306226 4.1690445 4.301924 4.52517  ]

Possible fix: cast logits to float32 within the function.
There is a performance penalty for large values (but practically negligible?). No penalty if logits are already float32.
b, c = 4096, 50_000  # ~200M elements
key = jax.random.key(42)
jitted_fn = jax.jit(optax.softmax_cross_entropy_with_integer_labels)

labels = jax.random.randint(key, (b,), 0, c)
logits_b16 = jax.random.uniform(key, (b, c), jnp.bfloat16)

# Warmup.
jitted_fn(logits_b16, labels).block_until_ready()

%timeit -n 100 jitted_fn(logits_b16, labels).block_until_ready()
1.17 ms ± 2.98 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

%timeit -n 100 jitted_fn(logits_b16.astype(jnp.float32), labels).block_until_ready()
4.06 ms ± 1.79 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

hey @MasterSkepticista . Perhaps I'm missing something, but it seems to me that the difference between the result in bfloat and float32 is what you would expect from a low precision type like bfloat. In your example, the relative error is around 1e-3, which is what I would expect since bfloat16 has only 3 significant decimal digits.

I'm closing this for now but feel free to reopen.


Return empty 200 Response as default

Like itty-router, just return an empty 200 Response as default if there's no response specified.
http methods include GET/POST/PUT/PATCH/DELETE. Most of the time only the GET method return a details response, other methods return an empty 200 Response.
So that, by implementing this feature, we can remove redundant return c.body() from the routes.

Hi @chientrm,
I don't think it's a good idea to return a 200 Response as the default. There is possibility that users don't desire it to be 200, but 404 or throwing the error instead. And, basically, we designed it so that the "handler should return a Response object."

@usualoma @metrue @ThatOneBro or others:
What do you think about it?

Hi @chientrm @yusukebe
If a handler returns undefined, it means "the handler did not process anything" and not "success, but the response is empty." This is a necessary specification for the routing fallback mechanism.
Also, in general, it is preferable to return a meaningful response from even other than GET.
Thus, I think no need to return an empty 200 response.

I agree with @usualoma, it can lead to unexpected behavior and false positives that a route is working. Probably better not to default to a 200 response.

We could close this.


[Feature Request] Grid View for platforms

Currently to choose platform i want to play i must press RT or LT about 10-15 times (buttons at the bottom are too small to click).
Add option to show installed platforms as grid:
Current view:

ATARI
NES
SMC
SNES
SMD
N64
PS1
PS2
PSP
NDS
...

Requested feature - add optional grid view like this:

ATARI
NES
SMC
SNES
SMD

N64
PS1
PS2
PSP
NDS

GCUBE
DCAST
Wii
3DS
...

This would be a good option for users with 15+ platforms installed - all platforms could be visible simultaneously.

It was planned irrc.


fixed: #735, #775 toast on back pressed in home activity, sorted Allergens alphabetically

Description

Now showing a toast saying "Please press back again to exit" on button pressed.
Allergens have been sorted alphabetically.

Related issues and discussion
#735 , #775
Screen-shots, if any

@teolemon , @policorn please review my latest commit as I've solved the issues mentioned.

Hi @AbsolutelySaurabh, this PR fix for #735 works as expected. Regarding #775 I'm missing the filtering option mentioned in the issue ticket, how difficult would it be to implement it? Can you split this PR so we can merge your fix for the exit confirmation?

closing this PR,  as I'll made new-one for both issues separately.


Property Widget 1.0

1 last thing, mind updating the root readme? moving up property-grid to 4.x compatible


Temporarily suppress push on platform-with-runcmd test

Suppress image push on the platform-with-runcmd as a workaround to get main builds working until #198 is resolved.

/test-force-approve
Successful ci-branch run here: https://github.com/devcontainers/ci/actions/runs/4114228096


#148 happens again

Just to let you know, #148 seems to be happening again.
Please double check but there's no more third-party licenses nor anything similar in the root of the project and the history has been wiped again.

avelino/awesome-go#1135 merged.

The commit that stripped the LICENSE.

Nulled and fixed

@julienschmidt hope it never happen again

thanks for awesome package!

@julienschmidt lol you're quite the scum xD

@julienschmidt I deeply apologize for @kryptodev's behavior.


Training on Scannet

Hello, I didn't find the data loader for Scannet, could I modify the file dataloader_nyu.py to load scannet data? Are there any specific modifications required for scannet? Thanks so much.

You should multiply the surface normal vector by -1. Besides that, the dataloader can be implemented in the same way.

Thanks so much for your reply.


UnhandledPromiseRejectionWarning: Error: write EPIPE

Given example is not working.
Please see the code
`
const fromPath = require("pdf2pic").fromPath;
const options = {
density: 100,
saveFilename: "untitled",
savePath: "./images",
format: "png",
width: 600,
height: 600
};
const storeAsImage = fromPath("./images/sample.pdf", options);
const pageToConvertAsImage = 1;
storeAsImage(pageToConvertAsImage).then((resolve) => {
console.log("Page 1 is now converted as image");
return resolve;

});
`
This results in an error
(node:20252) UnhandledPromiseRejectionWarning: Error: write EPIPE     at afterWriteDispatched (internal/stream_base_commons.js:156:25)     at writeGeneric (internal/stream_base_commons.js:147:3)     at Socket._writeGeneric (net.js:785:11)     at Socket._write (net.js:797:8)     at writeOrBuffer (internal/streams/writable.js:358:12)     at Socket.Writable.write (internal/streams/writable.js:303:10)     at ReadStream.ondata (internal/streams/readable.js:719:22)     at ReadStream.emit (events.js:315:20)     at ReadStream.Readable.read (internal/streams/readable.js:519:10)     at flow (internal/streams/readable.js:992:34) (Usenode --trace-warnings ...to show where the warning was created) (node:20252) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:20252) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
`
How can I resolve this?

Такая же ошибка!

you mean,
is this a known mistake?

I'm having this issue also.

bump

For Some reason below worked for me.
`
const fromBase64 = require("pdf2pic").fromBase64;
					const options = {
						density: 96,
						saveFilename: save_file_name,
						savePath: savePath,
						format: "jpg",
						width: 1654,
						height: 2339
					};

					const buffer = fs.readFileSync(res.full_pdf_file_name, {
						encoding: 'base64'
					});
					const storeAsImage = fromBase64(buffer, options);
					storeAsImage(1).then(async (resolve) => {
						console.log(' saveAs image ==>', resolve);
						console.log("Page 1 is now converted as image");
						if (resolve) {

							await UserPdfSchema.findOneAndUpdate({
								_id: req.params.id
							}, {
								pdf2jpgName: resolve.name,
								pdffilename: filename,
								isUploaded: true,
								pdf2jpg: true,
								isActive: true
							});

						}
					});

`

For Some reason below worked for me.
`
const fromBase64 = require("pdf2pic").fromBase64;
					const options = {
						density: 96,
						saveFilename: save_file_name,
						savePath: savePath,
						format: "jpg",
						width: 1654,
						height: 2339
					};

					const buffer = fs.readFileSync(res.full_pdf_file_name, {
						encoding: 'base64'
					});
					const storeAsImage = fromBase64(buffer, options);
					storeAsImage(1).then(async (resolve) => {
						console.log(' saveAs image ==>', resolve);
						console.log("Page 1 is now converted as image");
						if (resolve) {

							await UserPdfSchema.findOneAndUpdate({
								_id: req.params.id
							}, {
								pdf2jpgName: resolve.name,
								pdffilename: filename,
								isUploaded: true,
								pdf2jpg: true,
								isActive: true
							});

						}
					});

`

Is not working for me 👎

bump!

I'm having the same issue also. Could not make it run, while other npm packages with the same dependencies are fine.

Yes, I was able to do the same. using an older version of gm solved it

What version of GM?

"gm": "^1.23.1",
"pdf2pic": "^2.1.4",
(Check with above )


[WW-5096] Fix StaticParametersInterceptor param overwrite

Fixes WW-5096

Hi @lukaszlenart  and @akash5885 .
It looks like the original code of StaticParametersInterceptor addParametersToContext() was doing the same operations if overwrite was true or false, just in reverse-order depending on the boolean value. I think that was what @akash5885 stated in the JIRA ?
Did the different ordering of the old code blocks have any effect at all, or was the end result the same whether overwrite was true or false ?
Is there a possibility for a manual test example or test case that clarifies how the original code was not working as intended, and how the updated code does ?
I am not sure I am understanding the original problem correctly, so I figured I would ask.  😄

HI @JCgH4164838Gh792C124B5 @lukaszlenart
Apologies I didn't mention exact scenario that triggered this bug. But now I have added scenario in JIRA ticket WW-5096 which triggered this bug.
@JCgH4164838Gh792C124B5
yes, you are correct.original code of StaticParametersInterceptor addParametersToContext() was doing the same operations irrespective of overwrite is true or false.

Looks like the bug has emerged by here and we can just restore previous logic.
One test fails. And not sure if we simple have new test case from provided scenario?

@akash5885 thanks a lot for you description and test case - based on that I was able to prepare a few unit tests to cover the logic. Please review and let me know.

Coverage increased (+0.009%) to 47.305% when pulling d591b7e3b956ad34e1204da9e60382cc2fc4b2bf on WW-5096-fix-static into 482af41673a3883e904ea72391a5b4a03cbd5d94 on struts-2-5-x.

LGTM :100: Merely I can't recall if integration tests were done in PR builds but currently I see they aren't. Basically I remember STRUTS_IT=true was defined by me and was required to be set in Travis params, any idea?

Ooops sorry that was on only master branch and PRs; I realized now

OK I cherry picked against master; Lets merge if passed integration tests there as well :)


boost 1.69

Hi there,
just wanted to report an error I had during compilation. I run Ubuntu 18.04 with boost 1.69 and I get following errors:
/usr/bin/c++   -pipe -fmessage-length=0 -std=c++11 -Wuninitialized -fPIC -O3 -DNDEBUG  -pthread -static-libstdc++ -Wl,--strip-all -static-libstdc++ -rdynamic CMakeFiles/os_linux_test.dir/Os_Linux_test.cpp.o  -o os_linux_test ../../lib/libos.a -Wl,-Bstatic -lboost_date_time -lboost_unit_test_framework -lboost_program_options -lboost_system -lboost_filesystem ../../lib/libbase.a -lcrypto -Wl,-Bdynamic -lz -ldl
CMakeFiles/os_linux_test.dir/Os_Linux_test.cpp.o: In function _GLOBAL__sub_I_Os_Linux_test.cpp': Os_Linux_test.cpp:(.text.startup+0x26): undefined reference to boost::unit_test::unit_test_log_t::instance()'
Os_Linux_test.cpp:(.text.startup+0x58): undefined reference to boost::unit_test::decorator::collector_t::instance()' Os_Linux_test.cpp:(.text.startup+0xea): undefined reference to boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::test_case*, boost::unit_test::decorator::collector_t&, unsigned long)'
Os_Linux_test.cpp:(.text.startup+0x117): undefined reference to boost::unit_test::decorator::collector_t::instance()' Os_Linux_test.cpp:(.text.startup+0x192): undefined reference to boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::test_case*, boost::unit_test::decorator::collector_t&, unsigned long)'
Os_Linux_test.cpp:(.text.startup+0x1bf): undefined reference to boost::unit_test::decorator::collector_t::instance()' Os_Linux_test.cpp:(.text.startup+0x23a): undefined reference to boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::test_case*, boost::unit_test::decorator::collector_t&, unsigned long)'
Os_Linux_test.cpp:(.text.startup+0x267): undefined reference to boost::unit_test::decorator::collector_t::instance()' Os_Linux_test.cpp:(.text.startup+0x2e2): undefined reference to boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::test_case*, boost::unit_test::decorator::collector_t&, unsigned long)'
If I compile against boost 1.65 using the -DBOOST_ROOT= flag, then it compiles fine.
Regards..

Can you please explain what you're doing exactly?
Ubuntu 18.04 comes with Boost 1.65 if you install it from packages it should work fine, even without setting BOOST_ROOT (see installation guide and .travis.yml file).
If you compile boost by yourself, of course you need to set BOOST_ROOT and i guess you'd better debug cmake to be sure it finds all the required components. Also you need to be sure you compile boost with the right options... eg: compile static libraries, the right architecture...
Boost is only used by the license generator, not by the library (see example.exe) so it makes very little sense to update it, except if you want to contribute...

Hi,
thanks for the quick reply. Sorry for not clarifying.
As you say the default boost version for Ubuntu 18.04 is 1.65. But I had to upgrade to 1.67 manually, that is what I have in my system now. All tools that I compile and use boost use this version.  When I try to compile your tool with this version of boost I get the reported errors .
So then I downloaded the tarball of boost version 1.65 to test your tool, I compiled this boost version and when I compile your tool using BOOST_ROOT it compiles successfully.
I just want to say that your tool seem not to be fully compatible with boost 1.69.
Regards,

I could reproduce the bug, it's because the FindBoost module shipped with cmake 3.10.2 can't handle Boost 1.69. Upgrade to cmake 3.16 will solve the issue.


Migrated to dep

Coverage increased (+0.07%) to 48.681% when pulling 5c76151ea2d2d3f77387695c33aabc69e59d21e8 on egopher:dep_migration into 88735359bee1433a347a24d9488c7e3e9e1aad25 on supergiant:master.

Merge conflict.


Something wrong with String content in multiPart()

From<EMAIL_ADDRESS>on November 18, 2012 08:11:45
What steps will reproduce the problem? 1. multiPart(String, String, String)
2. post() 3. What is the expected output? What do you see instead? On the server side, I get no content payload when a String is passed as the second parameter to multiPart, and I expect the contents of the String to be transmitted as that part's content. What version of the product are you using? On what operating system? rest assured 1.7.1, Windows 7 64-bit Sun Java 1.6.0_34 Please provide any additional information below. Here's my post statement:
given()
.auth().preemptive().basic(username, password)
.formParam("folderName", folderName)
.formParam("fileName", fileName)
.formParam("lastModified", lastModified)
.multiPart("attributes", attrFile, "application/json")
.multiPart("content", srcFile)
.expect()
.statusCode(200)
.when()
.post(uploadUri)
.getBody().asString();
When attrFile is a String and srcFile is a java.io.File, the srcFile transmits successfully, but attrFile does not; empty string is what I get on the server side for attrFile.  If I passed a java.io.File for attrFile, then the file contents transmitted successfully.
I played with the various interface options for .multiPart() trying to understand what was going wrong.  When I passed a ByteArrayInputStream for the attrFile, rest-assured threw an exception that read:
java.lang.IllegalStateException: Cannot serialize object because no JSON serializer found in classpath. Please put either Jackson or Gson in the classpath.
This tells me there is some code that looks at the mimeType and tries to marshal/unmarshal before sending the mulitpart?  Is this done as a content type check?  That's a pretty costly operation to toss a marshal/unmarshal in there as a validation; at least give the caller a chance to opt-in to something like that in the .expect() clause?
When I pass a byte[], String, or InputStream for that second parameter, I think it is reasonable to expect rest-assured to transmit my bytes verbatim with no transforms irregardless of what mime type I'm setting.
The String payload being empty was curious, and I spent many days trying to figure out where my code was broken until I realized it was RA silently failing on me.
Original issue: http://code.google.com/p/rest-assured/issues/detail?id=206

From<EMAIL_ADDRESS>on November 22, 2012 12:56:38
Seems like you've undergone a very painful investigation to reach your conclusion, sorry about that!
I suspect that the problem is that you use "application/json" and RA tries to do some clever things to support object serialization that obviously doesn't work properly.
Thanks for reporting, I'll look into this a bit better when I find some time.

From<EMAIL_ADDRESS>on December 01, 2012 21:09:53
Johan,
Sorry that my report came off as disgruntled, that last bit about spending days troubleshooting was useless information that I should have kept to myself.
I find RA super useful, I use it both for testing and for automation tools.  There are not enough HTTP client API's out there that do a good job exposing the underlying protocol/options -AND- keep the interface simple.  I'd love to see what you would do on the server side.
Cheers,
Dave Woldrich

From<EMAIL_ADDRESS>on December 08, 2012 00:56:16
I ran into a similar issue with simple java beans, had to add this to the maven pom dependencies
  \<dependency>
    \<groupId>org.codehaus.jackson</groupId>
    \<artifactId>jackson-core-asl</artifactId>
    \<version>${jackson.version}</version>
    \<scope>test</scope>
  \</dependency>

  \<dependency>
    \<groupId>org.codehaus.jackson</groupId>
    \<artifactId>jackson-mapper-asl</artifactId>
    \<version>${jackson.version}</version>
    \<scope>test</scope>
  \</dependency>

The documentation actually talks about the magic selection https://code.google.com/p/rest-assured/wiki/Usage#Serialization - see "Content based Serialization"

From<EMAIL_ADDRESS>on February 21, 2014 02:06:38
Issue 313 has been merged into this issue.

From<EMAIL_ADDRESS>on June 02, 2014 23:57:34
Is this still an issue?

From krao03 on August 27, 2014 10:44:30
Using Rest Assured 2.3.2.
I have an issue where fileName is not sent to Server (getting null to server side). I would appreciate if someone can update on this issue.
_response = given().
auth().preemptive().basic(Constants.USER_NAME, Constants.PASSWORD).
contentType("multipart/form-data").
multiPart("file", new File(fileName), "application/json").
formParam("info", infoValue).
//expect().
//body("status", 200).
when().
post(path);

From<EMAIL_ADDRESS>on September 23, 2014 02:14:37
@Krao: This should be fixed in version 2.3.3. Are you still seeing this issue?


Schema 'citrus-telnet-config'  is missing

Please add the schema for the telnet-module on the server so that it can be accessed as http://www.citrusframework.org/schema/telnet/config/citrus-telnet-config.xsd like the others.

I am afraid I can not do this until this module is officially maintained within the master


verifier not returning the right response code

my skill got removed from the alexa skill store cause the cert verification is done working properly
now i checked the verification manually and the only difference i found compared to the Amazon Docs is that the verifier returns 401 instead of 400...
still waiting for a response from Amazon - but did anyone else have this issue already?

Are you using express.js to host the skill. If so, the middleware for parsing it was recently updated and that could be the problem. @mreinstein Has there been any breaking changes to your alexa-verifier that should be addressed?

using both old and new(1.0.0) - old on live and new on dev - both return 401 when the verifier fails
can change that easily myself, but was just curious if anyone else is facing that kind of issues with amazon

Try changing it to 400, and submit it for certification again. If it succeeds, that means it's the middleware's fault. Otherwise there maybe be something else wrong with your setup.

feedback after recertification: changed it to 400 and got certified again

@sbeatz Just released a patch (v1.0.1) which now returns a 400 on any error.

This was https://github.com/alexa-js/alexa-verifier-middleware/commit/38ec0c37b15c75ff93cbb650a27c4dda48b62457 and https://github.com/alexa-js/alexa-verifier-middleware/commit/c4d82c6e5f9d7dcb2362a3c6a4c2492d5d7e588f.


/dev/shm default size should be configurable easily

What is the problem you're trying to solve
The default size of /dev/shm is hardcoded which is annoying when you have a lot of similar containers that need more than 64MB, -> you will have to modify volume specs for each container you run.
Hardcoded here:
https://github.com/containerd/containerd/blob/c6da899e2fd5ec5f0863d4be5ff683ee785d155c/oci/spec.go#L187
Describe the solution you'd like
It would be great for this value to be configurable like it is in dockerd. (default-shm-size) or eventually forwarding the dockerd value.
Additional context
Moreover, It may also be great to be able to modify the size (not the default one) for kubernetes usage, cf: https://github.com/kubernetes/kubernetes/issues/28272

I submitted a PR that is linked to this issue with an Opts to modify the shm size.  If k8s supports this option, it will be easy to configure in cri-containerd

@crosbymichael thanks a lot for your reactivity and your contribution.
Do you know if there is any workaround while waiting this fix to be released?

How is this function used? I see the definition of WithDevShmSize, but there is no invocation of it..

how to use it?


merge from arlac77/template-arlac77-github,arlac77/template-esm-only

package.json

chore(deps): add ^3.10.0 remove ^3.9.0 (devDependencies.ava)

:tada: This PR is included in version 1.1.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Cannot find module '@azure/ms-rest-nodeauth'.

When testing out the new sdk's I get the following error on the @azure/ms-rest-nodeauth module as the package on npm is missing the dist folder.
filename.ts:1:33 - error TS2307: Cannot find module '@azure/ms-rest-nodeauth'.

Temporary workaround I'm using:
cd <my-project-directory>
cd './node_modules/\@azure/ms-rest-nodeauth'
npm run build

then in my .ts file use this for the import:
import * as msRestNodeAuth from './node_modules/@azure/ms-rest-nodeauth/dist/msRestNodeAuth';

This is fixed in version 0.8.2. Apologies for the problem!

@kpajdzik thanks for the quick fix!


Programmatically submitting forms - it seems refs are not forwarded to the Form component

I'm trying to protect a signup endpoint with hCaptcha (the invisible type, which appears when the submit button is pressed).
I need to use a ref to programmatically submit the form when the response has been received from hCaptcha.
Here is the code (with some parts redacted for brevity):
function SignupForm({}: SignupFormProps) {
  const [captchaResponse, setCaptchaResponse] = useState<string | null>(null)
  const { formatMessage } = useI18n()
  const formRef = useRef<HTMLFormElement>()
  const captchaRef = useRef<HCaptcha>(null)

  useEffect(() => {
    alert('captchaResponse state changed') // 👈 it reaches here
    if (captchaResponse && formRef?.current) {
      alert('submitting the form...') // 👈 but not here
      formRef.current.submit()
    }
  }, [captchaResponse])

  return (
    <Form
      noValidate
      method="post"
      action="UsersController.create"
      ref={formRef} // 👈 the ref is assigned to the form
      onSubmit={(e) => {
        e.preventDefault()
        if (captchaRef?.current) {
          captchaRef.current.execute() // 👈 this shows the captcha when the submit button is pressed
        }
      }}
    >

         {/* Form goes here */}

          <HCaptcha
            sitekey={process.env.PUBLIC_HCAPTCHA_SITE_KEY!}
            size="invisible"
            onVerify={(token) => setCaptchaResponse(token)}
            ref={captchaRef}
          />
        </div>
      </div>
    </Form>
  )
}

export default hydratable('SignupForm', SignupForm)

And here is the error I get in the console:
Warning: Function components cannot be given refs. Attempts to access this ref will fail. Did you mean to use React.forwardRef()?
Check the render method of `SignupForm`.

The formRef is undefined:
console.log(formRef.current)
undefined

Is there a possible workaround for this?

Since the Form component accepts generics, wrapping it with forwardRef would break that. In your case, it's best to directly use the useForm hook, which is now documented: https://radonis.vercel.app/docs/hooks/use-form
Update to version 2.0.3 to get access to types which were previously not expored.


App telemetry

I believe @Oliver2213 created a Sentry project for iOS tracing/metrics, but we haven't yet added any probes to the iOS code to collect anything. When we do, we should respect the telemetry collection toggle that's already in the app settings.

I've started work on this in the sentry-telemetry branch. It looks like Sentry doesn't support deleting data from a specific user (which AppCenter supported), so I removed that dialogue when turning off telemetry. (Though not the associated translations)


Wrong robot respawn

Robot respawns with not zero velocity and rotation.

Thanks for your report @ArterKrilOk!
Would you happen to have a minimal reproducible test case where this may happen? It would greatly help us figuring out where the bug is as well as figuring out the fix.
Thanks!

Thanks for your report @ArterKrilOk!
Would you happen to have a minimal reproducible test case where this may happen? It would greatly help us figuring out where the bug is as well as figuring out the fix.
Thanks!

Hello @ArterKrilOk. Could you please test if this problem occurs with the newest software? I'll close this issue for now. If you run into similar problem, feel free to reopen this issue.


Separate Functions in Scripts.js to Utils.js

Split functions into utils.js, so scripts.js contains only side effects when loaded by head.html. This will allow us to test functions individually without having to run those side effects every time during unit testing.
Resolves: https://jira.corp.adobe.com/browse/MWPW-137679
Test URLs:

Before: https://stage--express--adobecom.hlx.page/express/
After: https://split-scriptsjs--express--adobecom.hlx.page/express/?lighthouse=on

I also checked following urls, but due to the volume of files changed, please also check a few more places. In general if the page loads fine without console errors, we should be good, since most changes in this PR are just file import/export changes.
https://split-scriptsjs--express--adobecom.hlx.page/express/create/video/instagram/reel?lighthouse=on
https://split-scriptsjs--express--adobecom.hlx.page/express/templates/flyer/christmas?lighthouse=on
https://split-scriptsjs--express--adobecom.hlx.page/express/pricing?lighthouse=on
https://split-scriptsjs--express--adobecom.hlx.page/express/learn/blog?lighthouse=on

Codecov Report

Merging #354 (5e5f7c4) into stage (d31393a) will decrease coverage by 0.88%.
Report is 2 commits behind head on stage.
The diff coverage is 35.37%.

@@            Coverage Diff             @@
##            stage     #354      +/-   ##
==========================================
- Coverage   31.26%   30.38%   -0.88%
==========================================
  Files         122      122
  Lines       25051    24987      -64
==========================================
- Hits         7831     7593     -238
- Misses      17220    17394     +174

Files
Coverage Δ

express/blocks/animation/animation.js
26.86% <100.00%> (ø)

express/blocks/app-banner/app-banner.js
14.15% <100.00%> (ø)

express/blocks/app-store-blade/app-store-blade.js
14.69% <100.00%> (ø)

.../blocks/app-store-highlight/app-store-highlight.js
15.60% <100.00%> (ø)

express/blocks/banner/banner.js
88.09% <100.00%> (ø)

express/blocks/blog-posts/blog-posts.js
10.69% <100.00%> (ø)

...ss/blocks/browse-by-category/browse-by-category.js
24.17% <100.00%> (ø)

...browse-by-collaboration/browse-by-collaboration.js
30.76% <100.00%> (ø)

...xpress/blocks/bubble-ui-button/bubble-ui-button.js
11.99% <100.00%> (ø)

express/blocks/cards/cards.js
50.00% <100.00%> (ø)

... and 90 more

... and 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


margin-left and margin-right don't work

Tried setting "mode" to "dock" and to "overlay" but it doesn't work. The margin-left and margin-right parameters do not affect the position in any way. I have several bars, and I'm trying to set them on the same level with each other.
WM: Hyprland
Waybar version: v0.9.17

I noticed that if I don't specify the "width" parameter, then margin-left and margin-right work, but I need to specify width.

i can't believe how hard it is to do anything like this

mode: overlay disables input; not suitable
Removing mode and putting layer: top  and exclusive: false allows the bar to draw over windows: works!
Only having modules-right: pushes bar to right side of screen, correct!
margin-right allows for an offset: hooray!

BUT THEN

cannot click through 'invisible' parts of the bar (if you have a long monitor this is a lot of screen space being denied)
Setting passthrough disables input for the entire bar
Setting width works! The clickable area is shrunk to hopefully only the visible part of the bar
BUT width disables margin-left and margin-right

Feels like everything is designed to stop people doing this. Perhaps allowing passthrough where there's no modules would be the easiest way, or making overlay allow input (same thing). It is weird that width disables use of margin, but ideally I'd want the width of the bar to be just of my modules and have there be a separate module which makes it stretch across the monitor.

i can't believe how hard it is to do anything like this
* `mode: overlay` disables input; not suitable

* Removing `mode` and putting `layer: top`  and `exclusive: false` allows the bar to draw over windows: works!

* Only having modules-right: pushes bar to right side of screen, correct!

* margin-right allows for an offset: hooray!

BUT THEN
* cannot click through 'invisible' parts of the bar (if you have a long monitor this is a lot of screen space being denied)

* Setting passthrough disables input for the entire bar

* Setting `width` works! The clickable area is shrunk to hopefully only the visible part of the bar

* BUT `width` disables margin-left and margin-right

Feels like everything is designed to stop people doing this. Perhaps allowing passthrough where there's no modules would be the easiest way, or making overlay allow input (same thing). It is weird that width disables use of margin, but ideally I'd want the width of the bar to be just of my modules and have there be a separate module which makes it stretch across the monitor.

You can try to make a transparent bar that covers the entire screen and place several elements with indents in it, which will be your actual bars.
Example:


Native modes and scaler setting

proposed commits implements native modes for 1080i50 and PAL that helps to get interlaced video.
other addons implement scaler settings which helps to produce 480p streaming modes for some reason (it actually has a limitation, for Progressive PAL scaled resolution width greater then 768 gives strange artifact, but it works)
thanks for that project - you did a great job!

The scaler settings look good, perhaps just remove the extra ()'s not needed in many of the ?: places.
Could you explain a little bit more why/what the PAL/1080i50 native mode changes do / are needed? I'd rather not add extra configuration blocks. But that's an option if there's no other easy why to accomplish needed thing. I'd assume that for 1080i50 it changes the 1088/1080 scaling knob? But what it does with the PAL mode?

Removing parenthesis will change operations priority and as result wrong result, for example:
printf("%d\n", (1 < 2)?4:5 + 6);
printf("%d\n", ((1 < 2)?4:5) + 6);

will output
4
10
Native mode required if you need to keep original quality of signal for recording or transmitting encoded signal to another point. "Progressive" preset seems enable deinterlacer engine that blend fields of interlaced picture. it is good for web, but for not for broadcaster or production because it /degrade/ picture quality
Another interesting thing is that registers 0x0015a2 and 0x0015a4 seems responsible for output canvas. Even native 1080i50 mode ffprobe says:
[...]
width=1920
height=1080
coded_width=1920
coded_height=1088
[...]
i found that .r1430_l related what height 1080 or 1088 will stream has.

The extra parenthesis is on (ep->foo)?a:b can be written as ep->foo?a:b. The -> operator takes precendence over ?: operator. The other parenthesis are required as you noted.
So basically the 'program_fpga' enables hardware de-interlacer? Could native mode flag then be just tested together with  'program_fpga', 'convert_to_1088' and perhaps 'interlaced'?

setting program_fpga to 1 on native mode does not produce any data at all...
parameter convert_to_1088 should not be used (IMHO) used at all, because for 1080i it returns 1088 rows in all cases and should not be scaled...

to be more precise, convert_to_1088 should be optional for "progressive" mode, if user notice that original picture padded by black lines he could crop and scale usefull lines to full picture

i can resubmit new pull request if there are no more comments/regards

No more comments. The scaler thing is good after the removal of the few extra parenthesis. The 'progressive' mode stuff should be special cased via a flag - not duplicating the register values. It might even work out nicely as a single commit that way. But conceptually it's good. Thanks!


LNK1181	cannot open input file 'SystemConfiguration.lib'

When I try to build my Xcode project, I get the following error:
LNK1181	cannot open input file 'SystemConfiguration.lib'
I simply ran vsimporter, so all the options are the default options. A class I use in my app relies on the System Configuration framework. There is a "SystemConfiguration" folder in my winobjc folder.

@segadude Were you using a pre-packaged SDK or did you clone the git repository? In the latter case, you need to first build build/build.sln

I don't remember if I was using the pre-packaged SDK or not, I think I tried both. I downloaded the February 18, 2016 release from the releases page, and while my app still doesn't build successfully, I no longer get this error.

@segadude Please file additional issues for any remaining build failures that you are seeing. :smile:


KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance

JIRA ticket: https://issues.apache.org/jira/browse/KAFKA-15046

While any blocking operation under holding the UnifiedLog.lock could lead to serious performance (even availability) issues, currently there are several paths that calls fsync(2) inside the lock

In the meantime the lock is held, all subsequent produces against the partition may block
This easily causes all request-handlers to be busy on bad disk performance
Even worse, when a disk experiences tens of seconds of glitch (it's not rare in spinning drives), it makes the broker to unable to process any requests with unfenced from the cluster (i.e. "zombie" like status)

This PR gets rid of 4 cases of essentially-unnecessary fsync(2) calls performed under the lock:

(1) ProducerStateManager.takeSnapshot at UnifiedLog.roll

I moved fsync(2) call to the scheduler thread as part of existing "flush-log" job (before incrementing recovery point)
Since it's still ensured that the snapshot is flushed before incrementing recovery point, this change shouldn't cause any problem

(2) ProducerStateManager.removeAndMarkSnapshotForDeletion as part of log segment deletion

This method calls Utils.atomicMoveWithFallback with needFlushParentDir = true internally, which calls fsync.
I changed it to call Utils.atomicMoveWithFallback with needFlushParentDir = false (which is consistent behavior with index files deletion. index files deletion also doesn't flush parent dir)
This change shouldn't cause problems neither.

(3) LeaderEpochFileCache.truncateFromStart when incrementing log-start-offset

This path is called from deleteRecords on request-handler threads.
Here, we don't need fsync(2) either actually.
On unclean shutdown, few leader epochs might be remained in the file but it will be handled by LogLoader on start-up so not a problem

(4) LeaderEpochFileCache.truncateFromEnd as part of log truncation

Likewise, we don't need fsync(2) here, since any epochs which are untruncated on unclean shutdown will be handled on log loading procedure

Please refer JIRA ticket for the further details and the performance experiment result

To check if these changes don't cause a problem, below consistency expectation table will be helpful:

No
File
Consistency expectation
Description
Note

1
ProducerStateSnapshot
Snapshot files on the disk before the recovery point should be consistent with the log segments
- On restart after unclean shutdown, Kafka will skip the snapshot recovery procedure before the recovery point.- If the snapshot content before recovery point is not consistent with the log, it will cause a problem like idempotency violation due to the missing producer state.
Hence, the inconsistency after the recovery point is acceptable because it will be recovered to the consistent state on the log loading procedure

2
ProducerStateSnapshot
Deleted snapshot files on the disk should be eventually consistent with log segments
- On log segment deletion by any reasons (e.g. retention, topic deletion), corresponding snapshot files will be deleted.- Even when the broker crashes by power failure before the files are deleted from the actual disk, they should be eventually deleted from the disk.

3
LeaderEpochCheckpoint
All leader epoch entry (i.e. epoch and the start offset) in the log segments have to exist also in leader-epoch checkpoint file on the disk
- If some epoch entries are missing from the checkpoint file, upon restart after power failure, Kafka may restore stale leader epoch cache.- It will return wrong entry when reading the leader epoch cache (e.g. in list offsets request handling)
On the other hand, surplus entries (prefixes or suffixes) in the checkpoint file on the disk are acceptable, because even on restart after power failure, it will be truncated anyways on log loading procedure.

We can confirm the changes are valid based on above table like this:

Change (1): fsync for ProducerStateManager.takeSnapshot is moved to the async scheduler thread

This preserves consistency expectation No.1, since we still increment recovery point only after fsync is performed

Change (2): ProducerStateManager.removeAndMarkSnapshotForDeletion no longer flushes parent dir

This preserves consistency expectation No.2, since snapshot files will be deleted eventually even after power failure.

ScenarioA: Snapshot files are renamed to -deleted for segment deletion by log retention, but broker crashes by power failure before the rename is persisted to the disk

In this case, some snapshot files's name would be reverted to -deleted suffix stripped, then ProducerStateManager would load these snapshot files unnecessarily.
However, since producer state will be truncated based on log-segment upon log loading procedure, it won't be a problem.

ScenarioB: Snapshot files are renamed to -deleted for topic deletion, but broker crashes by power failure before the rename is persisted to the disk

Also, in this case, snapshot file's name would be reverted to -deleted suffix stripped.
However, on topic-deletion, parent log-dir is already renamed to -delete and it's fsynced anyways, the revert of snapshot file wouldn't be a problem. Parent log dirs will be deleted anyways after resuming topic deletion procedure.

Change (3): LeaderEpochFileCache.truncateFromStart doesn't call fsync

This preserves consistency expectation No.3, since we still call fsync on LeaderEpochFileCache.assign.

Change (4): LeaderEpochFileCache. truncateFromEnd doesn't call fsync

Same explanation as Change (3)

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Could you take a look? @showuon

@ocadaruma , thanks for the improvement! Some high level questions:
0. Although you've added comments in the JIRA, it'd better you add your analysis and what/why you've changed in the PR description.

moving fsync call to the scheduler thread in takeSnapshot case makes sense to me, since we have every info in memory cache. And the log recovery can recover the snapshot when unclean shutdown.
For removeAndMarkSnapshotForDeletion, I didn't see this fix, could you explain it?
For LeaderEpochFileCache#truncateXXX, I agree that as long as the memory cache is up-to-date, it should be fine. We can always recover from logs when unclean shutdown.
nit: In the PR, could we make less code change for easier review? That is, we can create a overloaded method, and take one more parameter (boolean sync), and delegate the original method implementation to the new method (default to true). So, the only place we need to change, is the places we want to false the sync flush, which will make the PR much clear IMO.

Thank you.

@showuon
Thank you for your review.

Got it. I revised the PR description to include analysis.
The actual change is made against SnapshotFile.java#renameTo, which is called from removeAndMarkSnapshotForDeletion.

We can always recover from logs when unclean shutdown.

Yes. However, precisely, removing fsync on LeaderEpochFileCache's truncation doesn't cause extra recovery even on unclean shutdown IMO. The reason:

Since we still fsync on LeaderEpochFileCache#assign, we can still ensure all necessary leader epochs are in leader-epoch cache file
Even when truncation is not flushed (so "should-be-truncated" epochs may be left on the epoch file on unclean shutdown), log-loading procedure should truncate the epoch file as necessary (based on the log start/end offset). It's a fairly right-weight operation comparing to the recovering from the log.
Hmm, I intentionally didn't create a overloaded method because I was afraid a bit that default (fsync: true) method is used casually in the future code change even for the place which fsync isn't necessary.

Hey @ocadaruma
This is an important change, so I despite low engagement on this from me, I do believe that this change is critical. Let's think carefully about this.
I would like to discuss the changes individually.
Let's start with (1)
The side-effect of moving producer snapshot flush to async thread is that it while earlier it was guaranteed that producer snapshot is present and consistent when segment (and others) are flushed. If for some reason, the producer fsync failed, we would not have scheduled a flush for segment and friends. But now, since we are flushing snapshot async & quietly, it might be possible that we have segment and indexes on disk but we don't have a producer snapshot.
This is ok for server restart, because on restart, we will rebuild the snapshot by scanning last few segments.
This is ok even if the server doesn't restart because we will be using in-memory value of producer state and we might take another snapshot associated with next segment later.
To summarize, Kafka does not expect producer snapshot on disk to be strongly consistent with rest of files such as log segment and transaction index. @jolshan (as an expert on trx index and producer snapshot) do you agree with this statement?
If we agree on this, then (1) is a safe change IMO.

@divijvaidya Thank you for your review.

But now, since we are flushing snapshot async & quietly
During reading your comment, I found that "quietly"

@divijvaidya Thank you for your review.

But now, since we are flushing snapshot async & quietly

During reading your comment, I realized that "quietly" could be a problem so we might need to change producer-state flushing to throw IOException in case of failure. (still "async" isn't a problem though)
If we ignore producer-state-flush failure here, recovery-point might be incremented even with stale on-disk producer state snapshot. So, in case of restart after power failure, the broker might restore stale producer state without rebuilding (since recovery point is incremented) which could cause idempotency issues.
I'll update the PR.

If we ignore producer-state-flush failure here, recovery-point might be incremented even with stale on-disk producer state snapshot. So, in case of restart after power failure, the broker might restore stale producer state without rebuilding (since recovery point is incremented) which could cause idempotency issues.

Great point. May I suggest that we document the consistency expectations of producer snapshot with segment on the disk. From what you mentioned, it sounds like "Kafka expects producer snapshot to be strongly consistent with the segment data on disk before the recovery checkpoint but doesn't expect after the checkpoint. The inconsistency after the checkpoint is acceptable because....blah blah"
We verify that expectations with experts such as Justine and Jun. Based on that we can make a decision of quietly vs. async etc. The documentation will also help future contributions reason about code base. Initially, you can put the documentation in the description of this PR itself and later we can find a home for it in Kafka website docs.
We need to do the same exercise for other files that you are changing in this PR.

Thanks -- just catching up with the discussion. Just to clarify when we say:

This is ok for server restart, because on restart, we will rebuild the snapshot by scanning last few segments.

In the restart case, we may take a slight performance hit on startup since we may have to scan more segments.
And yeah, we should definitely not update the recovery point until the flush is completed successfully.

If we ignore producer-state-flush failure here, recovery-point might be incremented even with stale on-disk producer state snapshot. So, in case of restart after power failure, the broker might restore stale producer state without rebuilding (since recovery point is incremented) which could cause idempotency issues.

@divijvaidya Hi, thanks for your suggestion. I updated the PR description to include consistency expectations and the analysis of the validity of the changes.

closing once to rebuild

Hmm, I couldn't get all-green even after several runs due to flakiness which is irrelevant to this change..

@ocadaruma : Thanks for rerunning the tests. The latest run still has 21 test failures. Are they related to the PR?

@junrao Oh I misinterpreted as all green with only checking pipeline-view but I had to check tests view.
I checked. Seems none of them are related to this change, and failures are due to the flakiness because all failed tests still succeeded on at least some JDK build.

@ocadaruma : Thanks for looking into the failed tests. If those are unrelated to this PR, it would be useful to file jiras for flaky tests not already tracked. Also, could you resolve the conflict?

@junrao Thank you for the suggestion.
I resolved the conflict.
Also created tickets for flaky tests which don't have corresponding JIRA ticket now.

From previous run

https://issues.apache.org/jira/browse/KAFKA-15917
https://issues.apache.org/jira/browse/KAFKA-15918
https://issues.apache.org/jira/browse/KAFKA-15919
https://issues.apache.org/jira/browse/KAFKA-15920
https://issues.apache.org/jira/browse/KAFKA-15921

From last run

https://issues.apache.org/jira/browse/KAFKA-15924


Only tfstate version 4 is supported

Hi.
I tried to export state from my AWS account and terraformer generated a couple of states with version: 3
{
    "version": 3,
    "terraform_version": "0.12.10",
    "serial": 1,
    "lineage": "0747742d-d010-6fa3-7ad2-297215dee091",
    "modules": [
...

I tried to use terrafactor for merging states to the one, but got an error:
$ terrafactor generated/aws/s3/eu-central-1/ generated/aws/s3/eu-west-2/
Found duplicate files, merging generated/aws/s3/eu-west-2/outputs.tf
Found duplicate files, merging generated/aws/s3/eu-west-2/s3_bucket.tf
Only tfstate version 4 is supported

I want to understand why 3rd version of state isn't supported by terrafactor, because terraformer generates v3 for me.
Thanks.

Wait, I have been very busy lately. I will get to it.

Fixed in version 0.1.2.
Can you confirm?

Yes, it works, it converts to the 4th version.
Thank you very much.

@AmetPreply
Since you were interested in this, I just released 0.2.0 with new features. Care to try it out?
Let me know if you like it or not.

Sure, I'm already testing it. I'll let you know a feedback.
Thanks!

Caught this bug:
$ ls -R generated_checked/aws/acm/
eu-west-1/ us-east-1/

generated_checked/aws/acm//eu-west-1:
acm_certificate.tf* outputs.tf*         provider.tf*        terraform.tfstate*

generated_checked/aws/acm//us-east-1:
acm_certificate.tf* outputs.tf*         provider.tf*        terraform.tfstate*

Trying to merge them:
$ terrafactor generated_checked/aws/acm/ test_dir
...
Modularizing
(node:46438) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'provider' of undefined
    at getNameFromResource (/usr/local/lib/node_modules/terrafactor/src/modularizer.js:26:41)
    at /usr/local/lib/node_modules/terrafactor/src/modularizer.js:196:18
    at Array.reduce (<anonymous>)
    at generateNameToHclLut (/usr/local/lib/node_modules/terrafactor/src/modularizer.js:195:35)
    at modularize (/usr/local/lib/node_modules/terrafactor/src/modularizer.js:261:24)
    at async /usr/local/lib/node_modules/terrafactor/src/index.js:20:3
(node:46438) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:46438) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Modularizing looks like a cool feature, but maybe it would be cool to make it optional?

Something like terrafactor generated_checked/aws/acm/ test_dir --modularize=false ?

Yes, looks legit.

@AmetPreply Released 0.2.1. Can you check if the crash still happens?
Also added the --modularize=false

Looks good, thank you.


Comply with new EQ schema and maintain backwards compatability

Good point @necrophonic - updated to be more explicit.


ModuleCache initializing takes a minute

I'm having problem where every reload is taking about a minute. With the debug mode on I was able to trace through the code to dependencies_patch Line: 115
Apparently iterating over ObjectSpace and mapping _mod_name takes a really long time.
For example, if I open up rails console and run
ObjectSpace.each_object(Module){|mod| mod._mod_name }
take about 100 seconds. The count returned is 26000+.
I've monkey patched Line 231 of same file module_cache = nil so that it doesn't reload the cache on every file reload. This improved the reload time significantly to around 1s, but I'm not sure what side effects I've introduced (I'm a ruby nub).
Any insights or point me to the right direction? Thanks!

This improved the reload time significantly to around 1s, but I'm not sure what side effects I've introduced (I'm a ruby nub).

This should have broke rails-dev-boost, ie. it will now fail to detect classes/modules' inter-dependencies.

The count returned is 26000+.

This indicates either a really huge app (with 26000+ classes/modules/.rb files, just booting a Rails console should take about 10 minutes) or the app is somehow generating a lot of classes/modules on the fly. Either way this shouldn't take 100 seconds to iterate over.
For example here the an output for an app I have (running ruby 2.0):
irb(main):001:0> ObjectSpace.each_object(Module) {}
=> 4235
irb(main):002:0> t=Time.now; 10.times {ObjectSpace.each_object(Module) {|m| m._mod_name}}; Time.now-t
=> 0.804799

As can be seen the app has 4235 modules/classes, and iterating over them 10 times (this should be equivalent to iterating over 42350 modules/classes) takes 0.8s.

Any insights or point me to the right direction?

Check how fast "pure" ObjectSpace.each_object(Module) runs, ie:
irb(main):003:0> t=Time.now; 10.times {ObjectSpace.each_object(Module) {|m| m}}; Time.now-t
=> 0.122112

This way we can rule out that native ObjectSpace.each_object(Module) is to blame.
Then, check whether ._mod_name method is the correct one (it should be an alias of a native Module#name method):
irb(main):004:0> ObjectSpace.each_object(Module).map {|mod| mod.method(:_mod_name).unbind}.uniq
=> [#<UnboundMethod: Module#name>, #<UnboundMethod: Class(Module)#name>]

Yeah the Rails app I'm working on is pretty ginormous. Here's the benchmarks:
[1] pry(main)> ObjectSpace.each_object(Module) {}
=> 26605
[2] pry(main)> t=Time.now; 10.times {ObjectSpace.each_object(Module) {|m| m }}; Time.now-t
=> 0.570916

and then calling _mod_name
[1] pry(main)> t=Time.now; ObjectSpace.each_object(Module) {|m| m._mod_name }; Time.now-t
=> 34.679376

but take note that I this is just ONE call. Iterating over 10x would take 5min+.
Lastly this is the output of the unbind:
[2] pry(main)> ObjectSpace.each_object(Module).map {|mod| mod.method(:_mod_name).unbind}.uniq
=> [#<UnboundMethod: Module#_mod_name(name)>,
 #<UnboundMethod: Class(Module)#_mod_name(name)>,
 #<UnboundMethod: StateMachine::HelperModule(Module)#_mod_name(name)>]

Mine current hunch is that you are running into an issue with Module#name (note that _mod_name is a pure method alias for native Method#name) being 2 to 3 orders of magnitudes slower for anonymous modules/classes (ie: when mod.name # => nil) than for "named" modules/classes (when mod.name returns a string, for example RailsDevelopmentBoost.name # => "RailsDevelopmentBoost").
This is because: if a module/class name is unknown (which it is for "anonymous" modules/classes) the whole Ruby constant namespace is traversed by MRI trying to find the name of the mod, if a name is found, it is cached and all subsequent mod.name calls return that cached result. If traversing constant namespace yields no matches, nothing is cached and next time mod._mod_name is called MRI has to re-traverse everything yet again.
You can check how many of the modules/classes of your app are "anonymous" like this:
irb(main):002:0> ObjectSpace.each_object(Module).select {|m| RailsDevelopmentBoost::DependenciesPatch::Util.anonymous_const?(m)}.size
=> 172

Does this yield thousands modules/classes for you (I'm expecting it to be something like 20000+)?

I think your hunch is right, here:
[1] pry(main)> ObjectSpace.each_object(Module).select {|m| RailsDevelopmentBoost::DependenciesPatch::Util.anonymous_const?(m)}.size
=> 19224

So this is interesting. I did read about anonymous module/classes and their side effects. I'll have to read up on it.

Is keeping a persistent set/hash of anon class to skip the _mod_name look up a good solution?
require 'set'
anon = Set.new([])

2.times do
  t = Time.now
  ObjectSpace.each_object(Module) do |mod|

    next if anon.include? mod
    m = mod._mod_name

    if !m
      anon.add mod
    end

  end
  puts Time.now - t
  puts anon.length
end

And here's the benchmark after:
33.183677
19226

0.102118
19226

Don't know what to do about that abysmal anon module/class mod.name performance on MRI. For example with my app I have 4009 non-anonymous modules/classes and calling .name on all of them takes 0.002s, while invoking .name on remaining 220 anonymous modules/classes takes 0.06s. What's even worse is - the bigger your app gets, the slower .name call for anon modules becomes (as the namespace structure grows, the longer it takes to traverse).
Not sure if I'll ever be able to fix this for you (short of fixing MRI).
You could alleviate the problem by giving your anonymous modules names, not sure have feasible that is given your app's constraints:
def deanonymize_module_or_class(mod)
  Object.const_set(:XYZ, mod)
  mod.name # ensure the name is computed and cached by MRI
  Object.send(:remove_const, :XYZ) # unset the const, `mod` will now forever retain XYZ const name
end

Note that Ruby will not care that 19224 of your classes/modules now have the same mod.name.

Is keeping a persistent set/hash of anon class to skip the _mod_name look up a good solution?

I can't, they wouldn't get garbage collected by Ruby VM then...
I also can't set a an i-var on them like this:
if mod.instance_variable_get(:@anon_module)
  # skip
elsif Util.anonymous_const_name?(mod._mod_name)
  mod.instance_variable_set(:@anon_module, true)
else
  # work with mod
end

Because next time around a previously anonymous module might no longer be anonymous and ModuleCache needs to keep track of it.

You could alleviate the problem by giving your anonymous modules names, not sure have feasible that is given your app's constraints:

This is what I was planning. In config/application.rb I've added the deanonymizing code in an after_initialize block. Obviously this adds about an extra minute to the application boot time, but much better than the 40s per reload time.
I guess unless we monkey patch MRI, this is good enough for me. What I'm wondering is how does languages like Python handles anon name look ups.

What I'm wondering is how does languages like Python handles anon name look ups.

This is purely an MRI implementation problem, other Ruby VMs (JRuby and Rubinius) don't suffer from this issue, they always keep track of whether a module/class has a name or not, if it doesn't nil is returned right away (the whole namespace/constants-world is never traversed). Also because returning a nil from mod.name call does not involve a memory allocation (whereas on the other hand for "named" module/class mod.name call always returns a new String instance), calling mod.name on anonymous modules/classes is actually faster :).
This is because MRI optimizes for boot-up performance (when a module acquires a name, MRI chooses not to bother storing it on a module's object, thinking "if somebody asks for a module name, we'll get around to it and figure it out") whereas JRuby and Rubinius always keep module object's name up to date (an anonymous module has a null pointer, if it ever acquires a name a pointer is updated with the proper name).
PS: personally I believe MRI behavior is a legacy artifact/quirk and switching it to JRuby/Rubinius "mode" won't result in any boot-up slow down.

@pxue not sure if rails-dev-boost is ever going to attempt to fix your problem, but there is hope on the MRI front! https://bugs.ruby-lang.org/issues/11119

@thedarkone Nice! we should back contribute to the discussion. we have some nice real world benches here of the problem.


Enhancement: Remove ROOT as a dependency.

Seriously :)
This should be a long-term goal because it would:
... make installing root_pandas so much easier.
... mean that a lot of users will not need to install ROOT at all.
I am aware that this is ambitious.
Since we depend on ROOT via root_numpy, the simlest way to go would be to remove the ROOT dependency in root_numpy. Alternatively, one could define a stripped-down version of root_numpy which contains only the parts that we actually depend on and then try to remove its dependency.
It is true, I have no idea how much work that is....
https://root.cern.ch/root/InputOutput.html

Do I get it right? You mean to reimplement writing and reading root files in python / a python c extension?

I don't know, I mainly wanted to start a discussion about it.
I certainly don't expect this issue to remain open until it is fixed.
Maximilian Nöthe<EMAIL_ADDRESS>schrieb am Di., 10. Mai 2016
17:19:

Do I get it right? You mean to reimplement writing and reading root files
in python / a python c extension?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/ibab/root_pandas/issues/23#issuecomment-218191405

+1

I'm definitely 👍 on this, but after trying to create a minimal (only IO) version of ROOT myself, I have to say that it looks like an extremely difficult task.
There are extreme interdependencies between each of the components of ROOT, and it seems extremely difficult to extract any part of it.
For example, I couldn't see a way of retaining the TTree part of the source code without also keeping the custom LLVM version that ROOT includes in its source tree.
We could also open an issue on this in the root_numpy repository.

Hi all,
As pointed out in issue https://github.com/scikit-hep/root_pandas/issues/44, this idea is not a reality to a very large extent, see the uproot package in this org - https://github.com/scikit-hep/uproot.
I would suggest that we hence close this issue and continue the discussion in issue #44.
@ibab, @chrisburr, @alexpearce, do close if you agree :-).
Thanks.

Agreed 👍


Correct test condition for webp

this has been fixed in the 1.0 rewrite, thanks for putting together this PR


Variable in script

Hello,
really great job and very easy to install. But I have a question. I was not able to use a variable date which is defined in .TPL file and comes from PHP File. I can show this variable date in the .tpl file as {$musiclist.user_date_contract}
How can I integrate this variabel into the script and replace '2015/01/01' :
$('#getting-started').countdown('2015/01/01', function(event) {
$(this).html(event.strftime('%w weeks %d days %H:%M:%S'));
});
Thanks for your help

Just formats the date {$musiclist.user_date_contract} like YYYY/MM/DD, I think you should ask this  question at the PHP framework board, instead of here:
$('#getting-started').countdown(<?= $musiclist.user_date_contract ?> // ...


incomplete loki plugin installation

What did you do?
click loki plugin install on dashboard
What did you expect to see?
all loki component pod start
What did you see instead? Under which circumstances?
only loki-redis-master started   , but dashboard shows install success ( with red uninstall button )

Kubernetes version:
1.25.0

Logs:

api-server log :

E0923 16:35:48.078522       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:48.081616       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
E0923 16:35:48.081642       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:49.486283       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
E0923 16:35:49.486311       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:50.100742       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
E0923 16:35:50.100770       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:50.105637       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
E0923 16:35:50.105684       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:52.121590       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
E0923 16:35:52.121620       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:52.130424       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
E0923 16:35:52.130483       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:54.140842       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
E0923 16:35:54.140868       1 dispatcher.go:184] failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")
W0923 16:35:54.158919       1 dispatcher.go:180] Failed calling webhook, failing open mutate.label-injector.dev: failed calling webhook "mutate.label-injector.dev": failed to call webhook: Post "https://kubegems-local-controller-webhook.kubegems-local.svc:443/label-injector?timeout=10s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubegems-ca")

What do you see when exec:
kubectl get plugins.plugins.kubegems.io -n kubegems-logging loki -oyaml


Nick: Watch chordlagu.id

Nick requests the watch of the watch_keyword chordlagu\.id. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

chordlagu\.id has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Cody Gray in Charcoal HQ


(atomically?) commit multiple dots at once

As a user of a distributed database using dotmesh for backend storage, I'd like to be able to take consistent snapshots of all the dots used by my database nodes.
"Atomicity" is undefined in a distributed environment; we probably just need to be "as quick as possible" at doing all the snapshots at once, but maybe some atomicity could be arranged by synchronously talking to the database cluster itself, requesting that it prepare a stable global state, waiting for it to confirm it's done so, snapshotting, then telling the DB it no longer needs to maintain that stable global state.

people should try and use subdots before they try to do this.


Predition jpynb is broken

https://github.com/pysal/gwr/blob/master/gwr/notebooks/GWR_Georgia_Prediction.ipynb

Thanks for this issue. Will work on getting a revise notebook up
demonstrating the prediction functionality.
On Fri, Dec 8, 2017 at 1:15 AM, Leonid Solntsev<EMAIL_ADDRESS>wrote:

https://github.com/pysal/gwr/blob/master/gwr/notebooks/GWR_
Georgia_Prediction.ipynb
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pysal/gwr/issues/2, or mute the thread
https://github.com/notifications/unsubscribe-auth/AHvdzXL_aCMl8mSeJGhtWTUkSLWIOgq8ks5s-PAugaJpZM4Q6vm7
.


qtbase: don't leak absolut path to recipe specific sysroot

If the qtbase recipe-sysroot no longer exists (e.g. because of rm_work)
a user of Qt5GuiConfigExtras.cmake will experience the following error:
| ERROR: Task (.../meta-qt5/recipes-qt/qt5/qtwebkit_git.bb:do_configure) failed with exit code '1'
| CMake Error in Source/WebCore/CMakeLists.txt:
|   Imported target "Qt5::Gui_EGL" includes non-existent path
|
|     ".../build/tmp-glibc/work/armv7at2hf-neon-imx-angstrom-linux-gnueabi/qtbase/5.11.2+gitAUTOINC+49efea26a5-r0/recipe-sysroot/usr/include/libdrm"
Signed-off-by: Max Krummenacher<EMAIL_ADDRESS>

This addresses #108

Fixed the .patch files a bit, updated the tags in meta-qt5/qtbase and added to master-next.

5832bf2..3fa1993  master -> master


ServiceMap's update method needs to be renamed before Dart 2.0.

flutter_tools has a class ServiceObject and a class ServiceMap that extends ServiceObject and implements Map:
/// Base class of all objects received over the service protocol.
abstract class ServiceObject {
  /// Update [this] using [map] as a source. [map] can be a service reference.
  void update(Map<String, dynamic> map);
  /// Implemented by subclasses to populate their model.
  void _update(Map<String, dynamic> map, bool mapIsRef);
}

class ServiceMap extends ServiceObject implements Map<String, dynamic> {}

Unfortunately, Dart 2.0 is going to include a new update method for Map that updates a value with the result of a function:
dynamic update(String key, dynamic update(dynamic value), { dynamic ifAbsent() });

In order for ServiceMap to still implement Map in Dart 2.0, the existing update method (and presumably _update method) must be renamed. Something like updateWith and _updateWith would work.

Closing since the PR was closed. Please let me know if we actually still need this.


Proxy Authentication using logged on network credentials, works fine in Windows PowerShell

Steps to reproduce
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials

(Invoke-WebRequest -Uri 'https://google.com').StatusCode

Expected behavior
200

Actual behavior
Error message being thrown from network proxy

Authentication Required
You must be authenticated to access this URL.

Environment data

Name                           Value
----                           -----
PSVersion                      7.0.0-preview.3
PSEdition                      Core
GitCommitId                    7.0.0-preview.3
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

This works fine on the same machine using Windows PowerShell but doesn't work in PowerShell 6 or above.

I'm encountering what I think is the same problem on 6.2.2.
I have a system proxy configured, and I am trying to specify custom credentials for this proxy using the following snippet
$systemProxy = [System.Net.WebRequest]::GetSystemWebProxy()
$systemProxy.Credentials = New-Object System.Net.NetworkCredential($proxyUsername, $proxyPassword)
[System.Net.WebRequest]::DefaultWebProxy = $systemProxy

On Windows PowerShell, Invoke-WebRequest successfully uses this configuration. On PowerShell Core, I receive a 407 response from the proxy server.
Are there any workarounds or alternative ways of configuring an invocation of Invoke-WebRequest to use the system proxy with credentials? I can't set the -Proxy parameter on Invoke-WebRequest because this is a system proxy, so I don't know the literal proxy URL upfront.

Happy to report Powershell 7 preview 5 solves this issue for me.
I don't need anything in $PROFILE either it just works 😃

I also have the same issue. Powershell Core is not working via a forward proxy with an authentication layer ( any attempt use commandlets that require internet access returns a 407; so Invoke-WebRequest, Find-Module, Install-Module etc ). Windows Powershell works fine using the Cred Cache:
(New-Object System.Net.WebClient).Proxy.Credentials = `
  [System.Net.CredentialCache]::DefaultNetworkCredentials

Tried on Powershell Core 6.2.3 and 7.0.0 rc2 on Windows 10 Build 1909.

Have there been any updates to this? I can confirm this is still an issue on PS7 and LTSB.

This is still an issue with PowerShell 7.1
Have set both [System.Net.WebRequest]::DefaultWebProxy, $env:HTTP_PROXY, and $env:HTTPS_PROXY and I get HTTP status code 407 when using Invoke-RestMethod
PS C:\Git> Invoke-RestMethod https://dr.dk
Invoke-RestMethod:
Notification: Proxy Authorization Required

The same configuration works perfectly with Windows PowerShell 5.1
This is Windows 10 version 1909

Steps to reproduce
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials

(Invoke-WebRequest -Uri 'https://google.com').StatusCode

Expected behavior
200

Actual behavior
Error message being thrown from network proxy

Authentication Required
You must be authenticated to access this URL.

Environment data
Name                           Value
----                           -----
PSVersion                      7.0.0-preview.3
PSEdition                      Core
GitCommitId                    7.0.0-preview.3
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

This works fine on the same machine using Windows PowerShell but doesn't work in PowerShell 6 or above.

This seems to work in PowerShell 7.1

It comes from .Net.

It comes from .Net.

What does that mean sorry?

What does that mean sorry?

I have this issue too closing as it comes from .Net is not very helpful 😦

I have this issue too closing as it comes from .Net is not very helpful 😦

@gabrielmccoll
I tried with 7.1.1
PS C:\Users\mikes> [System.Net.WebRequest]::DefaultWebProxy.Credentials =[System.Net.CredentialCache]::DefaultNetworkCredentials
PS C:\Users\mikes>
PS C:\Users\mikes> (Invoke-WebRequest -Uri 'https://google.com').StatusCode
200

It now works for me 🎉

@gabrielmccoll
I tried with 7.1.1
PS C:\Users\mikes> [System.Net.WebRequest]::DefaultWebProxy.Credentials =[System.Net.CredentialCache]::DefaultNetworkCredentials
PS C:\Users\mikes>
PS C:\Users\mikes> (Invoke-WebRequest -Uri 'https://google.com').StatusCode
200

It now works for me 🎉

Dang. I have 7.1 and can't get .1.1 until they package it haha.
Thanks for coming back

Dang. I have 7.1 and can't get .1.1 until they package it haha.
Thanks for coming back


flow and webpack --watch

I am using flowtype-loader in webpack to run flow before compiling which works pretty good.
I specified a folder where all my flowtypes are stored. This folder is not part of the webpack-compilation-process, so the watcher will not re-compile when I perform changes on the flow-types.
I tried requiring the types in my main-file, which totally fails, since flow-syntax is not supported by webpack.
Is there a smart way to do it?

I found a way of solving this:
I added my own watcher (using chokidar) on the flowtypes-folder. On every change I create a JSON-file with all the types. This file I include in my webpack-main file. Webpack registers a change and recompiles. 👍


Added correct Swift tag for code snippets

What's in this pull request?
I updated the MD file to allow syntax coloring by adding Swift tag

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    All supported platforms     | @swift-ci Please smoke test and merge
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

A smoke test on macOS does the following:

Builds the compiler incrementally.
Builds the standard library only for macOS. Simulator standard libraries and
device standard libraries are not built.
lldb is not built.
The test and validation-test targets are run only for macOS. The optimized
version of these tests are not run.

A smoke test on Linux does the following:

Builds the compiler incrementally.
Builds the standard library incrementally.
lldb is built incrementally.
The swift test and validation-test targets are run. The optimized version of these
tests are not run.
lldb is tested.

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    All supported platforms     | @swift-ci Please test and merge
    OS X platform               | @swift-ci Please test OS X platform
    OS X platform               | @swift-ci Please benchmark
    Linux platform              | @swift-ci Please test Linux platform

Lint Testing
    Language     | Comment
    ------------ | -------------
    Python       | @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

This make code easier to read by adding syntax coloring

@Scellow LGTM modulo one comment, could you push an update?

I'll fix it up post-merge.


tf.keras.layers.Conv3DTranspose ignores dilation_rate parameter

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): N/A
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): N/A
TensorFlow version (use command below): 1.13
Python version: Any
Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A

Describe the current behavior
Conv3DTranspose layer ignores dilation_rate parameter. The shape of the output tensor stays the same regardless of the dilation_rate parameter.
Describe the expected behavior
Conv3DTranspose should behave like Conv2DTranspose and handle dilated transposed convolution. Note that Conv3D handles dilated convolutions all right.
Code to reproduce the issue
import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Conv3DTranspose

sess = tf.Session()
x = tf.placeholder(tf.float32, shape=(None,3,3,3,1))
layer = Conv3DTranspose(filters=1, kernel_size=3, dilation_rate=2)
result = layer(x)
sess.run(tf.global_variables_initializer())
output = sess.run(result, {x:np.ones((2,3,3,3,1))})

## Expected output shape = (2,7,7,7,1)
## Actual output shape = (2,5,5,5,1)

Other info / logs
The problematic code seems to be here:
https://github.com/tensorflow/tensorflow/blob/r1.13/tensorflow/python/keras/layers/convolutional.py#L890

Have tried on Colab with TF 1.13.1 and was able to reproduce the issue.

Perhaps this is a feature request rather than a bug. I checked that Conv2DTranspose uses custom implementation from Keras submodule (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/keras/backend.py#L4633) which is not implemented for 3D transposed convolutions at all...

Conv3DTranspose does not have a dilation rate for now

Would you like to make a PR out of it? I will make this as contribution welcome.

Not sure if this is directly relevant to this thread, but is keras.backend.conv3d_transpose still a relevant function worth keeping? It seems like keras.layers.Conv3DTranspose calls on nn_ops directly instead of routing to keras.backend. This contrasts with similar layers like keras.layers.Conv2DTranspose, which calls on keras.backend.conv2d_transpose internally (which then eventually calls on nn_ops).
At any rate, I'll be editing the docstrings to erase the dilation_rate argument as of now since it is an unsupported feature as of yet.

Fixed here: https://github.com/tensorflow/tensorflow/commit/46c79db67db33d7a14e124fc0b5cf23e0910d039

The commit linked above does not fix the problem:
While dilation_rate is now passed to the parent constructor, it is not actually used - neither for output shape calculation in the call to conv_utils.deconv_output_length nor for the actual computation in the call to nn.conv3d_transpose.
The test case introduced in the commit is also wrong in that regard - the output shape with a dilation rate of 2 for an input of (5, 7, 6) should be (9, 11, 10).
Besides, dilation_rate is still not even stored in the config dict.