Stuck at "INFO - finding looplift candidates"

When training with multiple gpus, the programme stops at "INFO - finding looplift candidates" after one epoch of training? This info sentence might come from numba, but I am not able to exactly locate it. Is there anyone who meets the same problem?

I also met this problem and I wonder have you fixed it yet?

Nope. Maybe you can try mmdetection3d.

I also have no clue (my school server only gets cuda 10.0 so I am still using torch 1.1.0 for training and there is no issue in that version). I feel the problem is related to apex according to a few other recent issues and you may want to replace Apex syncbn with torch's sync bn. Will check this more after I finish this semester ( in two weeks)

I think I fix it tonight with 'cuda 10.1, torch=1.4.0, numba=0.53.1'.
And it's worth mentioning that iou3d_nms module can't be directly set up from the .sh file since it changed the root path(?).
Thanks again for the prompt replies @wusize @tianweiy !!!

I also encounter this problem.
How to fix it ??

I also encounter this problem.
How to fix it ??

I think numba version may be incompatible with other dependencies. You can try my version plan above or carefully follow the author's instructions for each package.

But there is no  cuda 10.1, torch=1.4.0,   i.e.    torch=1.4.0+cu101

But there is no cuda 10.1, torch=1.4.0, i.e. torch=1.4.0+cu101

It's said that "PyTorch 1.4.0 shipped with CUDA 10.1 by default, so there is no separate package with the cu101 suffix, those are only for alternative versions. "
And I suggest conda to install it. You can find the command in pytorch.org

The problem is related to ddp in recent torch versions.
It should be fixed now https://github.com/tianweiy/CenterPoint/commit/e30f768a36427029b1fa055563583aafd9b58db2
And you should be able to use most recent torch versions. Let me know if there are any further problems

I am still stuck in the training process after merge the last two commit https://github.com/tianweiy/CenterPoint/commit/e30f768a36427029b1fa055563583aafd9b58db2
and https://github.com/tianweiy/CenterPoint/commit/a32fb02723011c84e500e16991b7ede43c8b5097.
My environment is torch 1.7.0+cu101,  V100-SXM2 16G.

oh, interesting, do you get timeout error ? Because I also noticed a slightly large delay between epochs, but it does proceed after some time.
Could you try  a simple example? Basically, I push a new cfg just now to simulate the training process.
Could you run
python -m torch.distributed.launch --nproc_per_node 2 tools/train.py configs/mvp/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z_scale_debug.py

it will only take a minute or so for one epoch. I want to know if you still get stuck with this cfg ?

oh, interesting, do you get timeout error ? Because I also noticed a slightly large delay between epochs, but it does proceed after some time.
Could you try a simple example? Basically, I push a new cfg just now to simulate the training process.
Could you run
python -m torch.distributed.launch --nproc_per_node 2 tools/train.py configs/mvp/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z_scale_debug.py

it will only take a minute or so for one epoch. I want to know if you still get stuck with this cfg ?

Hi there, i use waymo dataset and i don't know the differences in your debug setting.
But i test load_interval=1000 training on waymo dataset, the stuck disappear.
I don't know why.

Got it. Yeah, I only change the interval to subsample dataset.
Uhm, weird then. Maybe just use 1.4 if it is fine for your case. I will look into this further

When load_interval = 5, it stuck, load_interval = 1000, it woks.
It confuses me.

thanks, another thing you can try is either changing
https://github.com/tianweiy/CenterPoint/blob/3fd0b8745b77575cb9810035aafc76796613f942/det3d/torchie/apis/train.py#L268
to
        model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model)

or use spconv2.0.
I am now running a few experiments to see if there are any performance difference due to these two changes and will update soon.
Besides performance, I am able to train the full nuScenes dataset with 8 GPUs ddp (titan v) and latest torch (10.1 version + CUDA 11.3).

I have tried several couples of combinations but still stuck.
The only way work for me is to add env NCCL_BLOCKING_WAIT=1 to start the training process.
However, it slows the training process and i don't know why.

maybe related to multiprocessing, like adding a line like this to train.py before init ddp
    if mp.get_start_method(allow_none=True) is None:
        mp.set_start_method('spawn')

no clue if this works or not though because I just could not reproduce your error

I have the same problem ,have you solved it ? I guess is it related to num_workers?

I have the same problem！！！！

I have the same problem ,have you solved it ? I guess is it related to num_workers?

Do you have any ideas

hello, i'am confused about the effect of load_interval.... Can you explain what does this parameter mean?

load_interval probably is not the root cause. It defines how we subsample the dataset (10 then we use 1/10 of the dataset).
Unfortunately, I am not able to reproduce this issue ... Also see https://github.com/tianweiy/CenterPoint/issues/314

thanks, another thing you can try is

changing

https://github.com/tianweiy/CenterPoint/blob/3fd0b8745b77575cb9810035aafc76796613f942/det3d/torchie/apis/train.py#L268
to
        model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model)

use spconv2.x

I am now running a few experiments to see if there are any performance differences due to these two changes and will update soon. (Update: results with spconv 2.x + torch nn syncbn is similar to the original version.)
I am able to train the full nuScenes dataset with 8 GPUs ddp (titan v) and latest torch (1.10.1 version + CUDA 11.3).

i still stuck by this way.....TAT

load_interval probably is not the root cause. It defines how we subsample the dataset (10 then we use 1/10 of the dataset).
Unfortunately, I am not able to reproduce this issue ... Also see #314

i guess the problem come from 'finding looplift candidates', i always stuck in this step after train an epoch, do you know what that means?

it has nothing to do with finding looplift candidates.
It is the byproduct of starting a new epoch.
Unfortunately, I don't know what the root cause is (someone gets this issue and some one doesn;t...)

i tried to change the load_interval from 1 to 100 just now, and seems to no stuck.

i tried to change the load_interval from 1 to 100 just now, and seems to no stuck.

i have try several ways including change the load_interval as mentioned above this issue.
i suggest you to try
"I have tried several couples of combinations but still stuck.
The only way work for me is to add env NCCL_BLOCKING_WAIT=1 to start the training process.
However, it slows the training process and i don't know why.
"

i tried to change the load_interval from 1 to 100 just now, and seems to no stuck.

i have try several ways including change the load_interval as mentioned above this issue.
i suggest you to try
"I have tried several couples of combinations but still stuck.
The only way work for me is to add env NCCL_BLOCKING_WAIT=1 to start the training process.
However, it slows the training process and i don't know why. "

ok, i'll try, thankyou~

i tried to change the load_interval from 1 to 100 just now, and seems to no stuck.

i have try several ways including change the load_interval as mentioned above this issue.
i suggest you to try
"I have tried several couples of combinations but still stuck.
The only way work for me is to add env NCCL_BLOCKING_WAIT=1 to start the training process.
However, it slows the training process and i don't know why. "

ok, i'll try, thankyou~

NCCL_BLOCKING_WAIT=1

hello, i come back~ i have tried this way for train recently, it seems no stuck anymore, and the speed  seems normal

wow, that's amazing!

this one may also be relevant for other people https://github.com/pytorch/pytorch/issues/50820

I try to utilize NCCL_BLOCKING_WAIT=1, but it does not work for me


Fix subsampling

Fixes #26 and #40.
Uses Hypothesis to test random subsampling parameters. Cool new toy, apparently it's good.

Trigger benchmark (CProfile)

Unchanged (330fps).

trigger_width_ratio: 2

Before: 281-298 (306)
After: 274-313 (333?)

trigger_width_ratio: 4

Before: 268
After: 326-327

subsampling: 4

Before: 308 (calculated region incorrectly)
After: 396

Render benchmark
subsampling: 1

Before: 213-197??
After: 211-214

subsampling: 4

Before: 135
After: 252


Update Pages “about”

Automatically generated by Netlify CMS

👷 Deploy Preview for meuble-salon-canape-moderne-tunisie processing.
🔨 Explore the source changes: 601a2e1cdf90cc46977ce9a5fb34409f87ea9fb7
🔍 Inspect the deploy log: https://app.netlify.com/sites/meuble-salon-canape-moderne-tunisie/deploys/61ddd33f7249e6000779c9fc


Restructure and expand test coverage

Puts all our test objects in a app folder, a la Promotion's test base
Maps our test files up with class files (IE ext classes have matching ext spec's to match the change of ext being split into separate classes)
Adds a shared example for delegate method checks since controller and screens both should be tested.

:+1:


Invalid Value for environment variable should not be thrown

Rancher versions:
rancher/server: 2.0.0-alpha9
rancher/agent: v2.0-alpha4
Infrastructure Stack versions:
healthcheck: v0.3.3
ipsec: v0.12.3
network-services: n/a
scheduler: n/a
kubernetes (if applicable): not sure
Docker version: (docker version,docker info preferred)
Client:
 Version:      17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:42:18 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.09.0-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:40:56 2017
 OS/Arch:      linux/amd64
 Experimental: false

Operating system and kernel: (cat /etc/os-release, uname -r preferred)
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
HyperV
Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)
2.0.0 Quick Start Guide
Steps to Reproduce:
Create a container with a colon (":") in the name of the environment variable. This is necessary for some .Net AspNetCore applications when overriding appsettings.json values.
Results:
spec.containers[0].env[0].name: Invalid value: "Redis:Host": a valid C identifier must start with alphabetic character or '_', followed by a string of alphanumeric characters or '_' (e.g. 'my_name', or 'MY_NAME', or 'MyName', regex used for validation is '[A-Za-z_][A-Za-z0-9_]*')

This is basically validated in Kubernetes here: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apimachinery/pkg/util/validation/validation.go#L305
If I read through posts and docs on people running .Net, they use double underscore as a replacement of the colon. Can you try if this solves your usecase?
From: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration?tabs=basicconfiguration
If : cannot be used in environment variables in your system, replace : with __ (double underscore

As of the release of Tech Preview 2, the architecture behind 2.0 has significantly changed. Any issues with any release prior to v2.0.0-alpha12 is most likely invalid.
Please file new issues after using Tech Preview 2 (any release v2.0.0-alpha12+), but we recommend using the preview tag to get the latest. We are currently at v2.0.0-beta3. As always, thanks for using Rancher!


Feature Request:  investigate if xlsx files holding grading criteria metadata could be stored as JSON files

Feature Idea
Currently its not easy to see changes to the xlsx metadata files holding the lab grading. Ideally they could be stored as JSON files, so developers can rtreack changes, investigation required that JSON files would be easy to read and of course update.
Relevant Input
No response
Relevant Output
No response
Reproducible Example/Pseudo Code
No response

@millerg23 i feel like this is more of a discussion topic - which I am keen on seeing happen!! but feels like this issue is too vaugue.
Going to move over to Discussions.


Add support for --color=always option

This way colors can be used in pipes etc.

obsolete as I use a TUI now


The alluxio-job-work pod restart When the minio cannot be connected

Alluxio Version:
What version of Alluxio are you using?
2.9.0
Describe the bug
A clear and concise description of what the bug is.
The alluxio-job-work pod restart When the minio cannot be connected
To Reproduce
Steps to reproduce the behavior (as minimally and precisely as possible)
Expected behavior
A clear and concise description of what you expected to happen.
The alluxio-job-work pod can work fine When the minio cannot be connected
Urgency
Describe the impact and urgency of the bug.
urgent
Are you planning to fix it
Please indicate if you are already working on a PR.
i canot fix it
Additional context
Add any other context about the problem here.

Could you provide some more error log when the minio cannot be connected? Thanks


Analysis of Miro images with no link to Sierra

[x] Find the total number of images currently on /works which do not currently have corresponding records in Sierra (ie do not have ‘b’ numbers associated with them in the API)


More check for async await: return Promise in try block (without await)

Search Terms
check, async

Suggestion
Add some suggestions for return statement which returns a Promise inside try block in async context. (That case takes me so many online bugs)

Use Cases

async function foo(): Promise<string> {
    if (Math.random() < 0.5) {
        throw new Error("foo throw")
    }
    return "foo"
}

async function bar() {
    try {
        return foo() // Hey! here! await!
        // return await foo() // Expected
    } catch (e) {
        console.log(e.message) // never been call
        return "bar"
    }
}

Examples
return foo()
// ^^^^^^^ is that missing await?

return (foo()) // ok, I believe that you know what are you doing

Checklist
My suggestion meets these guidelines:

[ ] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[ ] This wouldn't change the runtime behavior of existing JavaScript code
[ ] This could be implemented without emitting different JS based on the types of the expressions
[ ] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[ ] This feature would agree with the rest of TypeScript's Design Goals.

I don't agree:

maybe foo throws exceptions synchronously and you want to handle only that case without catching rejected promises
wrapping the call expression in parentheses shouldn't suppress any error as some formatters automatically add them for multiline return

This can be checked by a lint rule instead, e.g. https://github.com/fimbullinter/wotan/blob/master/packages/mimir/docs/try-catch-return-await.md

@ajafff an async function can't throw synchronously, doesn't it?

@payload it surely can. a function can throw an exception before returning a promise you can await on.
I do like the idea, but I think it's better suited for a linter as a default rule than the compiler itself. Because with linter you can mark it as intentional, while with a compiler you'll need a non-readable syntax marker.
I'm sure this is usually a mistake, but for the few cases where it's intentional it's great to add a comment disabling the linter, the comment itself indicates why it's there and why it's not a mistake.

@alitaheri consider that this
function foo() { if (x) throw new Error(); return Promise.resolve(5); }
is very different from this
async function bar() { if (x) throw new Error(); return 5; }
foo is a function which either throws synchronously or returns immediately a promise eventually resolving 5.
bar is an asynchronous function returning always immediately a promise which eventually resolves 5 or rejects with the error.
An async function can't throw synchronously, it will always return a promise.

@payload I didn't know that. thanks 👍
Then I guess this is valid for a function marked async 👍


odd scrolling on mobile when attempting to drag

I don't know if #126 was trying to fix this or caused it, but scrolling on mobile is weird.  If I click an item to drag, if I drag it up and down, it takes the screen and scrolls it too.
Happens here:
http://luckylooke.github.io/dragular/#/docs/exampleBasicWithModel

It leaves these two warnings in chrome that could be the cause:
dragular.js:575 [Intervention] Unable to preventDefault inside passive event listener due to target being treated as passive. See https://www.chromestatus.com/features/5093566007214080
startBecauseMouseMoved @ dragular.js:575
dispatch @ jquery.js:4737
elemData.handle @ jquery.js:4549

dragular.js:1190 [Intervention] Unable to preventDefault inside passive event listener due to target being treated as passive. See https://www.chromestatus.com/features/5093566007214080

Don't think it was the PR at all, in the last few versions of chrome, they changed the way you need to prevent default for scrolling.

https://github.com/alvarotrigo/fullPage.js/issues/2362
https://github.com/dimsemenov/PhotoSwipe/issues/1217
A bit more insight to this there.  From Chrome 56 on they made a breaking change to improve scrolling...
Apparently can be fixed with touch-action css

Yes it was working fine before... so it seems that the change you mentioned affected dragular, I will look at it asap (in few days I hope).. thanks for your report and suggestions! ;)

fixed, will be in next release asap ;)

Released: https://github.com/luckylooke/dragular/releases/tag/v4.3.4

Thank you so much, awesome that you got to all the issues I reported so quickly.  Great package, thanks for all the hard work :D

Yeah, I was sad that the lib looked to be so buggy, so it push me to solve everything asap :D But there is still one bug on IE which is still making me sad, bud I have no extra energy for IE now..


assertSnapshot causes memory leak

Describe the bug
When using assertSnapshot the view is not released at the end of the test
To Reproduce
You can reproduce this with the attached project: SnapshotIssue.zip
final class SnapshotIssueTests: XCTestCase {

    func testExample() {
        let controller = UIStoryboard(name: "Main", bundle: Bundle(for: ViewController.self)).instantiateInitialViewController()!

        trackForMemoryLeaks(controller)

        assertSnapshot(matching: controller, as: .image)
    }
}

private extension SnapshotIssueTests {

    func trackForMemoryLeaks(_ instance: AnyObject, file: StaticString = #filePath, line: UInt = #line) {
        addTeardownBlock { [weak instance] in
            XCTAssertNil(instance, "Instance should have been deallocated. Potential memory leak!", file: file, line: line)
        }
    }
}

Expected behavior
When snapshot test is done, the view should be released
Screenshots

Environment

swift-snapshot-testing version: 1.9.0
Xcode: 13.3.1
Swift: 5
OS: iOS 15.4

Does this issue persist when using SwiftUI?

I have not tried with a SwiftUI view. But I can say that this doesn't happen with plain UIViews. It seems to affect only UIViewControllers.

I have had a similar issue with 1.10.0 where when the view is passed, the assert snapshot returns and when there is a view controller with a hard dependency on a UINavigationController and the UI navigation controls making the controller a child of the UINavigationController resolves or at least permits the snapshot test to return.

Do you have any way to workaround this issue? This seems to happen in my case with a SwiftUI view and TCA. The store is somehow still around after the test is finished and causes test failures coming from Unimplemented dependencies (which seem to have escaped the task local scope by then).
My chain of events ist basically
start of snapshot test
snapshot test passed
start of another test (testing the reducer's functionality using TestStores)
action received in store from snapshot test
Unimplemented: @Dependency(\.date)

@stephencelis I noticed you also use snapshot tests for the GameOverView in isowords (which also has long running tasks added to a task group from the task action). But you don't run in to issues because you don't create a ViewState in the view store's observation closure which accesses a Dependency inside its init. (This is how it used to look pre observation).


refactor(css): provide component styles as a standard css lib

This refactors the ARC components to provide CSS component's style definitions for use when SSR is a strict project requirement. This enables project developers to use the ARC styling definitions to create server-sider renderable components in any given server-side framework, such as Angular Universial, NextJS, and NuxtJS.
Web Components make use of the shadow dom and the ARC component styling definitions are written originally with this in mind. In order to provide correct styling definitions for a CSS-only component SCSS has been used to split to share styles across two interval files.
.
├── src
│   └── components
│       └── button
|           ├── _arc-button.shared.scss   defines styles
|           ├── arc-button.scss           imports styles and defines selectors for css only component
|           └── arc-button.styles.scss    imports styles and defines selectors for web component

The build process is now:

Compile SCSS to CSS
Read all .styles.css files (the CSS files that contain the definitions for use within a Web Component) and write the contents as a javascript file that the Web Component can import.
Compile Typescript to Javascript
Create a Bundle with a rollup in the dist directory
Use rollup to copy all disired files to the dist directory (notably: Javascript and type definition files compiled in step three and the CSS files compiled in step one)

This results in each component's CSS style definitions being included in the built library. As the styles are no longer defined in JS this also results in a Typescript error as the JS file containing the Web Components styles imported by each component does not exist until build time. TS ignore comments have been used to address this.

Closing this due to the server side rendering compatible web components


Fix many warnings in libnczarr

This fixes about 100 out of 130 warnings from libnczarr. The remaining 30-ish warnings are basically either:

code doing subtraction or iterating backwards, where changing e.g. the loop index to size_t would give UB
public function/type declarations, for example filter IDs

One thing that gives rise to a bunch of warnings is maxstrlen. It's declared as an int in NCZ_VAR_INFO_T, but used as size_t in several places. I couldn't work out if this is publicly accessible or not, or if setting it to a negative value had some semantic meaning. I think it would be safe to change to size_t consistently.

Changes to the NCZarr code may be a waste of time. I am in the process of adding
Zarr V3 support and it required a major refactor of the code.
Can someone refrest my memory about the gcc flag I need to use to detect all of these int <-> size_t problems.

The CMake debug build enables: -Wall -Wconversion. -Wconversion should also enable -Wsign-conversion, but not -Wsign-compare (probably also worth enabling at some point)

@DennisHeimbigner GitHub isn't showing what you're commenting on, but if it's in reference to f8da95a, then it's due to implicit conversions:

loaded_plugins_max is not the total number of loaded plugins, but
the max ID, and defaults to -1. This clashes with the change of the
index to size_t

Please do not do this. I am in the process of making a major refactor of libnczarr.
All of these changes will just make it harder for me to resolve the merge later.


Add kapacitor example

Adds a simple example with explanation of why it would be useful. We should do a blog post on this idea in the future but this is a good start.

@joelegasse I branched off your branch. Sorry about that, this can wait till the other branch is merged.


[Bug]: pipeline not marked as modified after onlu change in field mapping in Table Output

Apache Hop version?
2.8
Java version?
jdk-11.0.22
Operating system
Windows
What happened?
To reproduce the problem:

open a pipeline,
change some mapping in Table Output hop and press ok
you will see that the pipe is not marked "to save" ( floppy still gray)
The pipeline is ok, but is not autosaved / it's impossible to save, so you have to make some other change to be able to save the modified version.

Issue Priority
Priority: 2
Issue Component
Component: Hop Gui

.take-issue

cannot reproduce, I tried changing the fields, the connection, schema/table name they all triggered that the pipeline was changed


add your GH sponsor link

@mfkl can you enable GH sponsors and add your link here https://github.com/CopyText/TextCopy/blob/master/.github/FUNDING.yml
Also can you create a tidelift account and i will add you

Ok, I'll do it soon.

Waiting on Stripe (and then GitHub) to approve my Sponsors account.
For tidelift the email for my account is<EMAIL_ADDRESS>

GitHub sponsors isn't supported in my location, according to GitHub support staff.

in that case i will remove mine for now


enhanced the node checking expression

using if(variable) {} is a shortcut for checking if the value exist or not so values like undefined, null, 0, "", false and NaN will not pass

you were checking if node is not equal to true or false and those are Boolean values so instead it will easier to check the value type if it's boolean or not

your version not logically equivalent. for example, failing for 0.

Existing (32 chars, worst case 3 comparisons, passes)
['', 0, 'x', true, false].map(x => !!(node != null && node !== true && node !== false))
=> [true, true, true, false, false]

Proposed (25 chars, worst case 2 comparisons, fails)
['', 0, 'x', true, false].map(node => !!(node && typeof node !== "boolean"))
=> [false, false, true, false, false]

I got a bit carried away there! As @leeoniya points out your proposal is not logically equivalent to what is in master at the moment. Adding != null to the first check of node seems to make the check equivalent (unless I am missing an edge case).
Alternative (31 chars, worse case 2 comparisons, passes)
['', 0, 'x', true, false].map(node => !!(node != null && typeof node !== "boolean"))
=> [true, true, true, false, false]

Which results in one less byte and one less check in most cases. I am no perf guru but I see this boiling down to which is more performant:
typeof node !== "boolean" or node !== true && node !== false
🏎:fire::fire::fire::fire:

Hi @KhaledElAnsari!
This line is what makes conditional rendering possible:
{true &&
Hello
}

or
{condition ?
Okay
 : null}

and also why you must convert booleans to a string if you wish to render the name:

{true + ""}

We can't remove the null check, so I guess this boils down to:
} else if (node != null && node !== true && node !== false) {

or
} else if (node != null && typeof node !== "boolean") {

So, which one is better? 🤔

@lukejacksonn Which one is more performant you saying?

Thank you guys for your points I'll edit my proposal now @lukejacksonn @leeoniya @jbucaran

Are typeof performance issues solved ?

@jbucaran yeh I have no clue which is more performant but I am going to assume that one less check is always going to win.
:boom:

@lukejacksonn That is assuming typeof is only one check! 😆

@jbucaran @lukejacksonn The current & proposed logic are roughly equivalent perf in Chrome on my computer (adapted examples from earlier in this thread).  I'm sure Firefox/Edge would have different results.
https://jsperf.com/ha-pr316/1

I'm sure Firefox/Edge would have different results. https://jsperf.com/ha-pr316/1

original is 10-15% faster in Chrome and Firefox and 22% slower in Edge. at least on Win10x64.

@jbucaran done

as a tangent, this PR makes me wonder what Codecov means by "99%". the original logic of this PR surely would have failed at least the two cases i pointed out. i doubt that this specific line happens to be that missing 1%. so, what's the point?

original is 10-15% faster in Chrome and Firefox and 22% slower in Edge. at least on Win10x64.

JS perf is so weird.  I'm seeing that 10% in my JSPerf now too (Chrome - Win7x64).

if you're on a laptop, thermal throttling often affects the later runs more than the earlier. all benchmarks should really pre-heat the CPU. the effect is more evident in some tests more than others. i dunno if this applies here, but i've seen testcase inversion change the outcome :/

Tested on Chrome 59.0.3071 / Windows 7 0.0.0. The PR is always 5-10% slower than original.
Typeof suffer since while of performance issues.

And there would be no difference in terms of bytes at the compressed level, so I think we are going to pass on this one.
Thanks anyway @KhaledElAnsari! 👋😄

@leeoniya thanks for settling that one with hard evidence!
What is it they say "Never assume.." 😅

@leeoniya thanks for settling that one with hard evidence!

not sure if anecdotal [past] experience and a single jsperf run qualifies as "hard evidence", but it's a slightly better than pure speculation. if anyone deserves the credit here, it's @SkaterDad

Ahh yeh sorry 😝 I thought you posted that jsperf link.. @SkaterDad nailed it 💯 @leeoniya your anecdotal experience is always insightful, so thanks regardless!


Dataset release schedule

Hi, thanks for your awesome work!
I recently do some researches about human body segmentation. I wonder do you have a schedule to provide the dataset which I am really interested in?

We are figuring out the details and hope to release them soon. Currently, we can't provide a fixed time schedule.

Hello,Congratulations for your work!
I wonder when the dataset will be released? Will it be released this year?

It is hard to say. We are working on it.

PhotoMatte85 dataset is now available for download.

PhotoMatte85 dataset is now available for download.

emmm...why the photo85 only have 84? Did you miss one photo?

emmm...why the photo85 only have 84? Did you miss one photo?

@AmberCheng Fixed. We reuploaded the zip with 85 samples now.

@AmberCheng Fixed. We reuploaded the zip with 85 samples now.

Just to make sure, the README says: This work is licensed under the MIT License. This includes the PhotoMatte85 dataset as well?
I noticed that for two images, a few pixels at the top-left are matted incorrectly.

I guess the images were obtained via greenscreen matting and the greenscreen was hanging too low. It probably does not matter much, but here is some code to fix those images.
from PIL import Image
import numpy as np

for filename in ["0051115Q_000001_0049.png", "0051115Q_000001_0085.png"]:
    image = np.array(Image.open(filename))
    print(image.dtype)

    # zero out top 100 rows of image to fix bad alpha matte
    print(image.shape)
    image[:100] = 0

    Image.fromarray(image).save(filename)

PhotoMatte85 dataset is now available for download.

When the VideoMatte240K dataset will be released?Thank you.

We finally finalized the licensing of the VideoMatte240K dataset. Currently planning to release it on April 12.

Actually there are still some delay. Likely to be in this week or next.

Dataset is released. Checkout the README of the repo.


Issue 523

Added test case
Added tests in test_cli.py and test_client.py
Added testCors in test_views.py
Added testCors in test_views.py
Added testGetDataset in test_simulated_stack.py
Adding 404 check to testGetDataset
Removing comment

How did @dcolligan become the author of this commit? This is weird!
@dcolligan (as the resident git expert), can you take a look at this and see what needs to be done to make it look like it should? Posting the exact commands would be helpful.
I accept git is confusing initially @pcingola, but it really is so much more powerful once you get the hang of it. We need to update the docs with a simpler method of rebasing and squashing I think.

I've used many versioning systems in my life.
No, it doesn't need to be this complicated.
It's just broken.

Looks like the newest changes from develop got put into this commit somehow.  Not sure how this happened (in general, it's hard to tell exactly what went wrong with screwed up branch X).
These commands will fix this:
# make sure your local develop branch is up to date
$ git fetch --all
$ git checkout develop
$ git merge --ff-only upstream/develop
# create a new branch; name it whatever you'd like
$ git checkout -b new_branch
# cherry pick your commit onto this new branch; since the changes you don't want to incorporate are already in the develop branch, this should only put the diff (that is, your desired changes) in the commit
$ git cherry-pick b27fea38f84f8e5b91c7fbf9585aadb31e47ae94
# give yourself authorship
$ git commit --amend --reset-author
# force push your new local branch to the same remote branch that is now screwed up, effectively overwriting it with the correct changes
$ git push -f origin new_branch:get_dataset_by_id_squashed_squashed

I recommend reading up a bit on git since it is what we use (not to mention probably the most-used RCS for new projects these days): https://git-scm.com/book/en/v2

Hi Pablo,
git a brilliant idea (although not unique) implemented in a
bizarre and confusing way.  It is amazing powerful.  The
documentation is a huge part of the problem.  The basic
documentation is not that good, especially on how to actually
apply it and why features are there.  The community advice you
get is conflicting, such as never rebase vs almost always
rebase.
It's the most complicated development tool I have ever seen.
The best thing to do is ask.  Danny is especially good at it.
The SourceTree application is really useful as well.
Mark
Pablo Cingolani<EMAIL_ADDRESS>writes:

I've used many versioning systems in my life.
No, it doesn't need to be this complicated.
It's just broken.
—
Reply to this email directly or view it on GitHub.*

This PR is one commit as required. Can we just merge it?

No, we're not going to merge in changes to a mainline branch that aren't the right ones.  At this point you can either:

follow the steps I outlined above
rebase your branch on top of develop, which will likely do the same thing, and resubmit it
have me submit a new PR with your changes and merge that, if you want me to take your changes from here

Sure, go ahead with option 3.

I'm sorry this has been so frustrating @pcingola. 99% of the time, all squashing entails is
$ git checkout topic-branch
$ git fetch upstream
$ git rebase -i upstream/develop
$ git push -f origin topic-branch

Squashing tends to go horribly wrong the first few times you do it, but completely smoothly after that when you get used to keeping your topic branches current and avoiding conflicts. I don't think we're being unreasonable in wanting to keep our history clean. This is really, really useful when it comes to maintaining many branches and releases, and it's fairly standard practise in git projects involving more than one developer.
Anyway, thanks for adding this functionality and I hope you're not put off from contributing more by the git requirements. We do need to improve the documentation for this, as it is the single biggest obstacle for new developers wanting to contribute.


Corrected spelling error

When copy-pasting from examples, this spelling error causes a break.

Coverage remained the same at 78.902% when pulling a2a68091dd7e40df4c1d8a5db9a14dbaaf3b54ab on beyerz:master into a52a7c9c4c7bef59cedae5e140e6c73c665eec7d on lukasoppermann:master.

Very good catch, thank you. 👍


how you set your environment variable

thanks for your suggestion to install tensorpack, but I meet a question. After I install tensorpack, there is also a warning :Env var $TENSORPACK_DATASET not set. how do you set your environment vairable TENSORPACK_DATASET and IMAGENET?

met the same problem. Have you solved it?

I've removed the environment variables so now these things are all specified in command line arguments, have a look at the changes in the README.


Fix extension for 4.0

The .mono folder was moved to .godot/mono (https://github.com/godotengine/godot/pull/38704)
The --remote-debug argument now requires the protocol (https://github.com/godotengine/godot/pull/37099)
The -q argument has been repurposed, now --quit should be used (https://github.com/godotengine/godot/pull/55696)

Although these changes should fix the extension for 4.0 while keeping support for 3.x, they feel hacky and it might be better to simply provide a configuration setting for the user that defaults to the current stable version of Godot instead of trying to detect the version.
Closes #38

In a week, we'll be merging the dotnet6 branch to master and this will no longer work. I can merge this if still desired, though.

I don't see how these specific changes are related to that merge, unless you mean that Godot 4.0 will be unusable due to the many things that were commented out and that need to be rewritten, in which case I still think these changes would eventually be needed, no?
Either way, I'm in no hurry to merge this as 4.0 is still in alpha, just looked into it because somebody asked and thought I'd leave this here for when we need it.

I don't see how these specific changes are related to that merge

The extension uses the Mono debugger, which won't be usable in 4.x because the editor (including the player) uses CoreCLR (as it uses the runtime from the .NET Sdk). This means the VS debugger won't be able to connect, which I think results in a launch error.

Hey! ✌
What's the status of this PR?
Godot 4.0 seems to be getting closer to release and is in bugfixing mode now. So there shouldn't be any big changes on the C# side anymore.
If we don't want to deal with backwards compatibility for Godot 3.x we could still release a preview package of this with Godot 4.0 support to get feedback and bug reports already now!

Superseded by https://github.com/godotengine/godot-csharp-vscode/pull/56


Write to HDFS hive partition.

In HdfsState.java, would you please add a way to config to write dynamic path.
I need this to write to hdfs hive partition by day.
It is like
drwxr-xr-x   - hdfs tbuser          0 2013-11-18 23:20 /staging/anvil/processed/parsed/access/dc=2013-11-18
drwxr-xr-x   - hdfs tbuser          0 2013-11-19 23:15 /staging/anvil/processed/parsed/access/dc=2013-11-19
drwxr-xr-x   - hdfs tbuser          0 2013-11-20 11:31 /staging/anvil/processed/parsed/access/dc=2013-11-20
Thanks
HdfsState.Options options = new HdfsState.Options()
.withFileNameFormat(fileNameFormat)
.withRecordFormat(recordFormat)
.withRotationPolicy(rotationPolicy)
.withFsUrl("hdfs://localhost:54310")
.withPath(path);

I have similar use case where data needs to be saved in a directory created by using values from the message being processed. My use case requires partitioning data by timestamp of the event. Events in the pipeline can get delayed by several hours or days. Pipeline gets events that occurred few seconds ago to few days. I need to save these events in different directories based on their timestamp. I am looking for some way of doing it in Trident implementation.

@rs-01 : I have almost the same problem, did you solve it?


Add helpful comments for the bootstrapped graphqlgen.yml config files

The bootstrapped graphqlgen.yml files should contain helpful code comments such as what resolver-scaffolding is about etc. The code comments should be as concise and minimal as possible while addressing the most common questions.

Fixed in #147


safeguard q parameter in template (#1037)

Overview
Fixes bug in #1037
Related Issue / Discussion
#1036
Additional Information
None
Contributions and Licensing
(as per https://github.com/geopython/pycsw/blob/master/CONTRIBUTING.rst#contributions-and-licensing)

[x] I'd like to contribute [feature X|bugfix Y|docs|something else] to pycsw. I confirm that my contributions to pycsw will be compatible with the pycsw license guidelines at the time of contribution.
[x] I have already previously agreed to the pycsw Contributions and Licensing Guidelines

FYI CI failure is unrelated.

@pvgenuchten all good!


fix(@ngtools/webpack): perform import eliding before TypeScript transforms

Due to the method used by TypeScript to emit decorator metadata via the emitDecoratorMetadata function, the import eliding algorithm within the Angular compiler plugin may errantly remove imports to type metadata included in the emitted decorator calls. By moving the eliding before TypeScript, the eliding can use the Type nodes to analyze for used imports when emitDecoratorMetadata is enabled. This fix also reworks the previous fix to prevent the eliding to errantly remove certain factory functions that TypeScript may use in emitted code but are not yet referenced in the input code. The previous fix was to move the eliding after the TypeScript transformations.  The new fix is therefore not as comprehensive as the original but covers the usecase within the originating issue (#13297).

Thank you for this, it's been driving me crazy for the last few hours.

Thank you for this, it's been driving me crazy for the last few hours.


Add an ability to use 'isRevealingMask' option

I've added an ability to use isRevealingMask with this component.
The description for this option is in
https://github.com/insin/inputmask-core#isrevealingmask--boolean

+1
I noticed this option today while reading over the inputmask-core docs and would love to have this integrated!
Specifically, I think it's annoying when the input is masked with and underscore ("_") or space (" ") and you have to delete back through all that to enter in more text. And when you set the placeholderChar prop to false or an empty string '' it just defaults back to the underscore.

+1 for this prop. I would also like to see this implemented.


feat(@angular-devkit/build-angular): warn when target greater es2015

When using differential loading with targets ES5 and ES2016+, the
browser with ESM support will pick script[type="module"] scripts even
without supporting ES2016+ syntax.
We want to warn users in this case.

This is good for 8.0.  But I think that long-term, it should leverage the Browserslist to only show a warning if the required browsers do not support the ES version.

@clydin currently the check verifies if there are browsers which require ES5 support and warns only in such case.


[BUG] Cannot export COCO 2017 to fiftyone.types.ImageSegmentationDirectory

Instructions
Thank you for submitting an issue. Please refer to our
issue policy
for information on what types of issues we address.
Please fill in this template to ensure a timely and thorough response.

Place an "x" between the brackets next to an option if it applies. Example:

[x] Selected option

Please delete this section (all content above this line) before submitting
the issue

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 20.04 LTS
FiftyOne installed from (pip or source): pip
FiftyOne version (run fiftyone --version): fiftyone v0.11.1, Voxel51, Inc.
Python version: Python 3.9.6

Commands to reproduce
As thoroughly as possible, please provide the Python and/or shell commands used
to encounter the issue. Application steps can be described in the next section.
save script below as errorrepro.py then run python errorrerpo.py
import fiftyone as fo
import fiftyone.zoo as foz
import eta

name = "cocopeoplesegmenations"
not_complete = True
while not_complete:
    try:
        dataset = foz.load_zoo_dataset(
            "coco-2017",
            split="train",
            label_types=["segmentations"],
            classes=["person"],
            max_samples=50000,
            dataset_name=name,
        )
        not_complete = False
    except eta.core.web.WebSessionError:
        pass

label_field = "ground_truth"  # for example

dataset.peristent = True
dataset.default_classes = dataset.distinct(f"{label_field}.segmentations.label")

export_dir = f"/media/sam/trainer/datasets/{name}"

dataset.export(
    export_dir=export_dir,
    dataset_type=fo.types.ImageSegmentationDirectory,
    label_field=label_field,
)

Describe the problem
Describe the problem clearly here. Include descriptions of the expected
behavior and the actual behavior.
Existing download of split 'train' is sufficient
Loading 'coco-2017' split 'train'
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [158.0ms elapsed, 0s remaining, 63.3 samples/s]
Dataset 'cocopeople' created
   0% |/-------------------------------------------------------------------------------------------------------------------------------|  0/10 [5.3ms elapsed, ? remaining, ? samples/s]
Uncaught exception
Traceback (most recent call last):
  File "/media/sam/trainer/gitlab/kognat/rotobot-trainer-docker/openimagefo.py", line 33, in <module>
    dataset.export(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/core/collections.py", line 5502, in export
    foud.export_samples(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 290, in export_samples
    write_dataset(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 343, in write_dataset
    _write_image_dataset(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 666, in _write_image_dataset
    dataset_exporter.export_sample(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 2352, in export_sample
    etai.write(segmentation.mask, out_mask_path)
AttributeError: 'Detections' object has no attribute 'mask'

The exporters class does not support segmentation data.
Code to reproduce issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem.
import fiftyone as fo
import fiftyone.zoo as foz
import eta

name = "cocopeoplesegmenations"
not_complete = True
while not_complete:
    try:
        dataset = foz.load_zoo_dataset(
            "coco-2017",
            split="train",
            label_types=["segmentations"],
            classes=["person"],
            max_samples=50000,
            dataset_name=name,
        )
        not_complete = False
    except eta.core.web.WebSessionError:
        pass

label_field = "ground_truth"  # for example

dataset.peristent = True
dataset.default_classes = dataset.distinct(f"{label_field}.segmentations.label")

export_dir = f"/media/sam/trainer/datasets/{name}"

dataset.export(
    export_dir=export_dir,
    dataset_type=fo.types.ImageSegmentationDirectory,
    label_field=label_field,
)

Other info / logs
Existing download of split 'train' is sufficient
Loading 'coco-2017' split 'train'
 100% |████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [158.0ms elapsed, 0s remaining, 63.3 samples/s]
Dataset 'cocopeople' created
   0% |/-------------------------------------------------------------------------------------------------------------------------------|  0/10 [5.3ms elapsed, ? remaining, ? samples/s]
Uncaught exception
Traceback (most recent call last):
  File "/media/sam/trainer/gitlab/kognat/rotobot-trainer-docker/openimagefo.py", line 33, in <module>
    dataset.export(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/core/collections.py", line 5502, in export
    foud.export_samples(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 290, in export_samples
    write_dataset(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 343, in write_dataset
    _write_image_dataset(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 666, in _write_image_dataset
    dataset_exporter.export_sample(
  File "/home/sam/miniconda3/envs/bigdata/lib/python3.9/site-packages/fiftyone/utils/data/exporters.py", line 2352, in export_sample
    etai.write(segmentation.mask, out_mask_path)
AttributeError: 'Detections' object has no attribute 'mask'

What areas of FiftyOne does this bug affect?

[ ] App: FiftyOne application issue
[ * ] Core: Core fiftyone Python library issue
[ ] Server: Fiftyone server issue

Willingness to contribute
The FiftyOne Community encourages bug fix contributions. Would you or another
member of your organization be willing to contribute a fix for this bug to the
FiftyOne codebase?

[ ] Yes. I can contribute a fix for this bug independently.
[ * ] Yes. I would be willing to contribute a fix for this bug with guidance
from the FiftyOne community.
[ ] No. I cannot contribute a bug fix at this time.

Hey @kognat-docs 👋
The issue is that the ImageSegmentationDirectory format is for semantic segmentation data stored in a Segmentation field, but you're trying to export instance segmentations stored in a Detections field.
If you wanted to export in ImageSegmentationDirectory format, you'd have to do the manipulations to paint the instance data into per-image masks stored in a new Segmentations field.
Here's an example of exporting in COCODetectionDataset format, which does support instance segmentations:
(FYI your related bug #1138 is noted)
import fiftyone as fo
import fiftyone.zoo as foz

# Load some person segmentations from COCO-2017 dataset
dataset = foz.load_zoo_dataset(
    "coco-2017",
    split="validation",
    label_types=["segmentations"],
    classes=["person"],
    max_samples=10,
)

session = fo.launch_app(dataset)

# Export in COCO format
dataset.export(
    export_dir="/tmp/coco",
    dataset_type=fo.types.COCODetectionDataset,
)

# Load the instance segmentations from the export back into FiftyOne
dataset2 = fo.Dataset.from_dir(
    dataset_dir="/tmp/coco",
    dataset_type=fo.types.COCODetectionDataset,
    label_types=["segmentations"],  # necessary or else only bboxes will be loaded
)

session.dataset = dataset2

So there is no way to pool all of the Instance Segmentation masks into a per image segmentation mask
I can do this with pycocotool.
But at that stage there is no need for voxel51's toolset.
This is summarises here:
https://forums.fast.ai/t/image-segmentation-on-coco-dataset-summary-questions-and-suggestions/36816
I thought given it is possible that you would have automatic convertion from COCODetectionDataset to ImageSegmentationDirectory as it is only a down cast of type.

The nomeclature is unclear it says "segmentations" but it means "instance segmentations" and you have made it clear that segmentations and instance segmentations called detections are not interoperable.
The docs do not explain this suitably.

Do any of the Dataset Zoo support "Segmentations" https://voxel51.com/docs/fiftyone/user_guide/using_datasets.html#semantic-segmentation that can be exported to fo.types.ImageSegmentationDirectory
Example code for a dataset of fo.types.ImageSegmentationDirectory with a background class and a single label class is all I am looking for. Hopefully with more than 5000 samples.

+1
I am also encountering this problem. I have a coco-dataset that has segmentation polylines for multiple instances of each class within a single image but get this error when trying to convert to ImageSegmentationDirectory.
example CoCo annotation:
{:id 0,
 :image_id<PHONE_NUMBER>,
 :category_id 0,
 :segmentation
 [[143
   342
   140
   341
   137
   339
   134
   336
   134
   334
   136
   333
   139
   334
   143
   336
   147
   339
   147
   341
   145
   342]],
 :iscrowd 0,
 :area 66.5,
 :bbox [134 333 13 9]}

This gets loaded and displayed correctly in FiftyOne with a detection (the bbox) and the segmentation mask. But convert fails.  With the same error :AttributeError: 'Detections' object has no attribute 'mask'
The native format that I can see after loading the COCODataset into the FiftyOne app has a mask attribute, eg.
        'detections': BaseList([
            <Detection: {
                'id': '60f914fb2e10664628a5dfe4',
                'attributes': BaseDict({}),
                'tags': BaseList([]),
                'label': 'class_1',
                'bounding_box': BaseList([
                    0.40625,
                    0.2796296296296296,
                    0.013020833333333334,
                    0.014814814814814815,
                ]),
                'mask': array([[False, False,  True,  True,  True,  True, False, False, False,
                        False, False, False, False, False, False, False, False, False,
                        False, False, False, False, False, False, False],
                       [False,  True,  True,  True,  True,  True,  True,  True, False,
                        False, False, False, False, False, False, False, False, False,
                        False, False, False, False, False, False, False],
                       [ True,  True,  True,  True,  True,  True,  True,  True,  True,
                         True, False, False, False, False, False, False, False, False,
                        False, False, False, False, False, False, False],...

Do I have to convert the COCODetection into the native FiftyOne format on disk and then to ImageSegmentationDirectory?

Yeah I doubt that will work either
You need to get down into the pixel array with numpy, Pillow or OpenCV and Union combine each of the instance segmentation masks and create a new segmentation mask data structure and then serialise that to disk to declare it as Segmentation rather than Detection
Then you can have the ImageSegmentationDirectory

Yes, it doesn't work.
Which is a shame because those segmentation masks look just fine in the app - seems like an export button into image masks from the app or similar might be nice feature.
Like @kognat-docs I was hoping that the convert would save me the trouble of manually doing it ....

We can make it happen. But it will take some development and testing. I have been taken in by the glossy front end assuming that there is as an all singing all dancing API that could take away all details of dataset conventions. But the support has been great. I was surprised with some of the conversion that it was as only utilising a single thread , as far as I could tell from CPU occupancy. When you have 48 threads available and a RAID NVMe being bottle necked by a single thread to end up in an error twenty times trying to work out if you are using the API wrong is more expensive than writing 20 lines of PyCOCO code and wrapping it in some worker threads .

Hi @kognat-docs and @pdmct:
Conversion from instances/polylines to semantic segmentation format definitely makes sense. Indeed, it is our goal with FiftyOne to abstract away as many nitty gritty details of dataset curation as we can.
IN FACT, check this out https://github.com/voxel51/fiftyone/pull/1142.
You can try it out for yourself via a source install:
git clone https://github.com/voxel51/fiftyone
cd fiftyone
git checkout --track origin/to-segmentation
bash install.bash

I have been iterating on semantic segmentations from COCO 2017 using the sample code you have supplied with a model that is well suited to the ImageSegmentationDirectory export data format.
The install was via a wheel and I think you should drop this into the docs ASAP and merge it into the main branch.
Thanks so much.


DOM manipulation with JavaScript

Article checklist
ATTENTION: In your PR - add the ISSUE (topic form) # that this PR corresponds to. You can do this by also adding a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.
NOTE: (Please ensure that you have only one open issue + linked pull request at a time. This will ensure that we complete the article in a timely manner from inception to publishing.)
If you have not already please go over our Resources Page for more tips and tricks on how to improve our overall technical writing.
Preliminary Checks - Formatting and Structure

[x] Does your article follow any of the suggested structure formats? - see example formats
[x] Is your article properly formatted in Markdown? - see Github Markdown guide
[x] Have you used the correct folder and file structure? - see Contribution Guidelines
[x] Is your article over 750 words? - Tool: Wordcounter.net
[x] Have you used our preferred heading structure? - H3 (###) and up
[x] Does your article provide enough value and detail about your topic? - Articles should be clear, accurate, and fully explained.
[x] Can your article be understood by beginners? - Assume the audience is smart but has no prior exposure to the common terminology in your article.
[x] Have you included a hero image that is 600x400 pixels and have the copyright to use it? - Tools: Pixlr Image Editor for resizing and Unsplash for Creative Commons images
[x] In your PR - add a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.

Grammar & plagiarism checks

[x] Have you spellchecked and grammar checked your article? - Tools: Write&Improve - ProWritingAid -  Grammarly
[ ] Please place your article through a 3rd party plagiarism checker? We suggest using Quetext, this tool is free and has a daily limit. We typically accept articles with 10% or less.
[ ] Have you checked your article for readability? - Tool: Hemmingway
[x] Have you added sources for quotes and images that aren't yours?

Technical checks

[x] Are your code snippets properly formatted for syntax highlighting - see Syntax guide

Contribution guidelines
For first-time contributors and for more details, see Contributing Guidelines

This fixes #2546

@LinusMuema Please any update, it's almost 9 days now :)

Hi @Samuel-2626
Sorry it took a while but I am finishing the review today.

plagiarism check done

Thanks for the review @LinusMuema, but you have not added the label for final review.

Sorry about that @Samuel-2626
The rule is we ate supposed to add it once the plagiarism check is done and it's been a little over an hour since that.
But it's okay now

@Samuel-2626
Overall great work - please be sure to go over any edits I may have made.
Looking forward to more great content 🚀
cc @LinusMuema
Please do the same - see the formatting edits I made as well.


Cirleci: create-relayers job is emitting warnings

Our circle ci create-relayers job requires some attention:
https://app.circleci.com/pipelines/github/beamer-bridge/beamer/845/workflows/cc326376-2ba3-4833-911a-ed6616d233e0/jobs/5836

This could be fixed by using Node v16.
The warnings come from the peer dependencies which are not fully supporting Node v18.


Bindings using RoadNetwork unique_ptr are not working.

Summary
maliput_py stopped relying on custom pybind11 version. This seems to be causing a failure in delphyne bindings that are using std::unique_ptr<RoadNetwork>
https://github.com/maliput/delphyne/blob/141bc619447546c429591e45211ec7c2f38633a9/python/delphyne/roads.cc#L86-L90
Here delphyne::roads::CreateMalidriveRoadNetworkFromXodr returns a std::unique_ptr<RoadNetwork>. When using this binding this it is throwing an error because:
     File "/__w/delphyne_demos/delphyne_demos/maliput_ws/install/delphyne/lib/python3.8/site-packages/delphyne/behaviours/roads.py", line 146, in setup
        delphyne.roads.create_malidrive_road_network_from_xodr(
    TypeError: Unable to convert function return value to a Python type! The signature was
    	(name: str, file_path: str, rule_registry_file_path: str, road_rulebook_file_path: str, traffic_light_book_path: str, phase_ring_path: str, intersection_book_path: str, linear_tolerance: float = 0.001, angular_tolerance: float = 0.001) -> maliput::api::RoadNetwork

When maliput_py's bindings are compiled using the custom pybind11 version it works correclty.

Some ideas:

Move the delphyne python api that depends on maliput to not use maliput python bindings.
Build a service like API in delphyne to interact with maliput. Agents would make calls to the service instead of maliput pointers directly.

Some ideas:

Move the delphyne python api that depends on maliput to not use maliput python bindings.
Build a service like API in delphyne to interact with maliput. Agents would make calls to the service instead of maliput pointers directly.

I like the idea and I think that for how things are right now it's the tidier option.
What I found a bit bittersweet is that because of just using unique_ptr on already binded maliput type we can't rely directly on the maliput_py bindings which, not only is what they are for, but also delphyne could be seen as an example for the community on how using the bindings.
I have a workaround in this branch for this.
We could drop a real nice docstring explaining there, and once we move to Jammy, (technically we would be able to deal with this particular case), we could remove this.
Wdyt?

I concur. Note my comment in https://github.com/maliput/delphyne/commit/a15cc3076d444715c17d200105092a7e16fc6721#r77070044 which I guess it should help to reduce the code maintenance in delphyne for this workaround.
The second part of the comment might not be as useful because of use cases though.


Bugfix/ios session web view

By submitting a PR to this repository, you agree to the terms within the Auth0 Code of Conduct. Please see the contributing guidelines for how to create and submit a high-quality PR for this repo.
Description

Describe the purpose of this PR along with any background information and the impacts of the proposed change. For the benefit of the community, please do not assume prior context.
Provide details that support your chosen implementation, including: breaking changes, alternatives considered, changes to the API, etc.
If the UI is being changed, please provide screenshots.

References

Include any links supporting this change such as a:

GitHub Issue/PR number addressed or fixed
Auth0 Community post
StackOverflow post
Support forum thread
Related pull requests/issues from other repos

If there are no references, simply delete this section.

Testing

Describe how this can be tested by reviewers. Be specific about anything not tested and reasons why. If this library has unit and/or integration testing, tests should be added for new functionality and existing tests should complete without errors.
Please include any manual steps for testing end-to-end or functionality not covered by unit/integration tests.
Also include details of the environment this PR was developed in (language/platform/browser version).

[ ] This change adds test coverage for new/changed/fixed functionality

Checklist

[ ] I have added documentation for new/changed functionality in this PR or in auth0.com/docs
[ ] All active GitHub checks for tests, formatting, and security are passing
[ ] The correct base branch is being used, if not master

Hey,
Not sure what the intention here is, but it looks like you are adding support for ionic.
Worth pointing out that we are making changes to the SDK ourselves to support ionic, but they are not in line with the changes we have in this PR.
Also I don't think we want to be changing things like the package.json file and including the dist folder.
Keep in kind that we advise you to open an issue before opening any PRs as well as to add a decent description to the PR so we are aware of what is being solved.
Closing this, but we are working on ionic support.


xSQLServer: Fix minor typos in CHANGELOG.md

Pull Request (PR) description

Changes to xSQLServer

Fix minor typos in CHANGELOG.md

This Pull Request (PR) fixes the following issues:
n/a
Task list:

[x] Change details added to Unreleased section of CHANGELOG.md?
[ ] Added/updated documentation, comment-based help and descriptions in .schema.mof files where appropriate?
[ ] Examples appropriately updated?
[ ] New/changed code adheres to Style Guidelines?
[ ] Unit and (optional) Integration tests created/updated where possible?

This change is

Codecov Report

Merging #815 into dev will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff         @@
##            dev   #815   +/-   ##
===================================
  Coverage    96%    96%
===================================
  Files        30     30
  Lines      3259   3259
===================================
  Hits       3142   3142
  Misses      117    117

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Documentation links broken

The two links in the notes box: Create a network installation of Visual Studio 2017 and Install certificates required for Visual Studio offline installation, both point to 404s. Please fix.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 228076d1-94bf-c502-c018-08f7c0faf377
Version Independent ID: c64a5cd5-6236-4192-2bfc-923aa20c94da
Content: Create an Offline Installation of Visual Studio - Visual Studio
Content Source: docs/install/create-an-offline-installation-of-visual-studio.md
Product: visual-studio-dev15
GitHub Login: @TerryGLee
Microsoft Alias: tglee

@lavahot  These were down today due to a site issue. They should be working again now.

Yep! Thank you!


Date on the article is confusing

It's listed as 08/16/2017, but the first sentence of the article has 'The Microsoft Threat Modeling Tool 2018 was released as GA in September 2018 ..', which over a year later.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7dea734e-e546-2acd-3a6c-c0a8390f9ee9
Version Independent ID: 3a4c0ad1-7a2f-9590-9b96-e1285ac9769c
Content: Getting Started - Microsoft Threat Modeling Tool - Azure
Content Source: articles/security/azure-security-threat-modeling-tool-getting-started.md
Service: security
GitHub Login: @jegeib
Microsoft Alias: jegeib

@smakarsky Thanks for your feedback! We will investigate and update as appropriate.

@smakarsky Thanks for the feedback !  I have assigned this issue to content author to investigate and update the document as appropriate.
@jegeib Can you please help here.  Also, is this date reflects the document last updated date and is it automated ?

@saurbhsh Is there official Azure Docs guidance as to what is supposed to be in ms.date field in the header? Currently it's the creation date rather then last updated.

Hi @smakarsky, @smakarsky, @saurbhsh, Date is still wrong...


Update README.md

geocoding: ^1.0.2 -> geocoding: ^1.0.3
:sparkles: What kind of change does this PR introduce? (Bug fix, feature, docs update...)
docs update
:arrow_heading_down: What is the current behavior?
:new: What is the new behavior (if this is a feature change)?
:boom: Does this PR introduce a breaking change?
No
:bug: Recommendations for testing
:memo: Links to relevant issues/docs
:thinking: Checklist before submitting

[x] All projects build
[x] Follows style guide lines (code style guide)
[x] Relevant documentation was updated
[x] Rebased onto current develop

Great catch, thank you for the update


The Vertical WebPart

The Vertical WebPart is having an error It does not fetch the Title and Description of the event only shows the static Date and time of the   ##events. The Dynamic animation where in the Title and Description of event slide into view is not working. Tried the latest version for deployment, link still this issue persists.
If you would reply then it would be very greatfull @OlivierCC

I'm experiencing the same behavior. Vertical Timeline only displays date and time, it's does not display the title or description.

I'm getting the same but it actually looks like the issue is that the animation isn't firing, when I inspected the elements, the content is actually loaded, it's opacity is just set to 0. Maybe this is an issue when there are too few events?
.timeline.animated .timeline-row .timeline-content {     **opacity: 0;**     left: 20px;     -webkit-transition: all 0.8s;     -moz-transition: all 0.8s;     transition: all 0.8s; }

@zshane15 this resolved the issue of the display but how do we keep the animation?

@mirahManlapig sorry it's been a while since I looked at this, but maybe play around with an animation delay class once you figure out how many items are returned? That way it would simulate an animation, otherwise it just works as normal.
.tooFewEvents {
  animation-delay: 2s;
}

@zshane15 , thanks for your response. However, I tried the following:
  .timeline.animated .timeline-row .timeline-content {     /**opacity: 0;**/     left: 20px;     -webkit-transition: all 0.8s;     -moz-transition: all 0.8s;     -webkit-animation-delay: 2s;     transition: all 0.8s;      animation-delay: 2s;   }
But the animation still didn't appear. I am using the workbench test data and i have 10 items in the timeline.

I found the answer. It seems like previously, inside the SharePoint page there's an id named pageContent. But now it needs class mainContent instead.
`    $(".mainContent").scroll(() => {
this.timelineAnimate();
});
}
private timelineAnimate(): void {
$(".timeline.animated .timeline-row").each(function (i) {
var bottom_of_object, bottom_of_window;
bottom_of_object = $(this).position().top + $(this).outerHeight();
bottom_of_window =
$(".mainContent").scrollTop() + $(".mainContent").height();
if (bottom_of_window > bottom_of_object) {
return $(this).addClass("active");
}
});
}`


fix country code not correct cause panic

@Dike90 Thank you for the PR but I was already working on the issue with other fixes and it should be working now...


Restricting the list of possible mapping_key values for a model

Added concept of mapping_key_list which allows to restrict which fields can be used for mapping.
Example - User model with Devise has lots of attributes which you might want to import but don't want to use for mapping.
Feel free to change the name if you do not like mapping_key_list.  It would be also helpful to update the docs to explain how to use it via array
mapping_key :email
mapping_key_list [:email, :some_unique_id]
Also added :c_at and :u_at to default_excluded_fields, used in Mongoid::Timestamps::Short

I can see how that can help if you have models with lots of fields. I'll merge this improvement in. Thanks!


Fill operator

In JavaScript, the fill operator is a very handy thing:
const template = {
  foo: "Hello",
  bar: "World",
  baz: "!",
};

const specialized = {
  ...template,
  bar: "Bob"
};

Since ... in lua is used for unpacking variable arguments and therefore can't be used here, I'd like to suggest that .. be used as the fill operator only when it comes before an identifier in a table:
str1 = "Hello"
str2 = "world"
str3 = str1.." "..str2.."!" -- Concat operator is used here

template = {
  foo: "Hello",
  bar: "World",
  baz: "!",
}

specialized = {
  ..template, -- Fill operator is used here
  foo: "Goodbye",
  ..template, -- Fill operator is used here
  bar: "Bob",
  baz: template.baz.."?", -- Concat operator is used here
}

-- specialized = { foo: "Hello", bar: "Bob", baz: "" }

Compilation for this might require a utility function to be baked in:
_G["月"] = {
  fill = function(from, to)
    local mt = getmetatable(from)
    if mt ~= nil then
      -- Checks for a __fill metamethod, in case classes or other special tables should be handled differently.
      if type(mt.__fill) == "function" then
        mt.__fill(from, to)
        return
      end
    end
    for key, value in next, from do to[key] = value end
  end,
}

local template = {
  foo = "Hello",
  bar = "World",
  baz = "!",
}

local specialized
do
  local o = { }
  _G["月"].fill(template, o)
  o.foo = "Goodbye"
  _G["月"].fill(template, o)
  o.bar = "Bob"
  o.baz = template.baz.."?"
  specialized = o
end

Of course, the utility function could be inlined, but I feel like thats probably a waste.

Yuescript is more like a macro system for Lua to introduce more syntax sugar and compiles to Lua. Since the table filling function is not implemented in Lua compiler itself, the Yuescript compiler has to add some inlined Lua codes to finish the job.  In this way, for different code files in the same project, the utility function will be duplicated.

There is a Coffeescript dialect named Livescript add many similar features to add new syntax functions by inlining many Javascript codes or functions, but Livescript provides function to merge multiple source files into one source file so that the duplicated inline codes could be eliminated.

I think thats true-- this would be better to implement if (when?) YueScript can compile several files together

latest yuescript version workaround (only index)
template = {
  foo: "Hello",
  bar: "World",
  "!"
}

specialized = {
  index#: template, -- "fill" operator workaround (uses metatable __index)
  bar: "Bob"
}

print specialized.foo .. " " .. specialized.bar .. specialized[1] --> "Hello Bob!"

Add an implementation for the Fill operator, works on both array table and hash table.
template = {
  foo: "Hello"
  bar: "World"
  "!"
}

specialized = {
  "a", "b", "c"
  ...template
  bar: "Bob"
}

Compile to:
local template = {
  foo = "Hello",
  bar = "World",
  "!"
}
local specialized
do
  local _tab_0 = {
    "a",
    "b",
    "c"
  }
  local _idx_0 = 1
  for _key_0, _value_0 in pairs(template) do
    if _idx_0 == _key_0 then
      _tab_0[#_tab_0 + 1] = _value_0
      _idx_0 = _idx_0 + 1
    else
      _tab_0[_key_0] = _value_0
    end
  end
  _tab_0.bar = "Bob"
  specialized = _tab_0
end

Get result:
specialized = {
  [1] = "a"
  [2] = "b"
  [3] = "c"
  [4] = "!"
  [foo] = "Hello"
  [bar] = "Bob"
}


Why does right_of(self, w, padding) use size of self instead of size of w

Given the following code:
let left = Button::default()
    .with_pos(0, 0)
    .with_size(80, 20)
    .with_label("left");

let right = Button::default()
    .with_size(30, 20)
    .right_of(&btn_left, 0)
    .with_label("right");

Expected
Quoting the docs: Positions the widget to the right of w, the size of w should be known, right should be positioned to the right of left, roughly:
|        left        ||right|

Where right.x() = left.x() + left.width()+padding
Actual
right is positioned based on its own size, yielding the result:
|       |right|     |

Where right.x() = left.() + self.width()+padding
Excerpt from fltk-drive/src/widget.rs#L833:
fn right_of<W: WidgetExt>(mut self, w: &W, padding: i32) -> Self {
    assert!(!w.was_deleted());
    assert!(!self.was_deleted());
    debug_assert!(self.width() != 0 && self.height() != 0, "right_of requires the size of the widget to be known!");
    self.resize(w.x() + self.width() + padding, w.y(), self.width(), self.height());
    self
}

In the function instead of using the size of w, the size of self is used.

If my expected behavior is correct, I've created a PR with the fix: #366

Hi
You're right! It was a miscalculation.

That's a very timely response =D. Aight, thanks for your hard work!

Thank you


Fix nil pointer exceptions in certain broken dependency scenarios

When there are one or more broken dependency problems, the final value of dep when evaluating all entries is the one used when preparing the problem item.
However, it is possible that after a problem is identified, subsequent entries are recording or alerting rules which either do not have any vector selectors, or none that match.
In this situation, the final value of dep is nil, which causes a crash.
This change fixes this by using the first broken dependency entry recorded for building this part of the string.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.86%. Comparing base (7cea6a7) to head (c3716b0).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #908   +/-   ##
=======================================
  Coverage   94.86%   94.86%
=======================================
  Files          93       93
  Lines        8797     8797
=======================================
  Hits         8345     8345
  Misses        311      311
  Partials      141      141

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Make TableControlBar more flexible

I've got a couple of tables where I need the selection state to be controlled from outside the table context, and therefore, I need the display of the selection state to also be controlled from outside of the table context. This allows me to do so, with some defensive programming added in to cover different ways someone might want to use the control bar.

Coverage decreased (-0.08%) to 89.757% when pulling ad3577b84945d61aa906c97f2cf606e3ac95dbb8 on lbirch/manualSelect into 8277f7b9970a31fa0ba313f3ef4f41aaca1878df on master.


Fix JQ error in device-virtual tests in jakarta and latest

Signed-off-by: Mengyi Wang<EMAIL_ADDRESS>

It always falls into the third and new condition:
$ sudo ./run-all-tests-locally.sh -v -t test-device-virtual.sh
testing snap from default test channel
running single test: test-device-virtual.sh ... FAILED:
DEFAULT_TEST_CHANNEL not set. Setting to latest/beta
snap "edgexfoundry" is not installed
Installing edgexfoundry with channel latest/beta
edgexfoundry (beta) 2.2.0-dev.28 from Canonical* installed
waiting for all services to come online. Current retry count: 1/300
waiting for all services to come online. Current retry count: 2/300
waiting for all services to come online. Current retry count: 3/300
waiting for all services to come online. Current retry count: 4/300
waiting for all services to come online. Current retry count: 5/300
all services up
Started.
check port 59900 status open services timed out, current retry count: 1/300
finding jq... found at /snap/bin/edgexfoundry.jq
{
  "apiVersion": "v2",
  "statusCode": 200,
  "totalCount": 5,
  "devices": [
    {
      "created":<PHONE_NUMBER>516,
      "modified":<PHONE_NUMBER>516,
      "id": "f203312b-a1a7-473a-bb15-f2440141c6fd",
      "name": "Random-Float-Device",
      "description": "Example of Device Virtual",
      "adminState": "UNLOCKED",
      "operatingState": "UP",
      "labels": [
        "device-virtual-example"
      ],
      "serviceName": "device-virtual",
      "profileName": "Random-Float-Device",
      "autoEvents": [
        {
          "interval": "30s",
          "onChange": false,
          "sourceName": "Float32"
        },
        {
          "interval": "30s",
          "onChange": false,
          "sourceName": "Float64"
        }
      ],
      "protocols": {
        "other": {
          "Address": "device-virtual-float-01",
          "Protocol": "300"
        }
      }
    },
    {
      "created":<PHONE_NUMBER>516,
      "modified":<PHONE_NUMBER>516,
      "id": "5f810257-8bbf-4c4c-95b4-2956587d9f59",
      "name": "Random-Integer-Device",
      "description": "Example of Device Virtual",
      "adminState": "UNLOCKED",
      "operatingState": "UP",
      "labels": [
        "device-virtual-example"
      ],
      "serviceName": "device-virtual",
      "profileName": "Random-Integer-Device",
      "autoEvents": [
        {
          "interval": "15s",
          "onChange": false,
          "sourceName": "Int8"
        },
        {
          "interval": "15s",
          "onChange": false,
          "sourceName": "Int16"
        },
        {
          "interval": "15s",
          "onChange": false,
          "sourceName": "Int32"
        },
        {
          "interval": "15s",
          "onChange": false,
          "sourceName": "Int64"
        }
      ],
      "protocols": {
        "other": {
          "Address": "device-virtual-int-01",
          "Protocol": "300"
        }
      }
    },
    {
      "created":<PHONE_NUMBER>517,
      "modified":<PHONE_NUMBER>517,
      "id": "48e8dce5-883e-44d7-91fa-1a968b99b1af",
      "name": "Random-Binary-Device",
      "description": "Example of Device Virtual",
      "adminState": "UNLOCKED",
      "operatingState": "UP",
      "labels": [
        "device-virtual-example"
      ],
      "serviceName": "device-virtual",
      "profileName": "Random-Binary-Device",
      "protocols": {
        "other": {
          "Address": "device-virtual-binary-01",
          "Port": "300"
        }
      }
    },
    {
      "created":<PHONE_NUMBER>516,
      "modified":<PHONE_NUMBER>516,
      "id": "3e42e749-063d-4fcc-af31-5b2f7d3cee1a",
      "name": "Random-UnsignedInteger-Device",
      "description": "Example of Device Virtual",
      "adminState": "UNLOCKED",
      "operatingState": "UP",
      "labels": [
        "device-virtual-example"
      ],
      "serviceName": "device-virtual",
      "profileName": "Random-UnsignedInteger-Device",
      "autoEvents": [
        {
          "interval": "20s",
          "onChange": false,
          "sourceName": "Uint8"
        },
        {
          "interval": "20s",
          "onChange": false,
          "sourceName": "Uint16"
        },
        {
          "interval": "20s",
          "onChange": false,
          "sourceName": "Uint32"
        },
        {
          "interval": "20s",
          "onChange": false,
          "sourceName": "Uint64"
        }
      ],
      "protocols": {
        "other": {
          "Address": "device-virtual-uint-01",
          "Protocol": "300"
        }
      }
    },
    {
      "created":<PHONE_NUMBER>515,
      "modified":<PHONE_NUMBER>515,
      "id": "198ec73d-3524-4f4d-a65b-8a1f78b8de64",
      "name": "Random-Boolean-Device",
      "description": "Example of Device Virtual",
      "adminState": "UNLOCKED",
      "operatingState": "UP",
      "labels": [
        "device-virtual-example"
      ],
      "serviceName": "device-virtual",
      "profileName": "Random-Boolean-Device",
      "autoEvents": [
        {
          "interval": "10s",
          "onChange": false,
          "sourceName": "Bool"
        }
      ],
      "protocols": {
        "other": {
          "Address": "device-virtual-bool-01",
          "Port": "300"
        }
      }
    }
  ]
}
Error finding the device created by device-virtual


Neumorphism login form using html/css

I want to contribute this project. Assign me under hacktoberfest21.

Assigning 🎊🎊


When using assembly link to dll, imported namespace is marked as an unresolved symbol.

I use this attributes for my tt file in Unity3D project. tt generates code that is compiled into Assembly-CSharp.dll. Then I use it to import its code.
<#@ template debug="true" hostspecific="false" language="C#" #>
<#@ output extension="cs" #>
<#@ assembly name="$(ProjectDir)\Temp\UnityVS_bin\Debug\Assembly-CSharp.dll" #>
<#@ import namespace="MyApp.Common.CodeGeneration" #>

It works however when I start VS2015 next time, Resharper marks MyApp as an unresolved symbol!

It's possible, it could be the same issue as https://github.com/MrJul/ForTea/issues/50
VS 2015 (14.0.24720.00 Update 1),
Resharper 10.0.2,
ForTea 1.5.0

I'm closing this as a duplicate of #50.


Add support for SSL

Description
This adds the option to configure SSL mutual auth for connections between twemproxy and a redis server [1]. Encrypting data in transit is a requirement for GDPR and this enables that.
[1] redis doesn't support SSL natively. In practice this will be a connection to an SSL proxy in front of redis.
How Has This Been Tested?

The existing non-memcached unit-tests pass (i.e. redis and non-backend specific tests).
No new warnings from non-library code (a few from OpenSSL).
Added C unit tests for a custom buffer manipulation function that was added.
Ran a redis process in vagrant and tested get/set commands from outside of vagrant for a few cases:

A single-server pool that only connects over SSL.
A single-server pool that only connects over non-SSL.
A two-server pool that only connects over SSL.
Two server pools: one with one SSL server and the other with one non-SSL server.

All function response codes are checked for errors and either handled or logged.
Sent a GET command over SSL, waited until redis disconnected from twemproxy, then sent another GET command to confirm the server reconnected.

Also repeated this test with a mix of SSL and non-SSL pools.

Ran with asan (address sanitizer)[2] to check for memory issues in the following cases:

Start server, read from both redii in the pool, wait for redii to disconnect, read from redii again, wait for disconnect again.

[2] Using these flags: -fsanitize=address -O1 -fno-omit-frame-pointer -g
TODO

[ ] Try to run msan but exclude files from twemproxy's master branch.
[x] Address FIXMEs/TODOs

Licenses
This adds openssl, which has a license that requires has a notice-style license but is acceptable to use at Google. We already use it elsewhere, which likely means we're in compliance.

Looks great!


Weird footer.

Not a huge problem that I have here, it is just something which could be improved or fixed on the party.js [website]

The centering of the "More, GitHub" on the footer is strange. It is neither centered nor is it aligned to a side
The fix I'd like to see for this would be aligning it to the right.
An alternative fix that would also work is centering it, though imo aligning it to the right would look cooler :smile:

Ah, good catch! I completely glanced over that, because there were 3 columns in the template before (Docs, Blog, More), but I got rid of the blog. I'll mess around with a solution sometime over the next days!


[BUG] Common Ransomware Extensions - Lack of Fidelity

If you have a Splunk Support contract, creating a support case for your issue may result in faster resolution.
Describe the bug
Correlation is somewhat noisy in a production environment.
Found that refactoring SPL results with better accuracy. Added additional logic to align correlation with false positive description text "If this is a true ransomware attack, there will be a large number of files created with these extensions" - which is typically true of ransomware behavior
Made the following changes:

Refactored SPL to add extra context and align correlation with use case.
Added an extraction (true_file_path) for file path without leaf/file included
Add stats on # of files and # of paths present per destination/device
Use where statement to reduce noise potential of correlation with respect to "true" ransomware behavior (multiple files/paths impacted)

| tstats `security_content_summariesonly` min(_time) as firstTime max(_time) as lastTime count latest(Filesystem.user) as user values(Filesystem.file_path) as file_path from datamodel=Endpoint.Filesystem by Filesystem.file_name Filesystem.dest _time span=1h
| `drop_dm_object_name(Filesystem)`
| rex field=file_name "(?<file_extension>\.[^\.]+)$"
| rex field=file_path "(?<true_file_path>([^\\\]*\\\)*).*"
| stats min(firstTime) as firstTime max(lastTime) as lastTime latest(user) as user dc(true_file_path) as path_count dc(file_name) as file_count latest(file_name) as file_name latest(true_file_path) as file_path by dest file_extension
| `security_content_ctime(lastTime)`
| `security_content_ctime(firstTime)`
| `ransomware_extensions`
| where path_count > 1 OR file_count > 20
| `common_ransomware_extensions_filter`

Expected behavior
Higher fidelity alerting with regards to potential ransomware infection.
Screenshots

App Version:

ESCU: [e.g. 3.3.1]

Additional context
Original Search: This search has completed and has returned 414 results by scanning 36,544,886 events in 38.897 seconds
Modified Search: This search has completed and has returned 3 results by scanning 36,591,795 events in 38.664 seconds

Thank you for the feedback on this. I agree with the changes and will work this into the next release!

Yes - I like that description a lot more! Apologies for missing that on this release. I'll get it out on the next.

Hey @nterl0k  Where are you pulling the $Name$ field from?

The 'ransomware_extensions' macro as part of a lookup.

merged!


Cannot open firefox on newly install Ubuntu 22.04 for Linux

Hi, I just downloaded Ubuntu 22.04 on Macbook and would like to dowload firefox. I've tried different installation commands such as "sudo apt install firefox" and "sudo snap install firefox", but I still cannot open firefox and the error message is shown in the picture

Hardware accelerated graphics seem the problem.
It's not the solution but a temporary workaround could be to choose a non gl display setting in UTM vm setting.
virtio-ramfb appears to work.
hopefully the devs. can pick this up and apply the fix soon.
Cheers!

Thank you so much for the workaround. It seems to affect Cinnamon desktop env rendering as well. But if I switch desktops and devices back to the non-accel drivers, firefox and all windows work again!

I have install Ubuntu on ARM and have deinstall the firefox snap and replace snap-Firefox to a PPA
sudo snap remove firefox
and install the
sudo add-apt-repository ppa:mozillateam/ppa && sudo apt update
and
sudo apt install firefox-esr firefox-esr-locale-de

+1

Hardware accelerated graphics seem the problem. It's not the solution but a temporary workaround could be to choose a non gl display setting in UTM vm setting. virtio-ramfb appears to work.
hopefully the devs. can pick this up and apply the fix soon.
Cheers!

Fixed following this steps and changing the display settings to virtio-gpu-pci

It's great that there are these workarounds, but is GPU Acceleration being broken getting fixed?
I'm not trying to be demanding or whatever but just wondering. Because it worked fine on UTM 3.x but it's broken on 4.x. I'd just like for the GPU acceleration to work again.
Cheers

I do have the same problem with firefox or any other software that tries to use VA:
vainfo
libva info: VA-API version 1.14.0
libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
vaInitialize failed with error code -1 (unknown libva error),exit

walter@ubuntu2204:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

Graphic Card: virtio-ramfb-gl

When I change to a different GPU I get no display output at all. To be honest... it makes UTM unusable for now.

When I change to a different GPU I get no display output at all. To be honest... it makes UTM unusable for now.

I have the same issue.

I solve the problem, with these commands:
sudo apt update;
sudo apt install snapd apparmor;
sudo systemctl enable snapd.apparmor;
sudo systemctl start snapd.apparmor;
sudo systemctl enable snapd;
sudo systemctl start snapd;

That worked! Thank you!


Implement Kubeapi and PVC specific API

Add kubeapi and PVC specific API

This commit implements the following:
1) API to get the kubernetes config.
2) API to waitForKuberntes
3) API to waitForLonghornReady
4) API to waitForPVCReady
5) API to waitForNodeReady, checks for everything k3s, kubevirt, longhorn
6) PVC specific API to create,delete,get PVC info.
7) kubenetes specific types in kube.go

NOTE: This commit defines the API. The callers of this API will be in future commits and well as network specific API.
Testing done:  Compiles pkg/pillar successfully on kvm based eve.

@andrewd-zededa and @naiming-zededa FYI.  Couple of API in kubeapi.go were added by you guys

This PR needs to be reviewed and merged. This is the base PR for many other PRs that follow. Basically other PRs will not even compile without this PR merged.

BTW, the kubevirt eve numbers I gave above are  not from this PR alone but from including all components (future PRs for networking, metrics, domainmgr etc). So consider those as the worst case numbers.

Maybe we need to fork eve for the kubevirt case? :)

(ns: pillar) da95779f-6b64-41d6-a843-917b5fac39e2:/# ls -l /opt/zededa/bin/zedbox -rwxr-xr-x 1 root root 101826600 Jan 8 18:28 /opt/zededa/bin/zedbox
This is running with all agents running (ns: pillar) da95779f-6b64-41d6-a843-917b5fac39e2:/# cat /proc/1927/status | grep VmRSS VmRSS: 588068 kB
Looks pretty high compared to kvm eve which is 114MB RSS and 51MB file size

@zedi-pramodh Does RSS increase also when you build kvm-variant on the kubevirt-poc branch? In other words, are we adding any overhead to non-kubevirt variants?

Maybe we need to fork eve for the kubevirt case? :)

We might eventually end up like that. Lets see.

Maybe we need to fork eve for the kubevirt case? :)

We might eventually end up like that. Lets see.

We can use build tags to build kubernetes dependencies only when HV=kubevirt, see e.g.: https://www.digitalocean.com/community/tutorials/customizing-go-binaries-with-build-tags

We can use build tags to build kubernetes dependencies only when HV=kubevirt, see e.g.: https://www.digitalocean.com/community/tutorials/customizing-go-binaries-with-build-tags

This is awesome, we should look at that.

(ns: pillar) da95779f-6b64-41d6-a843-917b5fac39e2:/# ls -l /opt/zededa/bin/zedbox -rwxr-xr-x 1 root root 101826600 Jan 8 18:28 /opt/zededa/bin/zedbox
This is running with all agents running (ns: pillar) da95779f-6b64-41d6-a843-917b5fac39e2:/# cat /proc/1927/status | grep VmRSS VmRSS: 588068 kB
Looks pretty high compared to kvm eve which is 114MB RSS and 51MB file size

Adding GetDiskActualSize() is a no-brainer since it doesn't bring in additional packages.
Will the other APIs only be invoked when we boot and/or deploy some new app instance?
If so maybe it would make sense to build a separate golang binary outside of zedbox with the other API calls as a command which can be invoked from eve using os.Exec("/opt/zededa/bin/kubecmds", "arg1", "arg2")
and that kubecmds will exit once done.
This avoids making zedbox binary and RSS grow so much. But we'd still need sufficient memory to be able to run the kubecmds binary when we need; maybe that means reserving more memory for EVE using cgroups in the kubevirt case.

BTW, the kubevirt eve numbers I gave above are not from this PR alone but from including all components (future PRs for networking, metrics, domainmgr etc). So consider those as the max size we will go.

That's why I suggested comparing RSS with a truncated device-steps.sh for both master branch and this PR.
On line 121 (after zedbox is started), add a
while true; do sleep 3600; done
Then you can see the RSS from just the init sections and global variables in zedbox.

@eriknordmark I did
make live; make run-live on my branch where it has all the kubevirt eve code. Basically compile kvm mode on that branch.
The RSS values of Redbox
linuxkit-525400123456:~# ps -ef | grep zedbox
1432 root      0:09 /opt/zededa/bin/zedbox
3404 root      0:00 grep zedbox
linuxkit-525400123456:~# cat /proc/1432/status | grep -i vm
VmPeak:	  829576 kB
VmSize:	  829576 kB
VmLck:	       0 kB
VmPin:	       0 kB
VmHWM:	  120464 kB
VmRSS:	  116352 kB
VmData:	  101548 kB
VmStk:	     132 kB
VmExe:	   39152 kB
VmLib:	    3048 kB
VmPTE:	     392 kB
VmSwap:	       0 kB
(ns: pillar) linuxkit-525400123456:/# ls -l /opt/zededa/bin/zedbox
-rwxr-xr-x 1 root root 101826600 Jan  8 18:28 /opt/zededa/bin/zedbox
Binary size doubled but RSS does not seem to be an issue.

We already merged code to reserve the group memory for kubevirt case.

But that is for the kubevirt image. Here you are making the RSS for pillar larger for kvm, xen, and kubevirt, right?
Or are you conditionally compiling pillar with different imports?

Or are you conditionally compiling pillar with different imports?

This is the plan, but not yet done.

But that is for the kubevirt image. Here you are making the RSS for pillar larger for kvm, xen, and kubevirt, right?
Or are you conditionally compiling pillar with different imports?

Yes it is conditionally done. Basically we use eve-hv-type.  Let me look for the PR.

Superseded by https://github.com/lf-edge/eve/pull/3730


How to install nightly

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
I install yt-dlp all the time with sudo python3.9 -m pip install --upgrade git+https://github.com/yt-dlp/yt-dlp.git@release
Can somebody tell me how I can install the nightly version with this code pip install .?!
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
No response

https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#update
# To install nightly with pip:
python3 -m pip install -U --pre "yt-dlp[default]"

Hi! I had version 03.12.2024 - now I have version after the command from you 2023.11.16
How come?


Create Order for Exchange from wQubic on Ethereum / Arbitrum to Qubic

Enable users to initiate an exchange order to convert wQubic tokens back to native Qubic tokens on the Qubic network.

Create a hook, a service and integration in client component to handle the create order request


Installed clean Grav, not able to work in Admin Panel

Running on shared hosting with SSH access.
Ran the following comands to install Grav into the public folder:
git clone -b master https://github.com/getgrav/grav.git
composer install --no-dev -o
php bin/grav install

After this I was able to successfully browse the Welcome page "Grav is Running!"
Now installing admin panel:
php bin/gpm install admin

This went without an issue. I was able to browse the admin section of the website, created the admin user, but once logged in into the admin panel using the browser, I constantly get "Fetch Failed" messages, and not able to install any themes or plugins using the Admin Panel UI. The "installing..." window just keeps hanging there forever. Screenshot below

Here is the screenshot of JS console:

Can it be related to not having SSL? (I do not have SSL on the website that I am trying to run).
I would appreciate any advise how to overcome this issue of not being able to install a theme or plugin using Admin Panel UI.

I'ts quite likely that it's SSL related as all the calls to check for GPM status etc require SSL to function.

Thank you Andy for your reply. Is it possible to run grav and admin plugin without having SSL? Or enabling SSL is the only option to solve the issue that I am facing?
I appreciate your help

Thank you Andy for your reply. Is it possible to run Grav and the Admin plugin without having SSL? Or enabling SSL is the only option to solve the issue that I am facing?
I appreciate your help