[Intel MKL] Fixes for unit test failures

Changes in partitioned_function_ops.cc are for passing
Global OpRegistry as default_registry in PartitionedFunction op
This fix addresses failure in MKL layout pass when PartitionedFunction
op calls graph optimization passes. The problem was that the function
library definition that is used to create function graph and corresponding
subgraphs after partitioning did not use global OpRegistry as the
default OpRegistry used for look of operator names. Because of that,
standard operators such as "Const" were not available to graph passes.

Changes in mkl_cpu_allocator.h are to address failure in
mkl_cpu_allocator_test which was expecting that max_bytes_limits is returned
via GetStats() in MKLCPUAllocator.

@tatianashp @penpornk This PR fixes the failures in the MKL CI, thanks.

@nhasabni Thank you for fixing the test failure!
Please fix the formatting issues so that we could get this PR in.

@tatianashp I looked into the failing test -- //tensorflow/python/eager:function_test seems to fail in spite of my fix. Can you confirm?

@tatianashp Sorry, my bad. The test is failing because of my fix. Will fix it.

Sounds good. Looking forward to the fix. Let us know if you need feedback from TF Eager folks.

@tatianashp I think I will need some input to fix the issue. I am not sure if the TensorFlow core API exists for what I am looking for. I think @mrry and @akshayka have worked on relevant piece of code. So if they can comment, that would be great.
The main issue that I was trying to address was: some unit tests fail with MKL graph passes because these graph passes need the definition of standard operators (like Const) in order to rewrite graphs. The definitions of these standard operators are only available in OpRegistry::Global, and the code in partitioned_function_ops.cc was not passing OpRegistry::Global() to the graph passes. That is why my change was to pass OpRegistry::Global() while creating graph in partitioned_function_ops.cc.
But with my change, python:eager_test is failing because it cannot find a symbol add_9n57JfTlx2g (The precise error is graph.cc:379] Non-OK-status: ops_.LookUpOpDef(node->type_str    ing(), &op_def) status: Not found: Op type not registered add_9n57JfTlx2g.
I tried to debug eager_test failure. I see that, by default, partitioned_function_ops.cc passes flib_def and the default_registry() for that flib_def from the graph for the input function while creating a graph that is passed to the graph optimization passes. I feel that my change passes OpRegistry::Global() instead of default_registry() for input flib_def while creating the graph. And I think this is creating the problem in `.
I think the solution should allow us to create a graph using both default_registry from input flib_def and OpRegistry::Global(). But I do not think such API exists.
Thanks.

What I do not understand here is why something is in OpRegistry::Global() and not in default_registry, which should always be a superset of OpRegistry::Global.

If you insist on creating a function library definition with the global registry then you should make sure to register all functions which you need in that new function library definition. Not sure if there's an API to get the list of registered functions, but it should not be hard to get one.

@alextp Thanks for input. Yes, I also did not understand. I thought that default_registry should always include global registry. But that does not seem to be the case. Especially, in nested_function_test of eager:function_test top-level functions flib_defseems to be passed as default_registry to child function.

The problem appeared after https://github.com/tensorflow/tensorflow/commit/eb71a1a3afbbe21407b2149d7adc4efa9e557b24 commit that added graph optimization passes to PartitionedCallOp.
@akshayka - Could you help to figure out what to do here?

@tatianashp Not sure if XLA failure is known? If not, I can debug.

@nhasabni I don't think it has been reported yet. Please feel free to go for it. :)

Sorry, I was wrong. This is known but hasn't been fixed yet. I'll see what we can do.

@nhasabni  Most likely Windows Bazel and XLA failures are not due to your changes since the tests don't use --config=mkl.
@qlzh727  Do you know if the failures above are specific to this PR?

I ran the XLA test without my PR and without MKL. It fails in my setup also.

Let me kick off a new test, from the dashboard, the XLA test is green at the moment, we will see if the new round of test passes or not.

@qlzh727 can you merge this PR? the XLA test is failing in tests:exhaustive_f32_elementwise_op_test_cpu, seems to be unrelated to this PR. Other PRs are also failing this XLA test. Thanks.

@qlzh727 The PR is stuck in "ready to pull". Can you take a look? Thanks and sorry to bother you.

It is failing for the our internal test, and @tatianashp will probably take more actions for it.

@qlzh727 Thanks. @tatianashp let me know how we can help.

In partitioned_function_ops.cc, could you please wrap TF_CHECK_OK around both your AddFunctionLibrary calls (and make sure the formatting is ok)?
Line 105,
TF_CHECK_OK(
            graph.get()->AddFunctionLibrary(global_flib.ToProto()));

Line 261,
TF_CHECK_OK(
          subgraph.get()->AddFunctionLibrary(global_flib.ToProto()));

Thank you!

Thanks @penpornk. Pls take a look.


Missing Inserts for Large Batches - Ebean 3.2.2

I have recently noticed an anomaly running concurrent large batched transactions with Ebean 3.2.2. The basic set up is as follows 3-6 transactions are run concurrently, each transaction inserts 50000 records into the same table and each transaction has a  batch size of 1000. It is important to note that to save each new record I am saving the Beans directly, calling Ebean.save.  About 75% of the time this will run without incident, but every so often when every transaction finishes the database will be missing a couple records (so far I've seen as few as 1 and as many as 10).
I've traced through the code and it seems that in com.avaje.ebeaninternal.server.persist.BatchedBeanHolder.java in the append method there is call to System.identityHashCode and the result of this call is used to determine if the current bean has all ready been added to the persist list. It is my understanding that because of the  large number of objects I am processing there are collisions when System.identityHashCode  is called (http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6809470), causing Ebean to believe it has all ready added a bean to the persist list when in fact it has not.
Currently I am processing my batch inserts using UpdateSql in place of persisting the beans directly to avoid losing any inserts. I am wondering if anyone else has experienced this problem, or is my current implementation a rare use case?

Interesting.  No I have not heard anyone report this issue before. I'll
have a look at it.
Cheers, Rob.
On 20/03/2015 2:01 AM, "evanthomjd"<EMAIL_ADDRESS>wrote:

I have recently noticed an anomaly running concurrent large batched
transactions with Ebean 3.2.2. The basic set up is as follows 3-6
transactions are run concurrently, each transaction inserts 50000 records
into the same table and each transaction has a batch size of 1000. It is
important to note that to save each new record I am saving the Beans
directly, calling Ebean.save. About 75% of the time this will run without
incident, but every so often when every transaction finishes the database
will be missing a couple records (so far I've seen as few as 1 and as many
as 10).
I've traced through the code and it seems that in
com.avaje.ebeaninternal.server.persist.BatchedBeanHolder.java in the
append method there is call to System.identityHashCode and the result
of this call is used to determine if the current bean has all ready been
added to the persist list. It is my understanding that because of the large
number of objects I am processing there are collisions when
System.identityHashCode is called (
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6809470), causing
Ebean to believe it has all ready added a bean to the persist list when in
fact it has not.
Currently I am processing my batch inserts using UpdateSql in place of
persisting the beans directly to avoid losing any inserts. I am wondering
if anyone else has experienced this problem, or is my current
implementation a rare use case?
—
Reply to this email directly or view it on GitHub
https://github.com/ebean-orm/avaje-ebeanorm/issues/258.

batch size of 1000
So yes there is a problem here. If the batch size is large (like 1000) then the chance of this increases. I think this current mechanism needs to change to use an IdentityHashMap -  I'll look to do that.

Fix for this included in the 4.5.6 release.


Adding spans for setup, teardown, and individual fixtures

In test suites with complex setups, teardowns, and fixtures, it's common to see
most of the test runtime happening in those stages rather than in the individual
tests themselves.
In this change, session- and module-scoped fixtures are attributed to the
overall test session, while function-scoped fixtures are attributed to the
setup for an individual test.  This will help with artificial skew from tests
that happen to be the first ones that request a higher-scoped fixture.
I'd welcome feedback on the layout of the spans after some folks have had time
to try it out.  pytest doesn't have a clearly defined session or module
setup stage where these fixtures are run, because they are invoked lazily
the first time a test requests them.  This is ideal for performance, but these
spans will end up kind of "hanging in mid-air" between other test suites.
Depending on the visualization tool (OpenObserve, Jaeger, etc), you may see
those higher-scoped fixture spans showing up in different spots.
Thanks to @drcraig for the idea and @sihil for the cheers!
Closes #25

I like it, this is very similar to how I had implemented it on my branch at https://github.com/drcraig/pytest-opentelemetry/tree/fixture-setup-teardown.
Regarding the higher scoped fixtures being under the session scope; I think it's actually okay for these to be under the scope of the individual test function that first called them.
I was envisioning the spans being organized like so, for a a simple case like
import pytest

@pytest.fixture
def fixt_1():
    yield

@pytest.fixture
def fixt_2():
    yield

def test_foo(fixt_1, fixt_2):
    assert True

[-----------session----------------------------------------------------------------------------------]
[-----------test_func-------------------------------------------------------------------------------]
[-----------test_func setup---------]
 [--fixt_1 setup--]
                   [--fixt_2 setup--]
                                     [--test_func call--]
                                                         [--------------test_func teardown---------]
                                                          [--fixt_2 teardown--]
                                                                               [--fixt_1 teardown--]

Where the "test_func" span is created by pytest_runtest_protocol, "test_func setup" is created by pytest_runtest_setup, "test_func call" is created by pytest_runtest_call, and "test_func teardown" is created by pytest_runtest_teardown. The fixture setup spans come from pytest_fixture_setup, and then the fixture teardown spans call from a combination of pytest_runtest_teardown and pytest_fixture_post_finalizer, which I'll describe in more detail later.
For session scoped fixtures, the spans would look something like the following for these tests. The setup for a shared fixture ends up underneath the span of the first test function that depended on it, and the teardown ends up under the last test function that depended on it.
import pytest

@pytest.fixture(scope="session")
def fixt_1():
    yield

@pytest.fixture
def fixt_2():
    yield

def test_func_1(fixt_1, fixt_2):
    assert True

def test_func_2(fixt_1, fixt_2):
    assert True

[-----------session--------------------------------------------------------------------------------------------------------------------------------------------------------------]
[-----------test_func_1------------------------------------------------------------------]
[---------test_func_1 setup---------]
 [--fixt_1 setup--]
                   [--fixt_2 setup--]
                                     [--test_func_1 call --]
                                                            [--test_func_1 teardown-]
                                                             [--fixt_2 teardown-----]
                                                                                [---------------------------------test_func_2---------------------------------------------]
                                                                                [--test_func_2 setup--]
                                                                                [--test_func_2 call--]
                                                                                [----------test_func_2 teardown-- ---------]
                                                                                [--fixt_2 teardown--]
                                                                                [--fixt_1 teardown--]

The way to capture the teardown of an individual fixture is to create a presumptive span in pytest_runtest_teardown, just before yielding. You don't know which fixture it will be that will get torn down, so we give it a temporary name, and then update the name and attributes to the actual fixture in pytest_fixture_post_finalizer. After the post_finalizer yield, we close the span, and create a new presumptive span for the next fixture in line (which may not exist if this is the last fixture, but we don't know that yet). Finally, after the yield in pytest_runtest_teardown, when we know all the fixtures are torn down, we delete the last presumptive span, since it wasn't used. When a fixture is scoped higher than a function, then we can rely on FixtureDef.finish() having cleared the cached result so that we don't end up with spans for the fixture for every function that depended on it.
It ends up looking something like
    @pytest.hookimpl(hookwrapper=True)
    def pytest_runtest_teardown(self, item: Item) -> Generator[None, None, None]:
        with tracer.start_as_current_span(
            'teardown',
            attributes=self._attributes_from_item(item),
        ):
            self._fixture_teardown_span = tracer.start_span("presumptive fixture teardown")
            yield
        del self._fixture_teardown_span

    @pytest.hookimpl(hookwrapper=True)
    def pytest_fixture_post_finalizer(
        self, fixturedef: FixtureDef, request: SubRequest
    ) -> Generator[None, None, None]:
        # If the fixture has already been torn down (such as a module or session scoped fixture),
        # or we otherwise don't have a span to work with, create a new span for the next fixture in line.
        if fixturedef.cached_result is None or not hasattr(self, "_fixture_teardown_span"):
            yield
            self._fixture_teardown_span = tracer.start_span("presumptive fixture teardown")
            return

        name = ...
        attributes = ...
        self._fixture_teardown_span.update_name(name)
        self._fixture_teardown_span.set_attributes(attributes)
        yield
        self._fixture_teardown_span.end()
        self._fixture_teardown_span = tracer.start_span("presumptive fixture teardown")

Here's what it would look like for multi-file set of tests with module and session level fixtures.

One final question: what would you think of using item.nodeid as the name of the span instead of item.name? I know it's longer, but it's also unique, and handles things like parametrization. That's how test results are presented in the reports generated by pytest-html, and it's easy to read in my experience.

Here's that exact same test report, but with nodeid as the name of the test, setup, call, and teardown spans. Makes it very clear what everything is.

Thanks for contributor status! I'll try to get a PR up by the end of the week (maybe sooner, if I can swing the time) for the presumptive span and nodeid changes.
The floating spans are tough, and totally agree there's not an obvious right answer. What nudges me to keeping it under the testcase, rather than floating, is that it's consistent with how pytest handles the accounting in other contexts. For example, pytest-html will include the logs of the module and session scoped fixtures in the teardown logs for the last tests that use them.
I think there is still a way to get some consistency out of test runtimes, regardless of the fixtures, through the "call" span provided by pytest_runtest_call. This would be only the duration of the test execution itself, and not include fixture setup or teardown.


fixed video_encode_values_array options

Dear @The1only , in strings.xml, we found there is a different number of elements in "video_encode_array" and "video_encode_values_array", which may cause setting issue. This PR added an item "0" in video_encode_values_array for the item "Very Slow" in video_encode_array.
Please review it and let me know if it is correct.
Best,
-- Luke

Thank you accepted.


Yarn run example error Ubuntu

Hi!
I keep getting this error when attempting to "yarn run example <example_name>":
module.js:471
throw err;
^
Error: Cannot find module 'shelljs/global'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object. (/home/sead/Desktop/ami/ami/scripts/router.js:1:63)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
I'm running Ubuntu 14.04 64 bit.
I installed shelljs - "sudo npm install -g shelljs" successfully.
Any help would be much appreciated!!!!

Can you please provide all the commands you typed to install AMI?

Googling around, it may be at NPM misconfiguration on your machine.

You shouldn't have to install "shelljs", it is already part of the NPM dependencies (https://github.com/FNNDSC/ami/blob/dev/package.json#L89)

The commands are as follows, after I installed node and yarn :
git clone https://github.com/FNNDSC/ami.git
cd ami
yarn run example geometries_slices
On Mon, Jul 10, 2017 at 4:49 PM, Nicolas Rannou<EMAIL_ADDRESS>wrote:

You shouldn't have to install "shelljs", it is already part of the NPM
dependencies (https://github.com/FNNDSC/ami/blob/dev/package.json#L89)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/FNNDSC/ami/issues/164#issuecomment-314234433, or mute
the thread
https://github.com/notifications/unsubscribe-auth/APKQb-_JBbeEQim_iK9a-0WFYY_j97rEks5sMo5NgaJpZM4OS6bD
.

And I figured that shelljs came packaged, I couldn't find any way to remedy
it though.
On Tue, Jul 11, 2017 at 1:18 PM, Sead Niksic<EMAIL_ADDRESS>wrote:

The commands are as follows, after I installed node and yarn :
git clone https://github.com/FNNDSC/ami.git
cd ami
yarn run example geometries_slices
On Mon, Jul 10, 2017 at 4:49 PM, Nicolas Rannou<EMAIL_ADDRESS>wrote:

You shouldn't have to install "shelljs", it is already part of the NPM
dependencies (https://github.com/FNNDSC/ami/blob/dev/package.json#L89)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/FNNDSC/ami/issues/164#issuecomment-314234433, or mute
the thread
https://github.com/notifications/unsubscribe-auth/APKQb-_JBbeEQim_iK9a-0WFYY_j97rEks5sMo5NgaJpZM4OS6bD
.

I'm running Ubuntu in a virtual machine, could that be an issue?
On Tue, Jul 11, 2017 at 1:19 PM, Sead Niksic<EMAIL_ADDRESS>wrote:

And I figured that shelljs came packaged, I couldn't find any way to
remedy it though.
On Tue, Jul 11, 2017 at 1:18 PM, Sead Niksic<EMAIL_ADDRESS>wrote:

The commands are as follows, after I installed node and yarn :
git clone https://github.com/FNNDSC/ami.git
cd ami
yarn run example geometries_slices
On Mon, Jul 10, 2017 at 4:49 PM, Nicolas Rannou<EMAIL_ADDRESS>
wrote:

You shouldn't have to install "shelljs", it is already part of the NPM
dependencies (https://github.com/FNNDSC/ami/blob/dev/package.json#L89)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/FNNDSC/ami/issues/164#issuecomment-314234433, or mute
the thread
https://github.com/notifications/unsubscribe-auth/APKQb-_JBbeEQim_iK9a-0WFYY_j97rEks5sMo5NgaJpZM4OS6bD
.

@Doomedfajita , it seems you are missing one step during installation:
$> git clone https://github.com/FNNDSC/ami.git
$> cd ami
$> yarn run example geometries_slices

I'm sorry to be a bother - which step would that be?   It seems as though
they are the same.
On Jul 12, 2017 6:30 AM, "Nicolas Rannou"<EMAIL_ADDRESS>wrote:

@Doomedfajita https://github.com/doomedfajita , it seems you are
missing one step during installation:
$> git clone https://github.com/FNNDSC/ami.git
$> cd ami
$> yarn run example geometries_slices
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/FNNDSC/ami/issues/164#issuecomment-314721836, or mute
the thread
https://github.com/notifications/unsubscribe-auth/APKQb5bRpr8sCKaFU0WAbh-Bc5F9tDqFks5sNKAygaJpZM4OS6bD
.

You didn't run 'yarn install --ignore-optional'

I figured it out!  Thank you for all your help.  I originally had included
the "yarn install --ignore-optional", but I didn't realize yarn was using
an older version (0.22).  It kept throwing an error about --ignore-option
not a valid command.  I updated it to 0.275 manually, and then used the
instructions and it worked!
On Wed, Jul 12, 2017 at 10:08 AM, Nicolas Rannou<EMAIL_ADDRESS>wrote:

You didn't run 'yarn install --ignore-optional'
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/FNNDSC/ami/issues/164#issuecomment-314780538, or mute
the thread
https://github.com/notifications/unsubscribe-auth/APKQb7pEezyBk5pDipkOFKKQEG_fXEoOks5sNNNfgaJpZM4OS6bD
.


Migrate package @mui/x-data-grid from v6 to v7

Description
Dependent package @mui/x-data-grid has been upgraded to major version 7 with breaking changes. Fix the issues breaking the build so we can build the library with the latest features and bugfixes.
Acceptance Criteria

[ ] package.json contains dependency to 7.x version of the package and compiles successfully

Additional Information
Official migration guide: https://mui.com/x/migration/migration-data-grid-v6/

topic seems to be have a big impact on our environment . therefor the estimation needs to be updated. as v6 is still supported we could do this in a later stage too.

v6 is still supported - here the official support map: https://v6.mui.com/x/introduction/support/#supported-versions

@jjeroch just to let you know. v8 is not out yet. we still have 7.22.1  till today
<EMAIL_ADDRESS>@oyo are we planning anything in this regards?


UI: Deleting the volume style requires adjustment, when the name is longer

Environment:
Longhorn version: v0.3.0
Steps to Reproduce:
Delete a volume with name is a longer number.
Results:
Name display is out of border.

This issue has been fixed.


add parallelization to queryToCSV

added threads to csv generation. in my performance testing with 10mb files it was 2-3x faster with just two threads.
row order is not guaranteed! speed is more important than order for my use case.

tested on CF2021 HotFix 11

Thanks for the PR. I considered making query2csv() run with parallel threads a while ago but was put off by the issue with ordering. But making it an option seems reasonable.
There are a couple of issues though:

Parallel iteration is only supported in Lucee and ACF2021+. The library currently still supports ACF2016 and 2018, so I've had to refactor your code to fail gracefully on those older versions.
For some reason I can't get a unit test of the parallel option to work with ACF2021/23 when running the whole test suite. In isolation it's fine but Testbox starts behaving weirdly when running that test as part of the whole suite. I've left the test in but disabled it.
Although Lucee runs the tests fine, it can also behave oddly with parallel loops depending on how long each iteration takes. I will need to make it clear in the docs that the option should be used with care and at the developer's own risk!

Please could you give the develop branch a try to make sure it's doing what you expect and report back?
Thanks.

I'll test on both tomorrow morning and let you know! Thanks!


export api getStore

Coverage remained the same at 100.0% when pulling 4e84738fde1a809424bc74a56fc63f031ff8f0fe on chenkaiC4:export_api_getStore into 72f8630f0ab5a0950924d09d59f0f0c931d667ab on mirrorjs:master.

i think a simple state manager is better than including render and router. so i close this pr, and follow this repo to write a tool just for state manage with smaller api.


Attempt to fix the remote install path resolving

Part of #141599

Looks like this got part of the way there as the server dir is now resolved:

Remaining problems:

[ ] Fix path separators for remote OS
[ ] Bundle the shell integration scripts with the server


do the printer has glass between lcd? and did u remove it?

great work!
i am plannig to buy the printer specifically for pcb exposure, did u remove the glass if there is any? also do u recommend anything else?

Thanks!
I did not modify my Anycubic Photon Mono 4, the glass and protective LCD film are in place. 23-25 seconds is enough to expose the noname dry film photoresist from Aliexpress and 0.1-0.12mm resolution (two traces between SOIC pads) is achievable.


Update vitrimer, analysis, and reaction protocols and add checkpointing to reaction

Update Chapter 3 of thesis code
Summary

Copy the updated scripts from Imperial HPC to the protocols folder
Add /pbs folder with job scripts and initialise_pbs.sh
Add /slurm folder with job scripts and initialise_slurm.sh
Add option --checkpoint [fname] to allow the loading of positions from a checkpoint (this may be a good additional feature in the code/reader section)

The --checkpoint [fname] should probably be implemented in the lowest level Reader and should set a flag in its __init__, then when reading positions, if checkpoint != None then positions should be read from the checkpoint - the alternative is just to create a copy of the system, and then load the checkpoint in - this actually may be easier

This would mean that the System class should have a clone method which returns a pre-simulation version of itself, to which a set of positions from a checkpoint can be added through built-in ReaDDy functionality.

Finished basic services as protocols - need to add checkpointing options

Checkpointing complete


ssr error instanceof Error is false

Version
2.3.0
Reproduction link
https://jsbin.com/hisubahepa/1/edit?html,js,output
Steps to reproduce
I use vue-server-render, found out vm.runNewContext result: error instanceof is Error === false
Then cause koa error，https://github.com/koajs/koa/blob/master/lib/context.js#L107
I think we should add Error at: https://github.com/vuejs/vue/blob/dev/src/server/bundle-renderer/create-bundle-runner.js#L18
What is expected?
error instanceof Error === true
What is actually happening?
error instanceof Error === false

This is a known behavior in SSR. You can turn off runInNewContext to correct this. (Personally I recommend to do so for any non-trivial project for performance.)
Please refer to official SSR guide. https://ssr.vuejs.org/en/bundle-renderer.html
I wonder whether the fix is appropriate. Since patching builtin will never be enough until we copy the whole context. But this breaks our original point for runInNewContext.

@HerringtonDarkholme
In my case, runInNewContext is false, The real problem is that the vm.runInnewContext output of the Error is a different thing, just like iframe.window.arr instanceof Array === false

@lonelyclick this will be fixed in the next version by using runInThisContext instead with runInNewContext: false. The current 2.3.0 behavior (evaluate the bundle once in a separate context) will be available under runInNewContext: 'once'.

runInThisContext is significantly faster! Looking forward to the new release!


feat(webpackDevServer): Add watchOptions for webpackDevServer

Add watchOptions to webpackDevServerConfiguration to conditionally enable polling option in watchpack
Remove additional blank lines from end of serve-watchpack.ts so that only one is remaining

In cases where Angular CLI is running in a shared directory on linux VM on a windows host the webpack dev server isn't detecting file changes from the host environment. (ex: Whenever a docker dev environment is running on a windows host.)
This is solved by adding poll option to the webpack dev server configuration. By default when no poll interval is set watchpack will use native file system change detection. This change allows for setting the poll interval which will turn on polling.
With this change the following can be added to the defaults object in  angular-cli.json to change the default watch options:
"watchOptions": {
  "poll": 1000
}

When watchOptions isn't included in angular-cli.json there will be no change, and native file system change detection will be used.
Here is the link to the webpack docks regarding watchOptions, and the options that are available:
https://webpack.github.io/docs/configuration.html#watchoptions-aggregatetimeout
Note: Also removed 4 out of the 5 blank lines at the end of the file that I'm assuming shouldn't be there.

Minor edit. instead of setting entire watchOptions in angular-cli.json, only set poll

rebased to master

restarting travis

Hi All,
The recent changes to config broke the change I made. I'm looking into it.

You need to update the schema JSON, not the d.ts. We're talking internally as to whether or not this change is wanted.

@hansl should it be in both or just the schema?
As stated previously, without the change there's no direct way to have webpack detect changes on shared directories between linux/windows via a VM such as in docker development environments. With the common "poll" flag option in defaults it can be applied to other addons which may want to conditionally add polling for detecting file changes.
Please let me know if anything else is needed.

Another note on polling, this may also resolve the watch issues for users developing in Window's 10's Ubuntu Usermode, I haven't tested that myself though.

I've updated this PR to reflect the typing decision made in https://github.com/angular/angular-cli/commit/a163186a703ab84926e80bad037422ce619231c9 to define the poll as only being a number and not also being boolean.

Having this will also allow docker users to setup the toolchain and support the watched building. I'd love to see this get merged.

FYI, now testing with docker on OSX. I'm having the same polling issues that I was having on Windows. So basically for anyone wanting to set up a docker dev environment using angular-cli will require polling in order for the watch tasks (serve/test) to detect file changes.

Can we please have this?
Development environments under virtualbox have problems propagating file system events.

I've been doing my best to keep this branch up-to-date with master.
https://github.com/angular/angular-cli/pull/1837 will also be required to properly integrate polling when running tests.

@hansl can you review? I've merged https://github.com/angular/angular-cli/pull/1837 already so that's not needed anymore, but this PR introduces changes in angular-cli.json.

+1 this feature would be crutial

Can we please get this into the next release?

LGTM. Looking for the tests to pass.

All good, just rebased with master to keep things clean for the pull. Thank you for taking the time to review this :)

Without documentation, I don't know how to ignore "node_modules" directory. Can anyone tell me how to?
https://webpack.js.org/configuration/watch/#watchoptions-ignored

@monad98 Currently there's no configuration options for this, the cli's webpack config would need to be updated to allow for setting this. https://github.com/angular/angular-cli/blob/master/packages/angular-cli/tasks/serve-webpack.ts#L100
A separate issue/PR should be created for this. And @hansl should weigh in on if there should just be a default to ignore node_modules or if this is something that should be configurable by the end-user.

Apologies for digging up this older PR but it's the first one with a comment that's my exact issue.
@monad98

Without documentation, I don't know how to ignore node_modules directory. Can anyone tell me how to? https://webpack.js.org/configuration/watch/#watchoptions-ignored

I'm running a Vagrant setup for a few things and consistently, if I use poll for webpack, I need to exclude node_modules or else it maxes out the CPU and over time will crash the VM as it eats up RAM.
When it comes to ng build, if I put --poll=1000 it sits at max CPU usage. Excluding node_modules has been the solution to this in the past for other Webpack setups. Personally, if node_modules changes, it'd be reasonable to restart the ng serve to ensure it's all set up properly.
@monad98 Were you ever able to resolve ignoring node_modules?


Add node to working cluster

Closes #783

Coverage decreased (-0.2%) to 61.349% when pulling a1d8f91b79065c69136b9e96bc57d4b81e22048d on stgleb:add_node_aws into 8b879b4be3ea52a1dfa8eeae5909b79dd40ca7fc on supergiant:master.

It look the same as https://github.com/supergiant/supergiant/pull/889

@saheienko It used to look so before I merge #889 Now it is separate


markdown-it-shiki-twoslash doesn't render code examples

I tried to swap out the default Shiki markdown support from Vitesse to use shiki twoslash. It worked well for my dev server, but when I went to build for prod, I realized it wasn't working. The async nature of the Markdownit/remark interop was causing a race condition to occur that was covered up by my build's HMR. 😢
Since it was a race condition, I was pretty sure it had something to do with Markdown It's need for sync and Remark's need for async being incompatible. Because the MarkdownIt and Remark interop relies on deasync I made a go at swapping that out for synckit which uses worker threads, instead of the (seemingly broken) deasync hack. I wasn't able to get very far.
Can someone make sure this plugin is still working?
Node: v14.17.4
OSX Monterey 12.0.1

I think you're between a rock and a hard place, shiki can't go sync and markdown-it won't support async calls


bugfix: schema model index validation

this PR makes sure that MultiIndex schema name order must always match the validated object's multiindex names. It also handles None-named indexes by replacing the name with the index integer position.

This should address the bug in error-reporting discussed in #357


Update Flux version to 1.5.0

Looking at the amount of removals I am not not entirely sure if Gopkg.toml and Gopkg.lock were in sync but as the Gopkg.toml should be the leader I made the required changes there.
Diff here looks different from what I got the first time on local :thinking:. Everything should be fine.

Thanks! I’ll run this on CI and merge this tonight. Also created #4 to remove toil here.


Weird white popping at some angles

Godot version
3.3.4.stable.mono
System information
Windows 10, GLES3, Intel UHD Graphics 600 (<IP_ADDRESS>81)
Issue description
https://user-images.githubusercontent.com/82517019/145540474-ea05b98d-bda6-45d0-b855-f8281a14dc75.mp4
Steps to reproduce
I have two seperate house scenes, and either one has the walls white while the other one is okay and vice versa.
Minimal reproduction project
No response

@scrscrnougat
Please upload a minimal reproduction project to make this easier to troubleshoot.

You can also try to update your Intel drivers (from the Intel website directly, not your OEM) to the latest version: https://www.intel.com/content/www/us/en/download/19344/intel-graphics-windows-dch-drivers.html

Actually, it was just a material issue and I fixed it.

Actually, it was just a material issue and I fixed it. (If only I can delete it...)

Could you describe the fix you applied on your project? It may be helpful to others stumbling upon this issue :slightly_smiling_face:


Supporting intersecting with multiple node labels

This issue is closely related to issue #297. Once extend supports multiple destination node labels. Intersect could also potentially gets two nodeIDVector with different labels and thus should sort with label&offset and comparing label&offset.

Currently, intersect does takes node table id into consideration. Tests can be added when we support extending to multiple dest node tables.


Ability to both pause and resume a WASM instance

At the moment, as far as I can tell, there's no way to temporarily pause execution of WASM only to re-enter it later, which is vital for running multiple WASM instances on the same thread without one being able to stall all of the others. Is this even on WASMTime's radar, or would I be better suited finding a WASM interpreter (or designing my own) to use for this purpose instead?

Thanks for the report! This is not yet possible but would be covered by https://github.com/bytecodealliance/wasmtime/issues/860, so I'm going to close in favor of that.


How to avoid adding extra 2 pages after 4 page

When i click on 4 page another option is added and i lose my mouse can't click since the paggination is widher now
when I click on 5 page i got another option added.
How to avoid that , and keep static widht and size , no adding extra pages ?

guess pageRange: null can solve your problem


[Feat] implement GC for S3 storage

At the moment GC is only supported on local storage because it depends on the umoci library. We should also implement GC for S3.

We should really just pull in or implement our own GC code just as the sync code.

Should be fixed with https://github.com/project-zot/zot/pull/1608

Done in https://github.com/project-zot/zot/pull/1656


Style/NestedParenthesizedCalls rspec whitelist is non-exhaustive

RuboCop triggers a violation on some rspec methods that are left out of the list of rspec matchers. In my case, this specifically occurs with be_an:
FNG145099:tmp nathan$ cat test.rb
should((be nil).or(be_an Integer))
FNG145099:tmp nathan$ rubocop -V
0.38.0 (using Parser <IP_ADDRESS>, running on ruby 2.1.5 x86_64-darwin14.0)
FNG145099:tmp nathan$ rubocop
Inspecting 1 file
C

Offenses:

test.rb:1:20: C: Style/NestedParenthesizedCalls: Add parentheses to nested method call be_an Integer.
should((be nil).or(be_an Integer))
                   ^^^^^^^^^^^^^

1 file inspected, 1 offense detected

I agree in principle, but by including some rspec matchers, it seems rubocop is somewhat dependent on rspec already, but still produces inconsistent behavior depending on which rspec matchers you're using. Yes, you can just disable the cop if you run into that issue, but I feel that you're sweeping the problem under the rug at that point. Perhaps it's better to remove any rspec dependent logic from the core of rubocop, and put that logic in an rspec-specific cop, e.g. Rubocop-Rspec.

Why do you think that is RuboCop dependent on rspec?
Probably it'd be best to drop this functionality and just leave it to people to configure the cop. Hardcoded config is bad, but the option to add/alter config is great.

Maybe "dependent" isn't exactly the word, but the correctness of this cop relies on the current state of Rspec. I agree though, having some ability to configure this whitelist would be super helpful.


Update docs

Add cli doc for recent change , #14 / #22
Check that docs for #20 / #21 is in place
Improve Identifyer Style section , for #11

I wonder if there is a way to preview the docs local, especially when playing around with mkdocs specials, like the
!!! note section

I wonder if there is a way to preview the docs local, especially when playing around with mkdocs specials, like the
!!! note section

I wonder if there is a way to preview the docs local, especially when playing around with mkdocs specials, like the
!!! note section

I've used Python mkdocs locally before on a project.  That project was hosted on a local Gitlab instance (not github) so I spent some time ensuring my MarkDown was compatible and rendered as intended with both the Gitlab and mkdocs rendering.

I wonder if there is a way to preview the docs local, especially when playing around with mkdocs specials, like the
!!! note section

I've used Python mkdocs locally before on a project.  That project was hosted on a local Gitlab instance (not github) so I spent some time ensuring my MarkDown was compatible and rendered as intended with both the Gitlab and mkdocs rendering.

you can preview the docs by running mkdocs serve.
This will render the docs with the default mkdocs theme.
The admonition blocks (!!! note) require a special plugin to be activated.
I opened an issue to improve the preview-ability in each repository. Thanks for your feedback, @a4z

you can preview the docs by running mkdocs serve.
This will render the docs with the default mkdocs theme.
The admonition blocks (!!! note) require a special plugin to be activated.
I opened an issue to improve the preview-ability in each repository. Thanks for your feedback, @a4z


fix(button): Отключение hover-эффекта на тач девайсах

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 80.686%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
7309

Relevant Lines:
8259

💛  - Coveralls

Фиксит сразу все темы?

Фиксит сразу все темы?

Да


Reconsider supported draining behaviour

For Unix (kqueue and epoll) to drain readiness all you have to do is read/write until you get a "short" read/write, i.e. less bytes are processed than your buffer size. However Mio doesn't guarantee that another event is return until the I/O operations hits a WouldBlock error. This means that to strictly follow Mio's docs you'll have to issue another pointless system call (read/write/etc.).
This is because on Windows Mio could only guarantee the readiness to be drained once a WouldBlock error was returned. However nowadays we control that ourselves in IoSourceState::do_io.  So, we could change it to reregister when e.g. the read bytes < buffer size.
Also find out if all of this is true for poll (see https://github.com/tokio-rs/mio/pull/1602).

Also find out if all of this is true for poll (see https://github.com/tokio-rs/mio/pull/1602).

If I understand correctly, it is. poll is level-based by default anyway, so we have full control over when we want to re-arm an edge trigger. https://github.com/tokio-rs/mio/pull/1602 currently does not implement this, but since for poll this solves other issues as well, I'm working on implement edge-based triggers using IoSourceState.

This can be a pretty impactful observation in certain workloads.

One thing we need to consider are the try_io functions on various types, e.g. TcpStream::try_io, which currently don't give us access to the amount of bytes read.

I don't think we can change the behaviour any more. The public try_io method doesn't have access to the internal success type of the function it's wrapping, nor could it know if e.g. a read was short. Furthermore I don't this working with the poll(2) implementation.
Based on that I'm going to close this and keep the current behaviour.
If someone does have an idea on how to support try_io and the poll(2) we can reopen this.


[BUG]: CLEANUP - Admin | Sync Template v3 - Two flows missing connectors

Describe the bug
Two flows appear to have missing connectors.
Component (please tell us which flow or app you are experiencing issues with):

Solution: Core Components
Version: 1.98
App or Flow:
CLEANUP - Admin | Sync Template v3 (Connection Status)
CLEANUP - Admin | Sync Template v3 (Orphaned Makers)

To Reproduce
Steps to reproduce the behavior:

Go to 'Solutions > Center of Excellence - Core Components'
Click on either of the impacted flows.
Review the screenshots.

Expected behavior
Required connections should be configured.
Screenshots

Additional context
This seems to be similar to report posted earlier.

Hello. this was fixed in this version of the toolkit, can you validate, did you follow the steps in installing updates when you upgraded? If not, then you may be blocked from seeing updates by managed layers.

Strange, I checked it later and the connectors are showing up now and I could start the flows.  So I guess this is a non issue.


Doesn't render properly in Spark MacOS

Only in MacOS Spark styles seem not to apply, on the iOS app it seems to work fine though...

Interesting. Haven't used it before. Do you know what rendering engine it is using? Is this the only client you're having issues with?

I am afraid I don't know the engine they use on that. However, it seems this is the only client with this issue for me, yes.

I think that this is a problem with Spark for Mac. I have the same issue, but if I press reply or forward, the referenced mail will be styles..

@ViggoV doesn't display for me correctly even if I reply / forward.

@mxmzb Mind sharing a screenshot to help me troubleshoot?

i think this was my bad, it wasn't the inline css version but it was with stylesheets. i didn't try if it works with inline version since then, but i am pretty sure that this was the reason. sorry for that!

@mxmzb No problem thanks for following up


Skip some path in list_file_with_suffix.

Which issue does this PR close?
Closes #1648.
Rationale for this change
We should filter out some path in list_file method,
like
https://github.com/apache/spark/blob/5b91381b9bcea27b7be6d9cef852efbd6c23d98a/core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala#L358
Some question
I am not sure is this situation in same on S3 or other object store?
Should we add a method in struct ObjectReader like should_filter_out

@houqp @andygrove Could you give me some advice?

@houqp  Could please take a look?

Nice work @Ted-Jiang :+1: @Jimexist @alamb you want to take a final look?

Thanks @Ted-Jiang


Cannot open game "unfroggable"

This game:
https://makecode.com/_Rq9cppgu5XHD
does not load. Instead there is a blank screen and these errors in the console:
simulating script
pxtembed.js:100154 upgrading dep tilemaps: github:microsoft/pxt-tilemaps#v1.7.1 -> github:microsoft/pxt-tilemaps#v1.8.1
pxtembed.js:100154 upgraded, downloading again
2pxtembed.js:170057 A namespace declaration cannot be in a different file from a class or function with which it is merged.
(anonymous) @ pxtembed.js:170057
pxtembed.js:170098 Diagnostics Array(2)
0: category: 1code: 2433column: 10fileName: "pxt_modules/tilemaps/overworld.ts"length: 7line: 33messageText: "A namespace declaration cannot be in a different file from a class or function with which it is merged."start: 885__proto__: Object
1: category: 1code: 2433column: 10fileName: "pxt_modules/tilemaps/tilemap.ts"length: 7line: 6messageText: "A namespace declaration cannot be in a different file from a class or function with which it is merged."start: 190__proto__: Objectlength: 2__proto__: Array(0)
(anonymous) @ pxtembed.js:170098
--run:212 simulator started...

Might be related to: https://github.com/microsoft/pxt-tilemaps/commit/a0cf09c901ea519e6aaa3a211ffbc05f88d6d61c
? Any ideas @jwunderl ?

When I open the game in the editor, I see all the custom function names have disappeared, which is strange:

switching to JS:

If I fix all the function names, then the game runs fine.. very strange

Kind of two separate issues here; first is that upgrade rules didn't apply to share page view (not a regression if I recall correctly); second is the weird one. As you noted, it's just dropping the fn name after applying upgrades:
should be

but after loading in live site it's dropping name as you saw / function_name is becoming a self closing tag:

Not really sure where that's coming from / seems unrelated to other upgrade rules? Have to walk through the upgrade rule loading path. @darzu Did you want to look into this one or should I?

Good to know, thanks. I'm in studio recording mode today, but I could pick it up tomorrow or tuesday if you don't want to

Okay, doesn't seem to be an upgrade rule issue; if you fetch the text from the file it matches the state of the live site:

@shakao does this seem like something that could have changed in one of the blockly updates -- it seems like it used to be lenient in v0.15.18 and just take the name from the mutation when the function name is missing / fill it in, but it doesn't now.

ah yeah this is a blockly merge thing i'll look into it

@abchatra fix is in blockly, do you want it for the upcoming release? if so i'll bump pxt-blockly and update here

Yes please. Let’s make it part of tomorrow release.

Still repros for me as of today:

@shakao ?

it should work if you import it into the editor! like joey said, this is really two issues--one is that upgrade rules don't run on the share page, the other was a blockly issue with function names

Ah okay, do we have an issue filed for the running upgrade rules on share page issue? Should this issue be re-named to be about just the blockly issue? If so, I'd propose a new umbrella issue is created because,
I do think it's important that this scenario works without importing otherwise a lot of our old forum links, student projects, etc could be broken. Share URLs are the way lots of users use to track projects esp since we don't have identity.

i think we should make a separate issue for the share page upgrade rules (linking this one/with unfroggable as an example), just to keep things clean!

I renamed this issue and made a new issue for the share page issue.


Can I pre configure a cluster?

Is your feature request related to a problem ?
I am trying to wrap my head around flux and how it works and then once I am there work out what the best installation method is for our situation. Right now this helm chart and the terraform method are both appealing to me because we have an extensive Terraform base.
Most howto's and instructions on how to setup a clusters take you to the CLI, which is something I won't be able to do: example:
https://github.com/fluxcd/flux2-kustomize-helm-example#bootstrap-staging-and-production
flux bootstrap github \
    --context=staging \
    --owner=${GITHUB_USER} \
    --repository=${GITHUB_REPO} \
    --branch=main \
    --personal \
    --path=clusters/staging

Is there a way in this chart to complete this setup?
Describe the solution you'd like.
Some sort of secret that one the chart is applied I can create that creates this config or better yet a secret in the chart.
Describe alternatives you've considered.
so far apart from the CLI method, I have only found this method in the terraform provider.
https://registry.terraform.io/providers/fluxcd/flux/latest/docs/guides/github
https://registry.terraform.io/providers/fluxcd/flux/latest/docs/data-sources/sync
Additional context.
Ohhhhhhhhhhhhhhh, it just dawned on me..... Is that what the flux2-sync chart is for in this repo? Please confirm. Maybe some doco and examples should be created in this repo and you can use my feature request here as a corner stone for an example.

Just a week since you asked and maybe you figured it out by yourself, but your last thought seems to be right. To configure gitrepositories your use the flux2-sync chart.

maybe someone following this journey and this info will help them. The bit that really tricked me is the resources it makes needs to be named flux-system.

Just to follow up on this (I know the issue is closed) - some documentation as what each chart is for would be great. It's really difficult to translate the outcome of the magic of flux bootstrap into these charts.


What's the extraction code for model A - model D with Baidu Cloud disk link?

I clicked the model A link, and it directs to a Baidu Cloud Disk web page. However, it's encrypted, but no password is given.

Password is given right next to the link within the README


执行Python manager.py migrate 出错

I guess you are using a Python < 3.10. Updating would solve the problem.

ok thanks, my python version is 3.9. I will update my version.

has been resolved, thank u


Adjusting help message

NOTE: Originally had #4812 open to merge these changes, but checks were failing in weird ways. @calebcartwright suggested rebasing my branch onto master, but I decided to just delete my fork and start again, and make the changes again on a new feature branch (remove-nightly-help-from-stable). I then merged into the master branch on my fork, and the checks passed there, so hopefully they'll be fine here.

On stable, running with --help|-h shows information about file-lines
which is a nightly-only option. This commit removes all mention of
file-lines from the help message on stable.
There is room for improvement here; perhaps a new struct called, e.g.,
StableOptions could be added to complement the existing
GetOptsOptions struct. StableOptions could have a field for each
field in GetOptsOptions, with each field's value being a bool that
specifies whether or not the option exists on stable. Or is this adding
too much complexity?

There is room for improvement here; perhaps a new struct called, e.g.,
StableOptions could be added to complement the existing
GetOptsOptions struct. StableOptions could have a field for each
field in GetOptsOptions, with each field's value being a bool that
specifies whether or not the option exists on stable. Or is this adding
too much complexity?

The original changes proposed will be quite alright for covering #4812, no worries there. However, if you're interested in making any follow up proposals to improve the handling that'd be fine too! Probably worth noting that one of the many changes on the experimental v2 version actually switched over to using structopt (https://github.com/rust-lang/rustfmt/blob/rustfmt-2.0.0-rc.2/src/rustfmt/main.rs) so perhaps worth considering doing the same on the 1.x mainline

Also for future reference I want to try to avoid merge commits as much as possible. They're strictly necessary for the sync process we use to get rustfmt pulled into the main repo for release, but outside those subtree push/pulls we shouldn't have them.
If you end up willing and interested in working on anything else it'd be best to submit the pulls from feature branches that have are up to date/rebased against the trunk branch here. Our branching strategy has been a little funky here lately too, so don't hesitate to reach out here and/or in GitHub if you have any questions!


If config does not exist, Wireframe folder is created in incorrect location

If a config does not exist and ./privateer sally wireframe is run, output files are placed in ./Wireframe instead of ./test_output/Wireframe

This is becoming pretty cumbersome, but I haven't had a moment to crack it open yet.


Generated copyWith() for nested lists doesn't keep its inner child values

Describe the bug
When creating models that contains nested lists, the code generation for copyWith() doesn't keep track of the inner child. Instead, an empty list is returned. The toJson() works as expected.
To Reproduce
Steps to reproduce the behavior:

Create a class like

class: NestedLists
fields:
  doubleValue: double
  listDouble: List<double>
  listListDouble: List<List<double>>

run serverpod generate
Create the following tests

import 'dart:convert';

import 'package:example_server/src/generated/protocol.dart';
import 'package:test/test.dart';

void main() {
  NestedLists baseObject() {
    return NestedLists(doubleValue: 0, listDouble: [
      1
    ], listListDouble: [
      [2]
    ]);
  }

  group('NestedLists', () {
    test('toJson() returns a valid json for nested lists.', () {
      var obj = baseObject();
      var expectedJson = jsonDecode(
          '{"doubleValue": 0.0, "listDouble": [1.0], "listListDouble": [[2.0]]}');
      expect(obj.toJson(), equals(jsonDecode(expectedJson)));
    });
    test('copyWith() does return the same values.', () {
      var obj = baseObject();
      var expectedObj = baseObject();
      expect(obj.copyWith(), expectedObj);
    });
  });
}

Run the tests
See error

Expected behavior
Expected: _NestedListsImpl:<{"doubleValue":0.0,"listDouble":[1.0],"listListDouble":[[2.0]]}>
  Actual: _NestedListsImpl:<{"doubleValue":0.0,"listDouble":[1.0],"listListDouble":[]}>

package:matcher                                          expect
package:example_server/src/test/nested_lists_test.dart 25:7  main.<fn>.<fn>

Serverpod versions

CLI Version: Serverpod version: 1.2.6
Library version: serverpod: 1.2.6

Platform information
Ubuntu 22.04.4 LTS through Docker.

Thank you for the detailed report!

I'll pick this one up to work on (I don't see a linked PR ... if anyone is already working this one, please let me know).

Go for it @bfemmer


Enable Cloud 7

+1

The -suse- job shouldn't be templatized. I'm also unsure whether we'll be able to share mkcloud parameters accross cloud versions. the nodenumbers are typically different..

It should be no problem to templatize the mkcloud parameters. When we need different parameter between the cloud versions we could add just another variable. It is also no problem to set variables per job instead of a global variable.

@rsalevsky good point. Looks actually rather nice now.. thanks!

@dirkmueller @toabctl I found a problem in JJB this causes that disabled is always true. Any Idea how to continue?
See https://review.openstack.org/#/c/287257/

@rsalevsky maybe you can remove the disabled field. The semantic of this field is: if is not there, is enabled, if is there, is disabled. So disabled: false is really telling you to disable it.

@aplanas When I remove it then the all tests for Cloud7 are enabled which is not yet wanted.

@rsalevsky actually I think we want to enable the cloud 7 tests...

@rsalevsky thanks for helping to setup jbb. The bug is in the parser and the fix here is:

disabled: '' will generate <disabled>false</disabled> in the XML
disabled: true will generate <disabled>true</disabled> in the XML

true can be any string, like false, bananas or None, this will generate true in the XML. Only the empty string  '' will generate false

@rsalevsky one more thing - the mkcloud6-suse.yaml job should be deleted. maybe you can simply rename it to the mkcloud7-suse.yaml ?

+1


Deobfuscator: full package name collides with class name

Hi, looks like option --no-imports doesn't resolve all possible problems. Download https://www.apkmirror.com/apk/tumblr-inc/tumblr/tumblr-11-4-0-02-release/, enable deobfuscation and open com.tumblr.creation.receiver.ShareActivity:
public class ShareActivity extends com.oath.o2.android.vrmsdk.kqe {

Now open com.oath.o2.android.vrmsdk.kqe:
public abstract class kqe extends com.oath.o2.android.vrmsdk.krt {

if you try to compile it, com.oath cannot be resolved to a type error will be given to you in kqe class. It took a several minutes to realize why it happens. The problem is in class com.oath.o2.android.vrmsdk.com which collides with package name. Is that possible to don't generate deobfuscated names which match with any package names at all to prevent such issues in the future? Thanks!

@sergey-wowwow can you check the latest commit? I add class renaming if name overlaps with any package name, is this fix your issue?


Update GitHub link to Apache org. and suggest new page for "code" (repos).

As we are not yet fully migrated as of May 8th, this can be problematic...
In addition, we need to allow users to find all "openwhisk" labelled repos. under a much larger Apache org. within GitHub.  As supplied by Carlos, the following link (with query) can find them all
https://github.com/apache?utf8=✓&q=openwhisk
I would suggest we create a dedicated page (on top-level, menu with drop-down that takes you to a new page) listing and describing all repos. as well since we lost this "family" feeling under GitHub structure of Apache.

How do we do something like this: https://cwiki.apache.org/confluence/display/OPENWHISK/Subproject+listing
in the Documentation page?  Section "Project structure"

List of repos. (as in Wiki)

Also see ,main repo. README: https://github.com/apache/incubator-openwhisk
Repository Structure
The OpenWhisk system is built from a number of components. The picture below groups the components by their GitHub repos. Please open issues for a component against the appropriate repo (if in doubt just open against the main openwhisk repo).
component/repo mapping

link to https://github.com/apache/incubator-openwhisk/blob/master/docs/dev/modules.md and update to make more current.

Done with PR #299


Regex capability

It would be really useful to be able to wildcard, pattern match etc against some content.
For instance, after a conversation, a customer "bans" the use of AWS ec2, in favour of lambda and fargate.
As there are a lot of ec2 modules (https://docs.ansible.com/ansible/latest/collections/amazon/aws/index.html#modules) it would be tedious and error prone to list out all of them in a policybook. We should be able to do something like this:
# This policybook checks if someone is using automation content that has been banned from use for whatever reason
# That could be for cost control, standards alignment or security reasons
# This will check both FQCN and non-FQCN for module checks
---
- name: General or common AWS policy checks
  hosts: localhost
  vars:
    disallowed_modules:
      - amazon.aws.ec2*
      - community.aws.eks_fargate_profile
  policies:
    - name: Check for disallowed module usage
      target: task
      condition: input._agk.task.module_info.fqcn in disallowed_modules or input._agk.task.module in disallowed_modules
      actions:
        - deny:
            msg: "You are not allowed to use these modules in automation tasks: {{ disallowed_modules }}"
      tags:
        - compliance

Additionally, it would be great to be able to put out the exact details of what's violated the condition in msg:


Threads left hanging after synthesizer is deleted

Speech SDK starts some native thread when speech synthesis is requested. These threads are kept alive even after synthesizer is deleted and garbage collected.
To Reproduce
Steps to reproduce the behavior:

Run the following code:

import os
import subprocess
import azure.cognitiveservices.speech as speechsdk
import threading

def print_thread_count(msg: str = ""):
    sys_th_count = len(subprocess.check_output(["ps", "-T", "-p", str(os.getpid())]).splitlines())
    print(msg, "- active python threads:", threading.active_count(), "; system thread count: ", sys_th_count)

class WriterCallback(speechsdk.audio.PushAudioOutputStreamCallback):
    def write(self, buffer: memoryview) -> int:
        return len(buffer)

    def close(self) -> None:
        pass

def create_synthesizer():
    compl_evt = threading.Event()

    def on_completed(eargs: speechsdk.SpeechSynthesisEventArgs):
        compl_evt.set()

    # This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
    speech_config = speechsdk.SpeechConfig(subscription=os.environ.get('SPEECH_KEY'), region=os.environ.get('SPEECH_REGION'))
    audio_config = speechsdk.audio.AudioOutputConfig(stream=speechsdk.audio.PushAudioOutputStream(WriterCallback()))
    audio_config = speechsdk.audio.AudioOutputConfig(use_default_speaker=True)

    # The neural multilingual voice can speak different languages based on the input text.
    speech_config.speech_synthesis_voice_name='en-US-AvaMultilingualNeural'

    speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config, audio_config=audio_config)

    speech_synthesizer.synthesis_completed.connect(on_completed)

    return speech_synthesizer, compl_evt

speech_synthesizer, compl_evt = create_synthesizer()
def do_speech() -> bool:

    print_thread_count("before input")
    print("Enter some text that you want to speak >")
    text = input()

    if text == "":
        print("Exit")
        return False

    print_thread_count("before speak")
    result_future = speech_synthesizer.speak_text_async(text)
    print_thread_count("after speak")

    compl_evt.wait()
    compl_evt.clear()
    print_thread_count("after wait")

    # Uncomment this to fix the issue.
    # result_future.get()

    return True

while do_speech():
    print_thread_count("loop...")

print_thread_count("Cleanup...")
del speech_synthesizer
print_thread_count("After del")
# At this point number of threads will be about +6 higher then before the while loop,
# unless result_future.get is called for each result_future.

Observe that the number of native threads did not decrease at the end of the script (just press enter to exit the script).
Ucomment # result_future.get() and run again.
Observe that the number of native threads did decrease to previous value at the end of the script.

Similar problem can be observed with recognizer too.

Duplicate of https://github.com/Azure-Samples/cognitive-services-speech-sdk/issues/2506


Add IDE docs

 Issue by abyrd
Thursday Jan 18, 2018 at 20:35 GMT
Originally opened as https://github.com/catalogueglobal/datatools-ui/pull/63

When working on the datatools-server, sometimes we want to have dependencies like gtfs-lib open in the same IDE so we can make concurrent changes to them. This explains how to get that set up.

abyrd included the following code: https://github.com/catalogueglobal/datatools-ui/pull/63/commits

 Comment by codecov-io
Thursday Jan 18, 2018 at 20:38 GMT

Codecov Report

Merging #63 into dev will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##             dev     #63   +/-   ##
=====================================
  Coverage   1.33%   1.33%
=====================================
  Files        318     318
  Lines      10647   10647
  Branches    3670    3670
=====================================
  Hits         142     142
  Misses      8639    8639
  Partials    1866    1866

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 067f619...b939452. Read the comment docs.

 Comment by landonreed
Thursday Jan 18, 2018 at 20:52 GMT

@abyrd, I was thinking that it might be helpful to mention the first few steps of the Deployment page near the beginning of the Development page so that developers know how to set up the databases, config files, and Auth0.  Thoughts?

 Comment by abyrd
Tuesday Jan 23, 2018 at 12:59 GMT

@landonreed if those instructions are identical for the two cases we should avoid duplication. I think once someone has a development environment set up, we'd just refer them to the deployment page with a note saying that of course you need to set up locally just as you would a remote deployment.

 Comment by abyrd
Tuesday Jan 23, 2018 at 13:00 GMT

Since I've got no other comments / requests on this I'm going to merge it so it'll show up on readthedocs, and I can share it with Zak.

 Comment by abyrd
Tuesday Jan 23, 2018 at 13:02 GMT

For additional small updates to the docs I might just edit them directly in the Github web UI. The main reason we'd include them in a PR is to commit docs and code changes together, or if there's some doubt about the content.

:tada: This issue has been resolved in version 4.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Scale fonts for device

Text in buttons is too small

Device: 5.1" Phone

Fonts were embiggened in 36601a7. Good now?

No complaints specific to buttons: closing.


UPD not clear enough

In the section "Step 3: Convert the UPDs to FSLogix profile containers"  it doesn't say where these commands are run from.
Can you clarify in the doc if you are running these commands on the legacy on-prem RDS estate or the new AVD host pools?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 177be313-669c-b534-575c-27eaa49b76b8
Version Independent ID: 33169923-9568-077a-7f12-19554a9dbb94
Content: Move on-premises Remote Desktop Services to Azure Virtual Desktop scenario - Cloud Adoption Framework
Content Source: docs/migrate/azure-best-practices/contoso-migration-rds-to-wvd.md
Service: cloud-adoption-framework
Sub-service: migrate
GitHub Login: @benstegink
Microsoft Alias: abuck

Thanks for reporting - this issue is under review. This is a Microsoft Internal DevOps Bug ID # 466659

The contoso scenario is an example, not a how-to article.  If you have a suggestion for how to make this content clearer, please let us know.  Thank you.

Closing this issue.  Thanks for the feedback.


Remove CoreLocation as a requirement

Replaces CLLocationCoordinate2D with OTRGeoPoint, thus removing the need for CoreLocation.
Fixes #6

This type has the same layout and types as CLLocationCoordinate2D,  so it should work to cast between the two, so there is no processing overhead to this change, just a little more code.
LGTM, but i'm curious, why the desire to not link CoreLocation?

Couple of reasons (which I'm interested to hear your thoughts on):

Didn't really see a need to add a medium sized dependency like CoreLocation for the sake of a single struct.
We're working on a wrapping layer for location services that will allow location to be retrieved from sources other than CoreLocation, along with injecting and mocking location traces to allow for things like replaying driving routes.

Gonna merge this. If any issues, open a new issue and I'll rectify.


Crushing Wheel makes server crash

I'm using an aternos server to play with this mod, i was building a gravel generator, and when it was time for the cobblestone to transform in gravel, the server just crashed, i left the machine on so i can't join my server anymore, any of my friends can't join now. (By the way i'm not 100% sure that the crushing wheel is the problem but, i think it has something to do with it)
Hopefully i recorded from start to end what i built, just saying in case that is needed.
In the console all i can find since i rejoined to the server is this:
[23:09:19] [Netty Epoll Server IO #4/ERROR] [minecraft/ArgumentTypes]: Could not serialize net.minecraftforge.server.command.ModIdArgument@1d4fef6d (class net.minecraftforge.server.command.ModIdArgument) - will not be sent to client! [23:09:19] [Netty Epoll Server IO #4/ERROR] [minecraft/ArgumentTypes]: Could not serialize net.minecraftforge.server.command.EnumArgument@8f1a909 (class net.minecraftforge.server.command.EnumArgument) - will not be sent to client! [23:09:20] [Server thread/ERROR] [minecraft/MinecraftServer]: Encountered an unexpected exception net.minecraft.crash.ReportedException: Ticking entity at net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:833) ~[?:?] {re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A} at net.minecraft.server.dedicated.DedicatedServer.func_71190_q(DedicatedServer.java:324) ~[?:?] {re:classloading,pl:accesstransformer:B} at net.minecraft.server.MinecraftServer.func_71217_p(MinecraftServer.java:764) ~[?:?] {re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A} at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:622) [?:?] {re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A} at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222] {} Caused by: java.lang.ClassCastException: com.simibubi.create.modules.contraptions.relays.belt.BeltTileEntity cannot be cast to com.simibubi.create.modules.contraptions.receivers.CrushingWheelControllerTileEntity at com.simibubi.create.modules.contraptions.receivers.CrushingWheelControllerBlock.func_176216_a(CrushingWheelControllerBlock.java:69) ~[?:mc1.14.4_v0.1.1] {re:classloading} at net.minecraft.entity.Entity.func_213315_a(Entity.java:533) ~[?:?] {re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A} at net.minecraft.entity.item.ExperienceOrbEntity.func_70071_h_(ExperienceOrbEntity.java:114) ~[?:?] {re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A} at net.minecraft.world.server.ServerWorld.func_217479_a(ServerWorld.java:587) ~[?:?] {re:classloading,pl:runtimedistcleaner:A} at net.minecraft.world.World.func_217390_a(World.java:684) ~[?:?] {re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A} at net.minecraft.world.server.ServerWorld.func_72835_b(ServerWorld.java:394) ~[?:?] {re:classloading,pl:runtimedistcleaner:A} at net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:829) ~[?:?] {re:classloading,pl:accesstransformer:B,pl:runtimedistcleaner:A} ... 4 more [23:09:20] [Server thread/ERROR] [minecraft/MinecraftServer]: This crash report has been saved to: /aternos/server/./crash-reports/crash-2019-10-16_23.09.20-server.txt [23:09:20] [Server thread/INFO] [minecraft/MinecraftServer]: Stopping server [23:09:20] [Server thread/INFO] [minecraft/MinecraftServer]: Saving players [23:09:20] [Server thread/INFO] [minecraft/ServerPlayNetHandler]: Gann4Life lost connection: Server closed [23:09:20] [Server thread/INFO] [minecraft/DedicatedServer]: Gann4Life left the game
Now i've found some code that's similar to that one...
[16Oct2019 23:09:19.201] [Server thread/INFO] [net.minecraft.server.dedicated.DedicatedServer/]: Gann4Life joined the game 63 | [16Oct2019 23:09:19.246] [Netty Epoll Server IO #4/ERROR] [net.minecraft.command.arguments.ArgumentTypes/]: Could not serialize net.minecraftforge.server.command.ModIdArgument@1d4fef6d (class net.minecraftforge.server.command.ModIdArgument) - will not be sent to client! 64 | [16Oct2019 23:09:19.249] [Netty Epoll Server IO #4/ERROR] [net.minecraft.command.arguments.ArgumentTypes/]: Could not serialize net.minecraftforge.server.command.EnumArgument@8f1a909 (class net.minecraftforge.server.command.EnumArgument) - will not be sent to client! 65 | [16Oct2019 23:09:20.556] [Server thread/ERROR] [net.minecraft.server.MinecraftServer/]: Encountered an unexpected exception 66 | net.minecraft.crash.ReportedException: Ticking entity 67 | at net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:833) ~[?:?] 68 | at net.minecraft.server.dedicated.DedicatedServer.func_71190_q(DedicatedServer.java:324) ~[?:?] 69 | at net.minecraft.server.MinecraftServer.func_71217_p(MinecraftServer.java:764) ~[?:?] 70 | at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:622) [?:?] 71 | at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222] 72 | Caused by: java.lang.ClassCastException: com.simibubi.create.modules.contraptions.relays.belt.BeltTileEntity cannot be cast to com.simibubi.create.modules.contraptions.receivers.CrushingWheelControllerTileEntity 73 | at com.simibubi.create.modules.contraptions.receivers.CrushingWheelControllerBlock.func_176216_a(CrushingWheelControllerBlock.java:69) ~[?:mc1.14.4_v0.1.1] 74 | at net.minecraft.entity.Entity.func_213315_a(Entity.java:533) ~[?:?] 75 | at net.minecraft.entity.item.ExperienceOrbEntity.func_70071_h_(ExperienceOrbEntity.java:114) ~[?:?] 76 | at net.minecraft.world.server.ServerWorld.func_217479_a(ServerWorld.java:587) ~[?:?] 77 | at net.minecraft.world.World.func_217390_a(World.java:684) ~[?:?] 78 | at net.minecraft.world.server.ServerWorld.func_72835_b(ServerWorld.java:394) ~[?:?] 79 | at net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:829) ~[?:?] 80 | ... 4 more 81 | [16Oct2019 23:09:20.602] [Server thread/ERROR] [net.minecraft.server.MinecraftServer/]: This crash report has been saved to: /aternos/server/./crash-reports/crash-2019-10-16_23.09.20-server.txt 82 | [16Oct2019 23:09:20.603] [Server thread/INFO] [net.minecraft.server.MinecraftServer/]: Stopping server
I just wanted to share this error (or maybe my mistake while building), but if any help could be provided, i will be thankful for that.

Thanks for reporting, for now you might be able to recover by

navigating to the saves/[world name]/serverconfig folder
open create-server.toml and look for [damageControl]
set the value of freezeCrushing to true and restart your server.

If you get a chance to revisit/recreate your setup, sharing a screenshot of it would help immensely.
From what I can see in the log, it seems that a belt was placed in the space where the crushing wheels would've had their active area.

Should be fixed in 0.1.1a, feel free to re-open if the issue persists.


Use pixi's multi-environment feature

Until recently, pixi didn't support multiple environments.
Because of that, we had to maintain an environment-minimal.yml for pyinstaller.
Now, multiple environments are supported we should switch to them: https://pixi.sh/configuration/#the-feature-and-environments-tables

We do this now 🎉


Add minio container

Container Contribution Checklist:

[x] You have added the new container as a module in the testcontainers directory (such as testcontainers/my_fancy_container.py).
[x] You have added any new python dependencies in the extras_require section of setup.py.
[x] You have added the extra_requires key to requirements.in.
[x] You have updated all python requirements by running make requirements from the root directory.
[x] You have added tests for the new container in the tests directory, e.g. tests/test_my_fancy_container.py.
[x] You have added the name of the container (such as my_fancy_container) to the test-components matrix in .github/workflows/main.yml to ensure the tests are run.
[x] You have rebased your development branch on master (or merged master into your development branch).

Codecov Report

Merging #271 (a50f4d9) into master (28e3a47) will increase coverage by 0.52%.
The diff coverage is 100.00%.

:exclamation: Current head a50f4d9 differs from pull request most recent head 77242ae. Consider uploading reports for the commit 77242ae to get more accurate results

@@            Coverage Diff             @@
##           master     #271      +/-   ##
==========================================
+ Coverage   86.20%   86.73%   +0.52%
==========================================
  Files          28       29       +1
  Lines         783      814      +31
  Branches       57       57
==========================================
+ Hits          675      706      +31
  Misses         78       78
  Partials       30       30

Impacted Files
Coverage Δ

testcontainers/minio.py
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


jest-environment-miniflare:  d1 migrations support

I'm trying to figure out how to apply migrations to d1 database using jest-environment-miniflare.
I have a config like this:
  testEnvironment: 'miniflare',
  testEnvironmentOptions: {
    ...
    d1Databases: ['__D1_BETA__DB'],
    ...
  }

And it works perfectly fine for my app. But the DB is created empty.
Of course, I can manually run DDL queries to create required tables. However, it will be amazing if there will be native migrations support. E.g. via providing path to the migrations directory.
Something like this:
  testEnvironment: 'miniflare',
  testEnvironmentOptions: {
    ...
    d1Databases: ['__D1_BETA__DB'],
    d1MigrationsDir: "./migrations",
    ...
  }

If miniflare can then apply all migrations as wrangler does it - it will be fantastic!

just a heads up, we used to do this (we have disabled this sort of testing since we upgraded to wrangler):
/* tests.ts */
import fs from "fs";
import { Kysely } from "kysely";
import { D1Dialect } from "kysely-d1";
import path from "path";

import type { DB } from "@/types/database";

export const setupDatabase = async (database: D1Database) => {
  const db = new Kysely<DB>({
    dialect: new D1Dialect({ database }),
  });

  const migrationDir = "./migrations";

  // List migration files in the directory
  const files = fs.readdirSync(migrationDir);

  // Filter out any non-SQL files
  const sqlFiles = files.filter((file) => path.extname(file) === ".sql");

  // Sort files by filename
  // eslint-disable-next-line @typescript-eslint/require-array-sort-compare
  const sortedFiles = sqlFiles.sort();

  // Read file contents and add to an array
  const fileContents = sortedFiles.map((file) => {
    const filePath = path.join(migrationDir, file);
    const fileContent = fs.readFileSync(filePath, "utf8");
    return fileContent
      .split("\n")
      .map((line) => line.replace(/--.*$/g, "").trim())
      .filter((line) => line !== "")
      .join(" ");
  });

  for (const migration of fileContents) {
    await database.exec(migration);
  }

  return db;
};

/* testFile.test.ts */

import { expect, it } from "vitest";

import { setupDatabase } from "@/tests/index";

import { list } from "./list";

const describe = setupMiniflareIsolatedStorage();
const bindings = getMiniflareBindings();

describe("test", () => {
  it("returns empty list for empty database", async () => {
    const { __D1_BETA__firmowid: database } = bindings;
    const db = await setupDatabase(database as D1Database);

    const result = await list({ db });

    expect(result).toMatchObject([]);
  });
});

until the update, it worked quite well :)


[FEATURE] Adding new section and updating CSS with fonts, responsiveness, and enhancing Hover effect

• Want to add a new section for My favorite resources
• Updating the font-family
• Improving the hover effects with smooth background color transition and more
• Enhancing responsiveness with new media queries for better layout on larger screens
• Adjusting font sizes and spacing for better readability
• Want to add box-shadow for enhanced visual depth
@michaella23 Can you please assign this issues to me ?
Thank You!

Thanks for taking this on. 🙂


collab-spark-extensibility-itp - powershell script doesn’t work

users were unable to get the powershell script to work on windows laptops. This was in the Explore Cisco Spark Extensibility Approaches & Platforms lab, Page 3.

Looks like Spark no longer supports TLS 1.0 connections (Powershell may default to 1.0), added a line to the script example to force TLS 1.2


Sliding & Snapshot images are hard to read in dark mode

Sliding & Snapshot images are hard to read in dark mode

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 63c64692-7533-cc84-251d-1994aab6a8fb
Version Independent ID: 10d6faea-cf4a-1e28-2dd6-76146128e1ce
Content: Introduction to Azure Stream Analytics windowing functions
Content Source: articles/stream-analytics/stream-analytics-window-functions.md
Service: stream-analytics
GitHub Login: @sidramadoss
Microsoft Alias: sidram

@matthiasguentert
Thanks for your feedback! We will investigate and update as appropriate.

@matthiasguentert
Thanks for your feedback! We have assigned the issue to author and will provide further updates.

Thanks @matthiasguentert we'll add a background to those images to prevent the dark theme from hiding the text. It should publish later today. Thx, Jason


No such file or directory - unprivileged_userns_clone

Am seeing the following error when attempting to install Sysbox on an RKE2 worker node.
sysctl: cannot stat /proc/sys/kernel/unprivileged_userns_clone: No such file or directory

Using the latest version of all system components, including the latest "longterm" version of the Linux kernel as per https://kernel.org.
os=Ubuntu 22.04.4 LTS, kernel=6.6.17-060617-generic, sysbox=0.6.3 CE, rke2=v1.28.6+rke2r1, rancher=v2.8.2

Here is a summary of results when using various kernel versions.

6.8-rc4 (mainline) - fails
6.7.5 (stable) - fails
6.6.17 (longterm) - fails
6.5.18 (from the mainline) - succeeds
6.5.18 (Ubuntu LTS HWE) - succeeds

Here are the Sysbox logs.

Detected Kubernetes version v1.28
2024-02-18T14:06:20.436036364+10:00 Adding K8s taint "sysbox-runtime=not-running:NoSchedule" to node ...
2024-02-18T14:06:20.502085520+10:00 node/w6 modified
2024-02-18T14:06:20.506327963+10:00 Adding K8s label "crio-runtime=installing" to node ...
node/w6 not labeled
2024-02-18T14:06:20.571532866+10:00 Deploying CRI-O installer agent on the host (v1.28) ...
2024-02-18T14:06:21.002092200+10:00 Running CRI-O installer agent on the host (may take several seconds) ...
2024-02-18T14:06:23.395783002+10:00 Removing CRI-O installer agent from the host ...
2024-02-18T14:06:23.908565022+10:00 Configuring CRI-O ...
2024-02-18T14:06:25.114476735+10:00 Adding K8s label "sysbox-runtime=installing" to node ...
2024-02-18T14:06:25.175698518+10:00 node/w6 not labeled
2024-02-18T14:06:25.194719353+10:00 Installing Sysbox dependencies on host ...
2024-02-18T14:06:25.205312310+10:00 Copying shiftfs sources to host ...
2024-02-18T14:06:25.206274620+10:00 Kernel version 6.6, which is above the max required for shiftfs (6.2); skipping shiftfs installation.
2024-02-18T14:06:25.206287839+10:00 Deploying Sysbox installer helper on the host ...
2024-02-18T14:06:25.499708961+10:00 Running Sysbox installer helper on the host (may take several seconds) ...
2024-02-18T14:06:28.052809028+10:00 Stopping the Sysbox installer helper on the host ...
2024-02-18T14:06:28.404940170+10:00 Removing Sysbox installer helper from the host ...
2024-02-18T14:06:28.704056754+10:00 Installing Sysbox on host ...
Configuring host sysctls ...
2024-02-18T14:06:29.921906091+10:00 sysctl: cannot stat /proc/sys/kernel/unprivileged_userns_clone: No such file or directory
2024-02-18T14:06:29.922058926+10:00 fs.inotify.max_queued_events = 1048576
2024-02-18T14:06:29.922064366+10:00 fs.inotify.max_user_watches = 1048576
fs.inotify.max_user_instances = 1048576
2024-02-18T14:06:29.922069990+10:00 kernel.keys.maxkeys = 20000
2024-02-18T14:06:29.922072582+10:00 kernel.keys.maxbytes = 1400000
2024-02-18T14:06:29.922075222+10:00 kernel.pid_max = 4194304

Also noticed the following vulnerability can be mitigated by disabling unprivileged user namespaces.

https://coder.com/blog/statement-on-the-recent-cve-2022-0185-vulnerability
https://ubuntu.com/security/CVE-2022-0185

sysctl -w kernel.unprivileged_userns_clone=0
Am assuming the Sysbox error encountered above relates to disabling unprivileged user namespaces with sysctl.

Found the following reference in the Sysbox user guide.

https://github.com/nestybox/sysbox/blob/release_v0.6.3/docs/user-guide/troubleshoot.md#unprivileged-user-namespace-creation-error

Along with the following fix.
sudo sh -c "echo 1 > /proc/sys/kernel/unprivileged_userns_clone"
This appears to allow the vulnerability identified above (as opposed to preventing it).
Attempting to execute the fix produces the following error.
sh: 1: cannot create /proc/sys/kernel/unprivileged_userns_clone: Directory nonexistent
Performing the following command shows that the /proc/sys/kernel directory exists.
ls -l /proc/sys/kernel

Hi @matthewparkinsondes,
Thanks for reporting the issue.
Context
Normally, Sysbox does not require the /proc/sys/kernel/unprivileged_userns_clone sysctl to be set, except in some scenarios (see below).  That's because the /proc/sys/kernel/unprivileged_userns_clone sysctl allows unprivileged users to create user-namespaces. Since Sysbox runs as a privileged process on the host, the sysctl does not apply to Sysbox per-se.
Having said that, inside a Sysbox container, all processes are unprivileged at host level since they run inside a user-namespace. If a process inside the Sysbox container wished to create a user-namespace (e.g., running Docker with userns-remap enabled for example), then that would fail unless /proc/sys/kernel/unprivileged_userns_clone is set at host level.
This is why the Sysbox K8s installer tries to set /proc/sys/kernel/unprivileged_userns_clone to 1. And sysbox-runc also checks for this when creating a container.
Though we could relax this so that Sysbox does not require it, in general having /proc/sys/kernel/unprivileged_userns_clone set to 1 is not a security problem, except when there are Linux kernel user-namespace vulnerabilities / CVEs (such as CVE 2022-0185) that allow an unprivileged process to escalate to root by creating a user-namespace.
The issue you reported

Here is a summary of results when using various kernel versions.
6.8-rc4 (mainline) - fails
6.7.5 (stable) - fails
6.6.17 (longterm) - fails
6.5.18 (from the mainline) - succeeds
6.5.18 (Ubuntu LTS HWE) - succeeds

This is interesting; normally Ubuntu/Debian distros have the /proc/sys/kernel/unprivileged_userns_clone sysctl, so something is wrong in 6.6.17 and above.
For the failing cases, what do you see under /proc/sys/kernel?

Hi @ctalledo,
Thanks, here are the contents of /proc/sys/kernel for the 6.5.21 passing case.
6.5.21

The 6.6.17 failing case removes the following from /proc/sys/kernel.

apparmor_restrict_unprivileged_io_uring
apparmor_restrict_unprivileged_unconfined
apparmor_restrict_unprivileged_userns
apparmor_restrict_unprivileged_userns_complain
apparmor_restrict_unprivileged_userns_force
unprivileged_userns_clone

6.6.17

The /proc/sys/kernel directory in the 6.7.6 failing case adds.

apparmor_restrict_unprivileged_unconfined

And it removes.

sched_child_runs_first

6.7.6

The /proc/sys/kernel directory in the 6.8-rc6 failing case is identical to 6.7.6
6.8-rc6

Thanks @matthewparkinsondes for the detailed response. Let me double check on my side and if confirmed, looks like we will need to change the Sysbox check for /proc/sys/kernel/unprivileged_userns_clone in the newer kernels.
Also, in the newer kernels, do you see /proc/sys/user/max_user_namespaces?
Thanks!

thanks @ctalledo.

Also, in the newer kernels, do you see /proc/sys/user/max_user_namespaces?

yes ... this is present in all of the newer kernels ... 6.6.21, 6.7.9 and 6.8-rc7

yes ... this is present in all of the newer kernels ... 6.6.21, 6.7.9 and 6.8-rc7

Ah, that likely means that starting with those newer kernels, Ubuntu distros are behaving as Fedora-based distros do, which use /proc/sys/user/max_user_namespaces as a way to enable unprivileged user-ns. If true, this is good news as it creates more consistency across the distros, though we need to update Sysbox to detect the kernel version and then check for the right file.

Hello! I am running into the same issue with kernel version 6.8. Is there a workaround?


App crashing when calling takePicture()

FATAL EXCEPTION: main Process: , PID: 3794
java.lang.RuntimeException: Unable to start service tarun0.gyananter.CaptureService@7a5c685 with Intent { cmp=tarun0.gyananter/.CaptureService }: java.lang.RuntimeException: takePicture failed
at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:3335)
at android.app.ActivityThread.-wrap21(ActivityThread.java)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1578)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:154)
at android.app.ActivityThread.main(ActivityThread.java:6123)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:867)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:757)

Caused by: java.lang.RuntimeException: takePicture failed
at android.hardware.Camera.native_takePicture(Native Method)
at android.hardware.Camera.takePicture(Camera.java:1461)
at android.hardware.Camera.takePicture(Camera.java:1406)
at com.androidhiddencamera.CameraPreview.takePictureInternal(CameraPreview.java:181)
at com.androidhiddencamera.HiddenCameraService.takePicture(HiddenCameraService.java:94)
at tarun0.gyananter.CaptureService.onStartCommand(CaptureService.java:63)
at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:3318)
at android.app.ActivityThread.-wrap21(ActivityThread.java)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1578)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:154)
at android.app.ActivityThread.main(ActivityThread.java:6123)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:867)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:757)

@passiondroid Can you please tell me the steps to reproduce the bug?

@kevalpatel2106 It started coming today only. Earlier it was not crashing. I am using HiddenCameraService and when takePicture() is called within the service from onStartCommand(). It throws exception with logs

android.hardware.Camera.native_takePicture(Native Method)

@passiondroid On which device you were testing?

@kevalpatel2106 Moto G4 plus

Great! Let me check the problem. Meanwhile if you find the issue, you can always generate PR.

Ok, I will clone the repo from here as soon as i get some time. I dont know why it stopped working. It was working fine earlier.

Are you using v1.3? or 1.2.1?

I am using 1.2.1

Can you test if this issue still occues in v1.3? I made some changes in v1.3. (See commit : https://github.com/kevalpatel2106/android-hidden-camera/commit/73ce010210641ae5421c924983f2997fecea0723)

I am still getting the same exception after changing to 1.3

@passiondroid I made some changes in latest version. Now, HiddenCameraService won't allow you to capture multiple photos simultaneously (like burst mode) because the camera is already acquired while capturing the second image.
Basically, HiddenCameraService won't allow you to take the second picture until the first picture is captured and processed.

Fix in release v1.3.2.

Maybe the question is still open
android.hardware.Camera.native_takePicture (Camera.java) android.hardware.Camera.takePicture (Camera.java:1473) android.hardware.Camera.takePicture (Camera.java:1418) com.androidhiddencamera.CameraPreview.takePictureInternal (CameraPreview.java:196) com.androidhiddencamera.HiddenCameraService.takePicture (HiddenCameraService.java:95) com.app.mytest.MyCamService$1.run (MyCamService.java:77) android.os.Handler.handleCallback (Handler.java:751) android.os.Handler.dispatchMessage (Handler.java:95) android.os.Looper.loop (Looper.java:154) android.app.ActivityThread.main (ActivityThread.java:6776) java.lang.reflect.Method.invoke (Method.java) com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:1518) com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1408)
On devices:
Samsung SM-A320Y (API 24) Android 7.0
Samsung SM-G950F (API 24) Android 7.0
Samsung SM-J530FM (API 24) Android 7.0
Samsung SM-J710F (API 24) Android 7.0
Samsung SM-G930A (API 24) Android 7.0
Motorola Moto G Play (API 23) Android 6.0.1
Xiaomi Redmi Note 4 (Android API 24) Android 7.0

I'm using v1.3.3 and still facing this problem too. Any idea?

can this https://stackoverflow.com/questions/21723557/java-lang-runtimeexception-takepicture-failed handle this problem ?

The problem has been in version 1.3.5...This version an be used to take images in burst mode..use that version instead.

@sid96 can you please show how to take images in burst mode?

Add:
SurfaceTexture st = new SurfaceTexture(MODE_PRIVATE);
pCamera.setPreviewTexture(st);
before:
pCamera.startPreview();
works for me on honeywell ED70 , android 7


add Restore option

as required in issue 1056 among others, I have implemented a function to restore a backup.

This doesn't include tribes (*.arktribe), tribute tribes (*.arktributetribe) or, mod persistent info (SaveGames).
Also, I wonder if it might be more concise to use something like:
local restorePath=
case "${file}" in
    SaveGames/*) # mod persistent info
        restorePath="${saverootdir}"
        ;;
    *.ini) # ini are the config files
        restorePath="${savedcfgdir}"
        ;;
    *.ark|*.arkprofile|*.arktribe|*.arktributetribe) # ark file in savedir
        restorePath="${savedir}"
        ;;
esac
if [ -n "${restorePath}" ]; then
    tar xjvf "${backupFile}" -C "${restorePath}" --strip-components=1 "${file}"
fi

Also make sure you quote any variable expansions that could have a space in them.

This doesn't include tribes (*.arktribe), tribute tribes (*.arktributetribe) or, mod persistent info (SaveGames).
Also, I wonder if it might be more concise to use something like:

So, is this a ready-to-use feature or still in beta?

This is in 1.6.53


Sorting on Tokens page broken

Sorting on the Tokens page is linked only to the pegged tokens. Example for sorting by the ticker, but the same effect can be reproduced on the name of the token, and probably all the other fields, but haven't continued further.

Hi @lion-200!
It isn't fixed yet, but I notice a new bug has been introduced instead (maybe it was already present, but didn't see it before): rows are "doubled". See what I mean here:

Hey mate @gadrian78 , thanks for the message. Some calls are changed on this page. Can you please check this again? Looks like the sorting is working now and double records are removed.

Yep, everything works perfectly about sorting @lion-200! Well, except one minor detail I've noticed now: the "Token Information" image link doesn't seem to do anything here on the Tokens page.

Hmm, I discovered today that sorting is partial or rather "in chunks" @lion-200 .
Here's what I mean. Both screenshots are taken from the same page, without any other action between them other than scrolling down a bit to take the second screenshot.


pulseaudio -k cannot be executed

The script only works if I enter the command "systemctl --user unmask pulseaudio.socket" an "systemctl --user start pulseaudio.service".
Otherwise, pulseaudio -k cannot be executed. This is necessary after each restart.
Is there an easy way to get everything back into the state. I'm using Debian testing here.

Well, as far as I know, even in bookworm pulseaudio is still the standard audio system.  This way it should be startet automatically with every fresh installation.
So I don't know why you had to unmask (make it able to start) pulseaudio (btw. the unit/service, not the socket!). Masked it in front?
After unmasking you should also "enable" the pulseaudio service to start after every reboot.
To get rid of the packet you installed via this scipt use "dpkg -r ".

installed packages are "libldac" and "pulseaudio-module-bluetooth"
might be you have to reinstall "pulseaudio-module-bluetooth" from the original repositories not to loose pulseaudio-bluetooth-support.