Set MySQL as the job cache for the Salt master

First of all, we can specify an external job cache. If we don't do that,
then the keep_jobs option only applies to the local cache. This means that
Salt will not clean up jobs, events and returns older than the specified
keep_jobs value (default: 24h) for the MySQL returner that we have already
configured.
Moreover, since we'd already be using MySQL as a job cache, we don't
have to use the local system (/var/cache/salt/master/jobs/) as a cache
(note that Salt would still be using this directory to avoid JID
collisions). The documentation also says that the local cache can be a
burden for large deployments.
See bsc#1044133
Signed-off-by: Miquel Sabaté Solà<EMAIL_ADDRESS>

To test this, I recommend setting keep_jobs to 1 as well (unless you want to keep your test running for 24 hours 😅).

Is this also cleaning the salt_events table?

Is this also cleaning the salt_events table?

From the source code (_purge_jobs function), it's cleaning: jids, salt_events and salt_returns.

I have one question: it seems from the source code (https://github.com/saltstack/salt/blob/d1f80bab3993059060ca51263e9d048e3a618051/salt/returners/mysql.py#L560-L581) that every loop_interval from the master will be called to clean_old_jobs, that in turn will look at keep_jobs, so in theory:

keep_jobs: 24
archive_jobs: False

Should call to _purge_jobs every loop_interval, which will perform a cleanup: https://github.com/saltstack/salt/blob/d1f80bab3993059060ca51263e9d048e3a618051/salt/returners/mysql.py#L472-L498.

I don't understand why we need to set the master_job_cache to mysql.

If you look at the code of the master, you'll see where the clean_old_jobs is called from the master. As you can see, it calls instead: https://github.com/saltstack/salt/blob/d1f80bab3993059060ca51263e9d048e3a618051/salt/daemons/masterapi.py#L189. It simply calls the clean_old_jobs of the current master_job_cache. If we do not assign this to mysql, then the local cache is assumed, thus never cleaning the MySQL DB and accumulating old events as the bug complains.
I'd agree that this could be better documented...

I see some big disadvantage of this solution, for medium and big installation all salt-minions will be connecting to MySQL db and report job status. It will be bottleneck for our solution, I would propose to use Redis as complimentary external job cache, which is memory database.

As for redis, we don't have an RPM for this. This could be discussed for future releases but is not going to happen for 1.0.

@mjura are you sure every minion connects? this is in the master config, and the config var even mentions "master" in it?
Re redis: agree with Jordi, deciding to add another DB at this point is too risky, and - even if it was available - I doubt it would be necessary.

@mjura are you sure every minion connects? this is in the master config, and the config var even mentions "master" in it?

That's right. This only makes the caching from the master side into the MySQL DB. Also, note that we are already storing all the events from minions anyways into the DB (which is fine), so this "bottleneck" was there.
In regards to Redis, we could give it a thought after the 1.0 release, but not before. So I agree with both @jordimassaguerpla and @kiall.

 Bug 1044133 - Velum's DB keeps growing

Rebase is needed

I see some big disadvantage of this solution, for medium and big installation all salt-minions will be connecting to MySQL db and report job status. It will be bottleneck for our solution, I would propose to use Redis as complimentary external job cache, which is memory database.

I don't think we will reach that point at any time soon. Anyway, if we reach that point where MySQL could seem to be a bottleneck, we could evaluate different alternatives. Personally, I would prefer not to add another database to our setup...

A couple of questions:

are we fine keeping jobs for 24 hours or do we want to increase the default value to something more (1 week?)
wouldn't it be better to be explicit about how much time everything is kept. I would be in favor of having keep_jobs: 24 written down into our configuration to avoid to search for the default value use by salt.

are we fine keeping jobs for 24 hours or do we want to increase the default value to something more (1 week?)

and

If anything - we should be reducing the time we keep them - the amount of events generated by a large cluster is huge, and causes MySQL to be a bottlekneck

In my opinion, Salt's default is quite good. It might be too low for small clusters, but it's actually to big for large deployments. I guess that we should tune this after getting feedback from customers (e.g. fetching what's the average cluster size).

wouldn't it be better to be explicit about how much time everything is kept. I would be in favor of having keep_jobs: 24 written down into our configuration to avoid to search for the default value use by salt.

👍
I'll rebase this PR now, so it can be properly reviewed.

Rebased. Please, take a look again 😇 /ping @ereslibre @flavio @kiall @inercia @grahamhayes

e2e tests are failing because of an older build (and we cannot trigger a re-build...). So, once I get more approvals I'll push -f it again to force a rebuild, and if it passes I can merge it myself.

@mssola : Please force push this change again to retrigger CI tests

Forced push.

@flavio

are we fine keeping jobs for 24 hours or do we want to increase the default value to something more (1 week?)
I think a week is a more reasonable setting TBH. We need to make sure that we collect all the information of a problem on a customer within this range, so I'd feel more comfortable with a week at least.

Since @mssola's explanation in https://github.com/kubic-project/salt/pull/132#issuecomment-310974323, I think despite the window this change is good to go.

The e2e tests are failing because of unrelated reasons, that's why I merged it.


Parameter to input 1.38 update Part 1.

Convert Parameters to Inputs marked as "uniform".  Note that according to the spec all "filename" and "string" types are uniform. All other inputs are marked with uniform="false".
Modify addInput() to make name, type required arguments and add in a isUniform argument (default=false) to allow a single method to create and set whether something is uniform. Note that according to the spec all filename and string types are uniform.
Add new getIsUniform() and setIsUniform() APIs.
Replace all get/set/isA<Parameter>() calls with new API calls.
Remove getPrimaryShaderParameters() API from Material Element.
Update Core, ShaderGen + tests. RTS passes for GLSL, MDL, and Vanilla OSL.

Note : While 1.38 is WIP, we keep the version at 1.37 so that the upgrade will be appied.

stdlib (defintion + node graph implementations) after upgrade. inputs with uniform attributes are converted parameters
1_37_to_1_38_updated .mtlx.txt

Looks good, thanks @bernardkwok!


pip modules and platform MULTIARCH identification

Following the pattern   <cpuarch>-<libc-api>-<libc-abi> , values defined for  sysconfig.get_platform() and sysconfig.get_config_vars('MULTIARCH'):
shell.wasm.sh : wasm_1_emscripten or wasm_1pic_emscripten
TODO: iirc there are variants API/ABI wasm/asm.js flavours regarding asyncify and threading for emscripten.
shell.armeabi-v7a.sh :  armv7a_19_android
shell.arm64-v8a.sh : aarch64_21_android
shell.x86.sh : i686_19_android
shell.x86_64.sh : x86_64_21_android
and also :
shell.wasi64.sh : wasm64_1_wasi or wasm64_1pic_wasi
shell.wasi32.sh : wasm32_1_wasi or wasm32_1pic_wasi
shell.asm.js.sh : asmjs_1_fastcomp
Exemple of a pip 3.9 wheel build session :
[PyDK:armeabi-v7a] /data/cross/pydk $ pip3-armeabi-v7a wheel numpy
Collecting numpy
  Using cached numpy-1.19.4.zip (7.3 MB)
Building wheels for collected packages: numpy
  Building wheel for numpy (setup.py) ... done
  Created wheel for numpy: filename=numpy-1.19.4-cp39-cp39-armv7a_19_android.whl size=13205762 sha256=98b79f11fdc6f2c30042551145d5814249f27ebc1921ac26f2bcc5d1658ffb01
  Stored in directory: /cache/pip/wheels/6f/d7/25/0fe5a4a476d4881c00f6e6914d4d40f9c42cfe334faafd79a3
Successfully built numpy
[PyDK:armeabi-v7a] /data/cross/pydk $ file numpy-1.19.4-cp39-cp39-armv7a_19_android.whl
numpy-1.19.4-cp39-cp39-armv7a_19_android.whl: Zip archive data, at least v2.0 to extract
[PyDK:armeabi-v7a] /data/cross/pydk $

Another example using :
sysconfig.get_platform()
sysconfig.get_config_vars('INCLUDEDIR')

Processing /data/cross/pydk/testing/pygame
Building wheels for collected packages: pygame
  Building wheel for pygame (setup.py) ... done
  Created wheel for pygame: filename=pygame-2.0.1.dev1-cp39-cp39-armv7a_19_android.whl size=3158603 sha256=8aab801a2f7f5bb4b1441ea8ef238ab7d87a6146474bc399348293686ba9ab98
  Stored in directory: /tmp/pip-ephem-wheel-cache-2cn1i_9y/wheels/b5/14/de/55664e14fd0cccd3089431b21da9fa1d8aa6deee0833e13657
Successfully built pygame
[PyDK:armeabi-v7a] /data/cross/pydk/testing/pygame $

is an installation of cross pip toolchain via pip (like p4a) viable ?
https://github.com/pygame/pygame/pull/2311#issuecomment-728741841
Preleminary discussion was engaged with @tshirtman on python-for-android, idea looks welcomed but time/manpower seems to lack both sides.
Meanwhile Beeware pip support seems stuck somewhere in an ungranted future.
Interesting side effect of multiarch pip shell with https://github.com/eerimoq/mys from @eerimoq is to allow transpiling + cross compiling a python subset to native code ( AOT ).
llvm-jit based python modules like pythran ( or transonic backends) not tested yet.

see  uname machine to reflect wasm architecture :
https://github.com/emscripten-core/emscripten/pull/13440 <= merged !
see
uname should return something more meaningful

not sure how it would affect pip downloads but it may:
No more exec from data folder on targetAPI >= Android Q
(  termux termux-app 1072 )


performance(aws_collector): don't persist shortcut reports

Shortcut reports are sent to update the UI quickly, on events like a container starting.
We don't need to persist them in the time-travel data since the same information will be covered by a regular report a few seconds later.

S3 charges per write (and read) as well as for monthly storage.
I believe shortcut reports are sent immediately to the UI.

I believe shortcut reports are sent immediately to the UI.

Even when time-travelling?

I was wondering whether this make the UI less fluid when time-travelling

Sorry I didn't read that last bit.
In time-travel mode, if you inch the time forward by one second you would be able to tell the difference.
I'm not sure I would call that "fluid" since it's quite hard to achieve such fine control, and a moderately-sized report will take a couple of seconds to render.
I don't think this is worth commenting.


Convert OTEL model directly to storage model

Elasticsearch exporter converts OTEL span to Jaeger model and which is passed to SpanWriter.WriteSpan. The writer converts the span to ES storage model. We could consider converting the span directly to ES model to save some cycles.
This will require a separate convertor and will have to refactor ES span writer and expose a write method accepting Jaeger ES model.
It has to be done for:

[x] elasticsearch
[ ] cassandra
[ ] badger (uses Jaeger model directly)
[ ] in-memory (uses Jaeger model directly)
[x] kafka

This has been done for Elasticsearch storage.
It has to be done for:

[ ] cassandra
[ ] badger
[ ] in-memory
[ ] kafka

We may want to consider if we want to support Cassandra with OTEL at all.

@pavolloffay did you run any benchmark to see the impact of direct model xform?

also, OTel trace model is slightly richer than Jaeger's, the translation to ES happened still to Jaeger's logical model? I think we should consider if we want to go to an extended, OTel-compatible model in the storage.

I haven't run any perf tests. The new implementation basically skips one translation (OTLP-> Jaeger model) I haven't tested how much resources it takes.

also, OTel trace model is slightly richer than Jaeger's, the translation to ES happened still to Jaeger's logical model?

The translation is done form OTLP to Jaeger ES data model directly. Agree we should have a look and extend storage models if necessary and also do appropriate handling in the UI.

Hey! I have implemented unmarshalling OTLP from protobuf https://github.com/open-telemetry/opentelemetry-collector/pull/1948 (marshalling was already implemented), so it's now possible to use OTLP directly for in-memory and badger. Just need a new version of collector with my PR

If this is possible to go from Kafka (OTLP) -> Jaeger ES what configuration does that require on the kafka Side?

yes at the moment it is possible to use OTEL collector with Kafka receiver with OTLP encoding and send data to Jaeger collector via Jaeger exporter. Jaeger collector will then store the data in ES.

@pavolloffay i think it would be useful to add this to the APIs page in the docs, ie the API for the ingester which is not documented there.

There is no useful information in this ticket, closing. It will come back in another form in #3500.


Strange String.fromCharCode behavior

var buffer = new ArrayBuffer(2);
var view = new Uint8Array(buffer);
view.fill(0);
view[0] = 65;
// view[1] -> 0

var str = String.fromCharCode.apply(null, view);

str += "B"

console.log(str.length); // -> OUTPUT: 3 -> B is included
console.log(str); // OUTPUT: `A`  -> char('A') + char(0) + char('B')

console.log(str.indexOf('B')); // OUTPUT: 2

str = [str, 'end'].join('-');
console.log(str); // OUTPUT: 'A' -> char('A') + char(0) + char('B') + "-" + "end"
console.log(str.length); // OUTPUT: 7

Spec (http://www.ecma-international.org/ecma-262/6.0/#sec-string.fromcharcode) says Let length be the number of elements in codeUnits.
Output(length) is matching to what spec says. However, see other cases above as the rest of it feels unexpected.
Shouldn't we, at some point drop char(0) ?

We have a similar behavior with SM. V8 drops char(0) and the rest on string concat.

In latest Chakra, Edge 38.14393, Firefox Nightly 56, and Chrome 56 - I'm seeing all the same behavior as defined by this modified test case:
function assertEqual(a, b) {
    if (a !== b) {
        console.log(`Failed: expected ${b}, actual ${a}`);
    }
}

var buffer = new ArrayBuffer(2);
var view = new Uint8Array(buffer);
view.fill(0);
view[0] = 65;
// view[1] -> 0

var str = String.fromCharCode.apply(null, view);

str += "B"

assertEqual(str.length, 3); // B is included
assertEqual(str, 'A\x00B'); // char('A') + char(0) + char('B')

assertEqual(str.indexOf('B'), 2);

str = [str, 'end'].join('-');
assertEqual(str, 'A\x00B-end');
assertEqual(str.length, 7);

console.log('PASSED');

All print PASSED and no failures. So, I'm not sure what difference in behavior you're seeing. I think in JavaScript strings, we don't treat a null character \x00 as a null terminator (truncating the string at that point), but instead we treat it as a character in the string. If we did treat it as a null terminator, I think that would be considered a bug.

I think in JavaScript strings, we don't treat a null character \x00 as a null terminator (truncating the string at that point), but instead we treat it as a character in the string.

This is the part has confused me. I'm not sure if v8 version I had checked was the part of latest Chrome. If they all have the same behavior and this doesn't sound like an issue, I also don't see a reason to keep this open on CC repo though.
It's just char(0) is NULL (ASCII) and having it represented in a JavaScript String as if it's just another char made me think there is something wrong there.

If v8 is terminating on char 0, then please file a bug with v8 - null characters don't mean anything in JS.


stylesheet - bad reference

Hello,
it seems that the baseUrl is prefixed to css and js stylesheet files in index.html.
For github depoyment at least, this causes css and js not to load properly. I removed the baseUrl by hand and now, this is alright. Would be nice to fix that in the theme.
	<link rel="stylesheet" type="text/css" media="screen" href="<baseUrl>css/normalize.css" />
	<link rel="stylesheet" type="text/css" media="screen" href="<baseUrl>css/main.css"/>

	<script src="<baseUrl>js/feather.min.js"></script><script src="<baseUrl>js/main.js"></script>

Thanks in advance!

Any idea how to fix this within the Hugo theme?

@JeremyWng
Thanks for reporting this issue. I am planning to refactor this project ASAP.


CSV Export

Please describe what your change is about.
This implements the CSV export endpoint.
What issues does it solve?
See https://openshift.io/openshiftio/Openshift_io/plan/detail/40

[test]

[test]


#696 - PsLinkedinTest - fixed PMD.ExcessiveMethodLength warning

Issue #696
Refactored PsLinkedinTest class. Anonymous classes were transformed in named inner classes.

@rui-castro Many thanks, I will find someone to review it before we merge

@mkordas review this one, please

@rui-castro I'll check it soon

@rui-castro see my initial review above

@rui-castro ping

@mkordas I will push a new version in the next hour.

@mkordas PR updated.
No more shared variables with test data. Test data is passed to inner classes as constructor arguments, but had to introduce @checkstyle ParameterNumberCheck  in inner classes constructors.
A possible alternative to avoid ParameterNumberCheck would be to put all test data inside another inner class (TestData) and pass this TestData to the constructors of TokenTake, PeopleTake and LinkedinScript. But, this TestData would be a "bag of data" and that is not desirable, right?

@mkordas the last commit updated @since tags to 1.1 and removed unnecessary checkstyle suppressionsClassDataAbstractionCouplingCheck and MultipleStringLiteralsCheck.

@rui-castro I'll check it again soon :)

@rui-castro on it now :)

@rui-castro

A possible alternative to avoid ParameterNumberCheck would be to put all test data inside another inner class (TestData) and pass this TestData to the constructors of TokenTake, PeopleTake and LinkedinScript. But, this TestData would be a "bag of data" and that is not desirable, right?

Yes, we don't want to have such objects... I don't have better idea for now

@rui-castro generally looks good, see my comments above

@mkordas removed dots in @param lines and removed peoplepath variable.

@rui-castro thanks, looks good!

@rultor merge

@rultor merge

@mkordas Thanks for your request. @yegor256 Please confirm this.

@yegor256 ping

@yegor256 please check this PR

@yegor256 ping

@yegor256 please check my PR. This delay is hurting my speed rating. At this pace it will take forever to be faster than @mkordas ;)

@yegor256 ping

@rultor merge

@rultor merge

@yegor256 OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@yegor256 Done! FYI, the full log is here (took me 26min)

@ypshenychka please, let us know what do you think about this ticket, according to our QA rules

@rultor deploy

@rultor deploy

@davvd OK, I'll try to deploy now. You can check the progress here

@rultor deploy

@davvd Done! FYI, the full log is here (took me 27min)

@davvd Quality is good.

@davvd Quality is good.

@ypshenychka thanks for the review

@mkordas paied 10 mins to @ypshenychka for QA review (payment ID is 94257964); Much appreciated! 40 mins was added to your account, payment ID 57961bc508baac6af2818534, time spent is 209 hours and 19 mins; review comments (c=25) added as a bonus; +40 added to your rating, at the moment it is: +5291


added ${fileBasenameNoExtension} for asset path

When handling images you can now use  eg.: ${fileBasenameNoExtension}.assets for asset path

This makes it compatible with how e.g. typora.io is handling images


Implement /get_prefs and /save_prefs API

Resolves https://github.com/fluid-project/personal-data-server/issues/5
This pull request implements /get_prefs and /save_prefs API. Other endpoints mentioned in https://github.com/fluid-project/personal-data-server/issues/5 have already implemented and merged.

@amb26, all review comments have been addressed. Ready for another review. Thanks.


Cannot install sui binaries

Steps to Reproduce Issue
I am trying to install sui binaries with
cargo install --locked --git https://github.com/MystenLabs/sui.git --branch devnet sui
Expected Result
Successful installation
Actual Result
Error
error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-arch" "x86_64" "-I" "snappy/" "-I" "." "-Wall" "-std=c++11" "-DNDEBUG=1" "-o" "/var/folders/0h/tf8nh7sj20dgzt8dchfgzwj40000gn/T/cargo-install8lFjM1/release/build/librocksdb-sys-6b1b9f93f978fa76/out/snappy/snappy.o" "-c" "snappy/snappy.cc" with args "c++" did not execute successfully (status code exit status: 1).
error: failed to compile sui v0.17.0 (https://github.com/MystenLabs/sui.git?branch=devnet#79100a56), intermediate artifacts can be found at /var/folders/0h/tf8nh7sj20dgzt8dchfgzwj40000gn/T/cargo-install8lFjM1
System Information

OS: macOS Catalina Version 10.15.7

Hi, I indeed updated my OS and the installation succeeded. Thanks for the help.

Thanks I updated the OS to 13.0.1 and the installation succeeded. Thanks for the help


Adding more books in the app

Currently there are a minimum of just 5, we need more atleast a 10-12 to showcase.

@Omafovbe would you like to?

So in fulfilling this task you need both image cover, price both on the home component as well as the basketReducer right?

should i get you assigned?

Yes please do

Got a report PR #37 was merge to the branch again and not into master. was this intended?

No, Its not merged to anything @Omafovbe

Yes I would love to. Thanks
On Wed, 17 Aug 2022, 13:32 Kamal Nayan, @.***> wrote:

@Omafovbe https://github.com/Omafovbe would you like to?
—
Reply to this email directly, view it on GitHub
https://github.com/legendarykamal/Lets-go-shop/issues/6#issuecomment-1217947877,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEXP2HYUFVCJJ6XKSWUL6S3VZTLU3ANCNFSM56ZGT7QA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Yes please do
On Wed, Aug 17, 2022 at 8:06 PM Kamal Nayan @.***>
wrote:

should i get you assigned?
—
Reply to this email directly, view it on GitHub
https://github.com/legendarykamal/Lets-go-shop/issues/6#issuecomment-1218388412,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEXP2H63P2Q4TATQB47U67DVZUZ2DANCNFSM56ZGT7QA
.
You are receiving this because you were mentioned.Message ID:
@.***>


[Docs]: How to configure up Gmail email provider for Appsmith instance

Users would like to know about having the Appsmith server use the Gmail SMTP server to send emails
See post here for reference and starting point

Published.


Routing from work selection to work detail

:goberserk:

This is quite a mystery.
Let's not worry about perma-url and just get the clicking working first.

Past-me over-complicated things. A custom router solves all issues.


Fixed size

Picture in documentation includes fixed size option which is no longer available

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 490a08fe-c26e-6cae-5495-0b4f4e71e35d
Version Independent ID: 79432f02-5059-2627-11e3-eccaf1109665
Content: Create a function triggered by Azure Cosmos DB
Content Source: articles/azure-functions/functions-create-cosmos-db-triggered-function.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

Hi @IslayJunkie - Thank you for your feedback! We have assigned this issue to the author to review further and take the right course of action.

The image is now updated to reflect current UI. We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


svg update according to annajiat suggestions, main naming and coloring

Follow up PR to provide updated svg file of the gh pages illustration as suggested by @annajiat
Further updates:

master -> main branch naming
coloring (please suggest better color palette for color blind friendlyness)

Why is the jpg conflicting? Do we need the jpg having the svg?

The jpg does not exist anymore in my fork. Don't know why that is a conflict.

Maybe because it got modified in the other PR and removed in this PR and this PR branched off from an earlier situation?

yes, that was it, I forked the repository much earlier and it did not contain some earlier changes as well. Now it should be fine? Please review and merge :)

Updated colors according to suggestions from link in issue #161 to red, orange and blue.
Please review and merge!

The svg is now moved onto A4 page (it was only partly on before), to be used as is without the need to generate jpg.
Please review in context

updated and ready for merge (from my side)


Exception at start

Hi Gerwin,
I get the following exception at start related to sunData, and sometimes also later. I merged the old ORTEP scripts using the sst sensor and model workspace with the most recent Demkit. In the ORTEP model the sunData is used to predict PV on both 3-phase and single-line inverters:

pvc = LivePvCtrl("PVCTRL_3P_L"+str(i+1), pv, ctrl, sun, sim)
pvc = LivePvCtrl("PVCTRL_SP_L3", pv, ctrl, sun, sim)

09:24:43 | MESSAGE: Starting
09:24:50 | MESSAGE: Simulating at time: 09-08-2023 09:24:50 CEST+0200
Exception in thread Thread-3 (zCallSingle):
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/home/dennis/work/ortep/demkit-github/components/core/core.py", line 682, in zCallSingle
return getattr(recv, func)(*args)
File "/home/dennis/work/ortep/demkit-github/components/ctrl/groupCtrl.py", line 225, in initiatePlanning
self.doInitialPlanning(s)
File "/home/dennis/work/ortep/demkit-github/components/ctrl/groupCtrl.py", line 261, in doInitialPlanning
results = self.zCall(self.children, 'doInitialPlanning', s, list(parents))
File "/home/dennis/work/ortep/demkit-github/components/core/entity.py", line 143, in zCall
return self.host.zCall(receivers, func, *args)
File "/home/dennis/work/ortep/demkit-github/components/core/core.py", line 654, in zCall
return self.zCallList(receivers, func, *args)
File "/home/dennis/work/ortep/demkit-github/components/core/core.py", line 671, in zCallList
result[recv] = getattr(recv, func)(args)
File "/home/dennis/work/ortep/demkit-github/components/ctrl/devCtrl.py", line 158, in doInitialPlanning
result = copy.deepcopy(self.doPlanning(signal,  False))
File "/home/dennis/work/ortep/demkit-github/components/ctrl/loadCtrl.py", line 133, in doPlanning
p[c] = self.doPrediction(time-(time%timeBase),  time-(time%timeBase)+timeBaselen(signal.desired[c]))
File "/home/dennis/work/ortep/demkit-github/components/ctrl/live/livePvCtrl.py", line 82, in doPrediction
result = self.predictProduction(sunPrediction, startTime,  endTime)
File "/home/dennis/work/ortep/demkit-github/components/ctrl/live/livePvCtrl.py", line 171, in predictProduction
result.append(min(0, max(sunData[idx]['GHI']*self.model[b][0] + sunData[idx]['DNI']*self.model[b][1], self.maxProduction[b]) ) )
IndexError: list index out of range
Kind regards,
Dennis

I did some debugging, on the arguments of result.length and found the following, apparently self.maxProduction is empty, while it worked fine until 01:00 this morning after I started yesterday. I tried changing solcast
12:17:41 | WARNING: [PVCTRL_3P_L1] livePvCtrl model b=20 idx=0, len(model)=48, len(maxProduction)=0

Apparently not all realized values are available after restarting demkit, had to also change the following om loadCtr.py:

  				self.realized[c][time] = prediction[idx]

  				try:

  					self.realized[c][time] = prediction[idx]

  				except:

  					self.realized[c]={ time : prediction[idx] }


bigquery support on google cloud platform

Description
Make Harmony blockchain data available on google GCP bigquery
Context
Google BigQuery provides a powerful interface for data queries and analyses.
We would like to make Harmony blockchain data available on Google BigQuery, as public data set.
For example, the following lists the current public data set available on big query platform.
https://cloud.google.com/blog/products/data-analytics/introducing-six-new-cryptocurrencies-in-bigquery-public-datasets-and-how-to-analyze-them

Acceptance Criteria

[ ] submit Harmony blockchain data to Google big query public data set
[ ] provide an automation solution/script from reading harmony blockchain data to updating the big query dataset
[ ] complete documentation on the setup of the submission and update pipeline

Reward
US$2,000 equivalent amount of Harmony ONE token.

I m interested @LeoHChen . Should I take this up?

Thanks! Started work

I am happy to help too!

@cpurta I looked at your entry which is so far a really great submission. I had multiple issue trying to run your bounty :

there are lots of typo in your README:

docker image name doesn't match the docker build and run ('gcr' vs 'grc')
environment variable GOOGLE_APPLICTION_CREDENTIALS

the docker image was missing dynamic libraries and hence couldn't be used
additionally the software doesn't run without :

GCP bigquery DB and tables are not created
model used is using a reserved keyword hash

Please help out to fix all the issues including the addition of some improvement logic like testing the existence of the database and table then create it if it not created.
Thanks

@sophoah thank you for the feedback on the submission.

I have addressed most if not all of the typos in the README.
Can you elaborate/provide more context into missing libraries in the docker image (it is just an alpine image running a compiled binary)
I am currently addressing the issue of reserved hash keywords in the model and performing automatic table creation if they do not exist.

@cpurta the compilation is building a binary that refers to dynamic libraries that needs to be present in the system (or image in our case) when i tested and build the image via your Dockerfile glibc was missing, give it a try and you'll understand.
let me know once the changes has been pushed and I can help test again.

@sophoah I have pushed the changes that check for the needed tables and create them if needed. I also attempted to dig into the compilation issue you mentioned but I am unable to recreate it without much context. Can you provide some of the commands you are using when compiling the binary?

@sophoah do you have any other feedback on the project?

@cpurta sorry for the delay
without the dockerfile change above I have this error :
standard_init_linux.go:219: exec user process caused: no such file or directory

I think it's best for the binary to be build the run as part of the docker image creation directly
With a working image, when I start I have this error :
error running program: googleapi: Error 404: Not found: Dataset harmony-bigquery:crypto_harmony, notFound

so dataset is not created automatically
Manually creating the dataset and running the container again worked and syncing is now ongoing.
Stopping the container and restarting works. The process detects the last synced block and start again a sync from there
Can you please fix the Dockerfile so the binary is created during the build and add the auto dataset creation.
@LeoHChen this bounty is really close to finish

@sophoah I believe that I resolved both of the issues that you mentioned. The binary is now being compiled within a golang docker container and is then mounted onto the alpine Linux image using a multi-stage build. I have tested that the docker build works and the application container will run the binary. I updated the project README with an updated docker build command that is now run from the project root directory as opposed to the docker directory. So make sure to use that command as the previous one will not work now.
Also, I added a check for the project dataset_id specified and should create the dataset within the project if it does not exist. I was unable to smoke test this as my free tier for GCP has expired. Do you mind testing the changes to make sure that they are up to par with the specifications of the bounty?

@cpurta the build part is working fine no thanks
Now when I start the container and I didn't delete the data that was being indexed so far, i have :
{"level":"error","ts":1632302303.588839,"caller":"runner/backfill.go:179","msg":"unable to insert block transactions into BigQuery","block_number":10158169,"error":"googleapi: Error 404: Not found: Table harmony-bigquery:crypto_harmony.transactions, notFound","stacktrace":"github.com/cpurta/harmony-one-to-bigquery/internal/runner.(*BackfillRunner).backfillBlocks\n\t/go/src/github.com/cpurta/harmony-one-to-bigquery/internal/runner/backfill.go:179"}

so on that I've created manually the transactions table, and i then had :
{"level":"error","ts":1632302541.6222308,"caller":"runner/backfill.go:179","msg":"unable to insert block transactions into BigQuery","block_number":10160571,"error":"googleapi: Error 400: The destination table has no schema., invalid","stacktrace":"github.com/cpurta/harmony-one-to-bigquery/internal/runner.(*BackfillRunner).backfillBlocks\n\t/go/src/github.com/cpurta/harmony-one-to-bigquery/internal/runner/backfill.go:179"}

so please help out to auto-create the table and and make sure all blocks that has been indexed before have their transactions filled up in the transaction table.
Thanks

@sophoah I just got around to addressing the issues you presented. I just updated the program to create the dataset and tables needed even after deletion. Let me know when you have tested it out and if there are any issues you encounter.

@cpurta
{"level":"error","ts":1633421894.5614786,"caller":"runner/backfill.go:186","msg":"unable to insert block transactions into BigQuery","block_number":16340108,"error":"googleapi: Error 400: The destination table has no schema., invalid","stacktrace":"github.com/cpurta/harmony-one-to-bigquery/internal/runner.(*BackfillRunner).backfillBlocks\n\t/go/src/github.com/cpurta/harmony-one-to-bigquery/internal/runner/backfill.go:186"}

@cpurta we had transaction since block 0 :) but true there wasn't a lot. I think INFO/WARNING is fine for me, it's just to inform there were no tx on a given block.
Doing a backfill through separate command to sync missing transaction is ok, if that can work at the same time as the current main command running, that would perfect. Let me know once you have it since I believe if no other error, that would be the last item.
On a side question, how difficult would that be to include other shards (s1, s2, s3) or even network like testnet?

@cpurta
docker exec -it gcp-bigquery hmy-bq-import --start-block 1
Incorrect Usage. flag provided but not defined: -start-block

NAME:
   hmy-bq-import - command line tool to import Harmony One blockchain data into GCP BigQuery

USAGE:
   hmy-bq-import [global options] command [command options] [arguments...]

VERSION:
   v0.0.1

AUTHOR:
   Chris Purta<EMAIL_ADDRESS>
COMMANDS:
   backfill  pull historical block data from Harmony One blockchain and insert into GCP BigQuery
   help, h   Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help (default: false)
   --version, -v  print the version (default: false)
error running program: flag provided but not defined: -start-block
[ec2-user@ip-172-31-19-248 harmony-one-to-bigquery]$

Thanks @sophoah. I'd close this bounty, Please create an issue for ourselves to keep updating the indexing.  also, please work with the @cpurta to write a blog post on how to use the big query on gcp for harmony blockchain data. Thanks.


Give SigVerify and ShredFetch threads unique names

Problem
See https://github.com/anza-xyz/agave/issues/34 for context
Summary of Changes

solTvuFetchPmod ==> solTvuPktMod + solTvuRepPktMod
solSigVerifier ==> solSigVerTpu + solSigVerTpuVot

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.8%. Comparing base (ce34f3f) to head (4758844).

Additional details and impacted files
@@            Coverage Diff            @@
##           master      #98     +/-   ##
=========================================
- Coverage    81.8%    81.8%   -0.1%
=========================================
  Files         838      838
  Lines      225941   225940      -1
=========================================
- Hits       184945   184884     -61
- Misses      40996    41056     +60


🐞TS type declaration packages should be in devDependencies, not dependencies

Category

[x] Question
[ ] Typo
[ ] Bug
[ ] Additional article idea

Expected or Desired Behavior
NPM packages that are not needed for production & only for development, such as TypeScript type declaration files, should only be in the devDependencies section of package.json within a Node.js project. This means the following two packages should be in devDependencies:

@types/webpack-env
@types/es6-promise

Observed Behavior
The two packages listed above are in the dependencies section of package.json
Steps to Reproduce
Create a SPFx project with v1.5.1 and look at the package.json file

Thanks for the feedback.  This is not urgent but we have added it to our backlog.

Yup... not urgent as it doesn't impact anything, just making things more correct. Accept PR's to the generator and I'll submit the fix ;)
BTW... same comment for React-based projects too. The two React @types/* packages should also be moved => devDependencies


fix: Alignment and New Features

Description
[Please include a brief description of the changes or features added]
Related Issues
[Cite any related issue(s) this pull request addresses. If none, simply state “None”]

Closes #

Type of PR

[ ] Bug fix
[ ] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
[Attach any relevant screenshots or videos demonstrating the changes]
Checklist

[ ] I have gone through the contributing guide
[ ] I have updated my branch and synced it with project main branch before making this PR
[ ] I have performed a self-review of my code
[ ] I have tested the changes thoroughly before submitting this pull request.
[ ] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[ ] I have commented my code, particularly in hard-to-understand areas.

Additional context:
[Include any additional information or context that might be helpful for reviewers.]

@asr2003 Can fix the conflicts?

Yes, resolved

This looks fine now.

@asr2003 Submit another PR with fixing the layout issues as discussed


Change handling of supercharged coinbase

The current payout calculator choices are biased toward locked tokens and unfairly pay a portion of a supercharged coinbase to the locked tokens, without sharing a relative supercharged portion of the "normal" coinbase with any unlocked accounts in the pool.
After considering a range of options, it seems a better approach would be to alter that payout to simply reserve the supercharged coinbase in a separate pool that is only paid out to the unlocked tokens. That would change the payout waterfall to:
1 - straight pro-rata on the normal (720) coinbase based on the account balances delegated.
2 - straight pro-rata of the subset excluding O1 Labs and Mina Foundation for the fees generated.
3 - straight pro-rata of the subset excluding 01 Labs, Mina Foundation, and locked accounts for the supercharged remainder (additional 720).
This will make the supercharged payout lumpy for the unlocked accounts, but will also guarantee the locked tokens will always receive their full share.

Can you make it more clear a bit. I remember it should be equal in the long run. If it was favouring locked on how it was doing that.  Wasn't the existing system does the same. What you outlined here?
Also, I have a proposal to just cut out all the fees generated out instead of just the foundation one. My reasoning is I don't believe these fees never reach any significant amount. When there is some high fee like 500 etc. then we will send it back to the owner which mixes up calculation more. :) From the start, I am just suggesting cutting the fees out of the block when they go significantly towards the coinbase because there is always a possibility of 300-500 snark happening on a block. If there is a significant fee increase where snarks are really ending up to something that we had much more competition on snark marketplace which will drive snark prices up and drop the BP's gainfrom it.

It was favoring unlocked. The old waterfall was:
1 - pro-rata on the first 720
2 - supercharged weight on the remainder
That meant that locked tokens were sharing in the reward from unlocked, which they should not. That will not happen with the proposed change - and the unlocked tokens will get a big award when an unlocked block is generated, but will not get a "supercharged" allocation against normal blocks as in minaexplorer's initial implementation.

Another option may be to skip step w, which I think you may be suggesting -- which leaves the fees to the BP?

I don't really think fees are adding up to something or will be some amount significant for a long time. On the other hand, it creates a lot of confusion and complications.
You can pay up to 700 more if it's supercharged for single snark for coinbase tx while the snark fees for transactions can't be higher than the fee user game. It's mostly around 0.01 range which doesn't sum up to any significant amount. Let's say you receive a block with an 1000 mina fee because someone sends it by mistake and you pay it back, Good luck with the calculation. In the future when tx fees increase because of network activity that would increase snark fees as well so even fees increase bp's gain from it never going to be significant.

For your fee proposal, are you suggesting that the BP keep the fees? And if they are negative, payout the pool for the full amount anyway and the pool owner takes a loss? Or something else? Not sure what you mean by ignoring fees.

@jrwashburn my suggestion was since snark fees for transactions are not ending up being something disregarding is probably better. If you hit by a snark fee for coinbase with +5-10 then you are on negative you should calculate that, I am just saying from transactions snark fees I never see any significant amount to even affect something. If snarks price increase then transaction fees going to increase but the majority of these fees go to snark sellers not to block producers.


brs doesn't automatically unbox parameters and return values

RBI automatically boxes function parameters and return values. As it turns out, RBI also unboxes function parameters and return values to match a function's declared signature. Consider this example:
sub main()
    boxed = createObject("roString")
    boxed.setString("lorem ipsum")
    print "type(boxed) = " type(boxed) " | boxed = " boxed

    result = unboxing(boxed)
    print "type(unboxed) = " type(unboxed) " | unboxed = " unboxed
end sub

sub unboxing(s as string) as string
    print "type(s) = " type(s) " | s = " s
    return s
end sub

Inside of the unboxing function, s is always a primitive string and never an instance of roString. Similarly, the return value of unboxing is also always a primitive string and never an instance of roString.
brs currently doesn't handle this demotion properly, and throws type mismatch errors at runtime 😢

Another scenario with this same unboxing issue can be reproduced this way:
a = CreateObject("roInt",1)
? a+1

causes an error:
REPL(1,2-5): Attempting to add non-homogeneous values.
    left: Object
    right: Integer**

In case of a comparison is even worse, as it does not return an error so it silently return false if you compare a roInt with value 1 and the constant 1 like this:
a = CreateObject("roInt",1)
? a
1
? a=1
false


Add todostuslibros.com

This is a website that allows you to buy in many local libraries in Spain

Thank you!


changed app name from 'PathCheck BT' to 'PathCheck'

Description:

The GPS app was named "PathCheck BT".  I changed it to "Path Check"
Linked issues:

Fixes #1051
Screenshots:

How to test:

Go through the onboarding process and you will see the dialog above -- the 'BT' has been removed

Already done


remove ODBC preview binaries

This change is

Codecov Report

Merging #728 into dev will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              dev     #728   +/-   ##
=======================================
  Coverage   79.02%   79.02%
=======================================
  Files          25       25
  Lines        7188     7188
=======================================
  Hits         5680     5680
  Misses       1508     1508

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0de3d8a...c9ea990. Read the comment docs.

Coverage remained the same at 73.334% when pulling c9ea990a493c63ddf89818882b04445e67116d24 on lilgreenbird:nukeodbc into 0de3d8a0f209d62e5d01d5264be96de4edaa75fc on Microsoft:dev.


[launcher] Using runner in dev/PowerLauncher doesn't open launcher on alt+space

I just checked out latest dev/PowerLauncher. I'm running the branch on release x64 with runner as the startup project. PowerToys runs perfectly but hitting alt+space doesn't open the launcher.

Update: I see "Launcher" tab in settings and the launcher is button is also on. I've tried switching the button to off and on again but still doesn't see the launcher on alt+space.

This is due to double compile issue. The problem is fixed when the project is recompiled.


Have the slurm logs save to FSS

Move the default
SlurmctldLogFile=/var/log/slurm/slurmctld.log
SlurmdLogFile=/var/log/slurm/slurmd.log
to FSS

@steven-basart Has this been done?

We might need to move the control demon back if latency gets too high.

@steven-basart What latency are you referring to?


[feature request] Blood Sense

Can we have one that detects creatures that are under 50% hp? It would work roughly the same as detect magic

Yep. And only one character gets it when he casts a specific spell

Similar request (always homebrew), but with more details commisionid and still open for anyone interested https://discord.com/channels/732325252788387980/1127554586622836838

Similar request (always homebrew), but with more details and commisioned on the League Server for anyone interested https://discord.com/channels/732325252788387980/1127554586622836838

This link doesn't give access to the server on which it can be viewed on

Closing. I'll likely won't add another version of Blood Sense than was added.


Picture taken not saving in gallery

Hi again!
I´m still using your library, which is very good, but i´m getting an issue.
Sorry if i´m doing something wrong but i´m new in Android.
When I pick the imagens from gallery, it´s working fine, but when i take a picture from camera, it´s not saving in the gallery.
So when I try to create a file of the picture taken and upload it with retrofit, i´m getting the following issue:
/storage/emulated/0/capture1435779162195.png: open failed: EISDIR (Is a directory)
Below is the code where I get the picture that was taken.
private File photo;
@Override
public void onPickedSuccessfully(String[] strings) {
    photo = new File(strings[0]) // Consider i´m only taking one picture.
}

Regards,
Guilherme Batista

The paramater isn't String[] , but i will consider that as typo
I have intended before to use robust approach for taking photos from camera to solve compatibility problems
Anyway , may I know the system version + The phone name ?

Android 4.4.4 - Motorola Moto X

I got into your documentation and the interface declaration is:
public interface PickListener {
void onPickedSuccessfully(String[] var1);
    void onCancel();
}

So that´s why i´m passing an array of string. I don´t know if i´m using an old version of your library, but below is the dependecy i´m using:
compile 'net.yazeed44.imagepicker:imagepicker:1.0.0'
I also setted the following code in my app's build.gradle file
repositories {
maven { url 'https://dl.bintray.com/yazeed44/maven' }
}

Oh ok then , I still haven't updated the gradle dependacy yet , i will work on a few features then publish update 1.1

Ok, thanks for replay!

And thank you for submitting it here , By the way i would like to hear about your opinion about UX and UI of this library on my email
<EMAIL_ADDRESS>


IPA runtime

Hello,
I am running IPA on a homozygous 11 Gb plant genome and I am wondering if the runtime I have is expected.
The input data is ~24x HiFi reads (19 kb mean size, about 255 GB in a fasta file), the job is running on a single machine (48 CPUs, 1 TB RAM), the command is
ipa local --nthreads 12 --njobs 4 --run-dir Hda_i3 -i lib1_all.fa.gz -i lib2_all.fa.gz
the output folder is 660 GB
It is going since January 2, and the stdout is regularly updated - now it prints this:
>|> 20210115 13:18:35.071 -|- INFO -|- Runner -|- 0x7f9192021740|| -|- Loaded all query blocks in 5 sec / 5.43 CPU sec
>|> 20210115 13:18:35.071 -|- INFO -|- Runner -|- 0x7f9192021740|| -|- About to map query blocks: {43}: num_seqs = 226617
>|> 20210115 13:48:06.545 -|- INFO -|- Runner -|- 0x7f0e65bad740|| -|- Mapped query block in 4439 sec / 52570.3 CPU sec.
>|> 20210115 13:48:07.983 -|- INFO -|- Runner -|- 0x7f0e65bad740|| -|- Loading the query blocks: {43}.
>|> 20210115 13:48:12.078 -|- INFO -|- Runner -|- 0x7f0e65bad740|| -|- Loaded the query SeqDB cache block after 4 sec / 4.1 CPU sec
>|> 20210115 13:48:13.428 -|- INFO -|- Runner -|- 0x7f0e65bad740|| -|- Loaded the query SeedDB cache block after 5 sec / 5.45 CPU sec
>|> 20210115 13:48:13.428 -|- INFO -|- Runner -|- 0x7f0e65bad740|| -|- Loaded all query blocks in 5 sec / 5.45 CPU sec
>|> 20210115 13:48:13.428 -|- INFO -|- Runner -|- 0x7f0e65bad740|| -|- About to map query blocks: {43}: num_seqs = 226617

is there a way to guess how far we are in the assembly?
Also, due to maintenance I will have to kill this job, I will restart it with --resume, but is the assembly going to restart from a checkpoint? E.g. am I going to lose the last 3 days of computation?
Thanks,
Dario

Hi @dcopetti,
Did you start with reads.bam?
Best,
Zev

Hi @dcopetti,
Did you start with reads.bam?
Best,
Zev

Hi @zeeev,
the input file you mean? No, it was two fastas, made concatenating data from multiple SMRT cells of two different libraries (13 SMRT cells in total).
Does that matter, maybe at the polishing step?

Hi @zeeev,
the input file you mean? No, it was two fastas, made concatenating data from multiple SMRT cells of two different libraries (13 SMRT cells in total).
Does that matter, maybe at the polishing step?

I found the mapping stage can take a long time for certain blocks (#357). In some cases, some shards were taking several hundred CPU hours. Repetitive regions seemed to be the suggested cause, so it might be even more noticeable here.

I found the mapping stage can take a long time for certain blocks (#357). In some cases, some shards were taking several hundred CPU hours. Repetitive regions seemed to be the suggested cause, so it might be even more noticeable here.

@ASLeonard, minimap2 (via pbmm2) is no longer used, and that was the known reason for slow-down on repetitive regions. If the environment is up-to-date, this must be a different problem.

@ASLeonard, minimap2 (via pbmm2) is no longer used, and that was the known reason for slow-down on repetitive regions. If the environment is up-to-date, this must be a different problem.

@pb-cdunn, the latest pbipa available is still v1.3.2 from October in the repo and conda. Is the version without pbmm2 available elsewhere?

@pb-cdunn, the latest pbipa available is still v1.3.2 from October in the repo and conda. Is the version without pbmm2 available elsewhere?

Hi @dcopetti,
I'm not surprised that it's taking a long time to complete, considering it's a pretty large genome running on a single 48 core machine. Also, it very much depends on the repeat content of the genome.
As an example, for the Redwood genome (~26 Gbp 1N genome size), the overlap stage of IPA took 28625 CPU hours.

is there a way to guess how far we are in the assembly?

If you are still running the overlapping stage (like in your snippet in the original message), you might be able to estimate the remaining overlap time based on the completed overlap steps.
Run the following to see how many shards there are in total for overlapping:
$ wc -l RUN/03-ovl_asym_prepare/shards/all_shard_ids

Then, take a look in the RUN/04-ovl_asym_run/ folder and check out how many non-empty files called log.ovl_asym_run.sort.memtime you can find. The sort happens once all overlap runs in a shard are done.
In any of those shard folders which were fully done, there are one or more log files called "log.ovl_asym_run.pancake.*.memtime". Those contain time measurements for every Pancake run in this folder. If you sum up all the real time values, you'll get a general estimate of how long it took for a single shard to complete.

Also, due to maintenance I will have to kill this job, I will restart it with --resume, but is the assembly going to restart from a checkpoint? E.g. am I going to lose the last 3 days of computation?

If you resume the job after it was stopped, the assembly will not recompute any of the previously completed jobs. (For example in overlapping, any shard which was done will not be rerun, but those shards which were partially done before the process was killed will start from scratch).
P.S. Did the job perhaps finished running in the meantime?
Best regards,
Ivan.

Hi @dcopetti,
I'm not surprised that it's taking a long time to complete, considering it's a pretty large genome running on a single 48 core machine. Also, it very much depends on the repeat content of the genome.
As an example, for the Redwood genome (~26 Gbp 1N genome size), the overlap stage of IPA took 28625 CPU hours.

is there a way to guess how far we are in the assembly?

If you are still running the overlapping stage (like in your snippet in the original message), you might be able to estimate the remaining overlap time based on the completed overlap steps.
Run the following to see how many shards there are in total for overlapping:
$ wc -l RUN/03-ovl_asym_prepare/shards/all_shard_ids

Then, take a look in the RUN/04-ovl_asym_run/ folder and check out how many non-empty files called log.ovl_asym_run.sort.memtime you can find. The sort happens once all overlap runs in a shard are done.
In any of those shard folders which were fully done, there are one or more log files called "log.ovl_asym_run.pancake.*.memtime". Those contain time measurements for every Pancake run in this folder. If you sum up all the real time values, you'll get a general estimate of how long it took for a single shard to complete.

Also, due to maintenance I will have to kill this job, I will restart it with --resume, but is the assembly going to restart from a checkpoint? E.g. am I going to lose the last 3 days of computation?

If you resume the job after it was stopped, the assembly will not recompute any of the previously completed jobs. (For example in overlapping, any shard which was done will not be rerun, but those shards which were partially done before the process was killed will start from scratch).
P.S. Did the job perhaps finished running in the meantime?
Best regards,
Ivan.

Hi Ivan,
Thank you for your feedback.
Unfortunately, it looks like we are still a long way to complete this step. There are 40 all_shard_ids files and 24 log.ovl_asym_run.sort.memtime files. A set of 4 of them gets produced every 3 days approximately.
I started IPA on this genome since when I assembled it with hifiasm it took three days, so I was hoping for a similarly quick turnaround. I will let this assembly go for now, but I will kill it (and maybe restart it later) if I get more urgent things to run.
Are the steps after 04 still very computationally demanding, or are they faster like e.g. in FALCON?
Do you have any comparison (or approx idea) of how IPA would compare in terms of assembly stats with hifiasm for a plant genome (maize, barley, or something of this level of complexity)?
Thanks,
Dario

Hi Ivan,
Thank you for your feedback.
Unfortunately, it looks like we are still a long way to complete this step. There are 40 all_shard_ids files and 24 log.ovl_asym_run.sort.memtime files. A set of 4 of them gets produced every 3 days approximately.
I started IPA on this genome since when I assembled it with hifiasm it took three days, so I was hoping for a similarly quick turnaround. I will let this assembly go for now, but I will kill it (and maybe restart it later) if I get more urgent things to run.
Are the steps after 04 still very computationally demanding, or are they faster like e.g. in FALCON?
Do you have any comparison (or approx idea) of how IPA would compare in terms of assembly stats with hifiasm for a plant genome (maize, barley, or something of this level of complexity)?
Thanks,
Dario

Hi @isovic ,
Eventually, step 04 completed - all 40 log.ovl_asym_run.sort.memtime files have stats on the time it took.
Now I found that the job stopped because there was not enough space:

As of now, I see that where I am working I have 13 TB free:
20T  7.3T   13T  37% /scratch
do you think that indeed step 05 took all that space? I did not have anything else running at the same time.
This is how the run looks like now:
(ipa_env) bash-4.2$ ls -lrth Hda_i3/
total 5.0K
-rw-r--r--.  1 copettid mpb  104 Jan  1 22:00 input.fofn
drwxr-xr-x.  2 copettid mpb 4.0K Jan  1 22:00 qsub_log
drwxr-xr-x.  2 copettid mpb 4.0K Jan  1 22:00 01-generate_config
drwxr-xr-x.  2 copettid mpb 4.0K Jan  1 22:50 02-build_db
drwxr-xr-x.  3 copettid mpb 4.0K Jan  1 23:08 03-ovl_asym_prepare
lrwxrwxrwx.  1 copettid mpb   10 Jan 18 08:21 assembly-results -> .snakemake
-rw-r--r--.  1 copettid mpb  218 Jan 18 08:21 config.yaml
drwxr-xr-x. 42 copettid mpb 4.0K Feb  3 17:04 04-ovl_asym_run
drwxr-xr-x.  2 copettid mpb 4.0K Feb  6 18:13 05-ovl_asym_merge
-rw-r--r--.  1 copettid mpb 1.5K Feb  6 18:13 ipa.log
(ipa_env) bash-4.2$ ls -lrth Hda_i3/05-ovl_asym_merge/
total 1.0K
-rw-r--r--. 1 copettid mpb 1.4K Feb  6 18:09 sorted.fofn
-rw-r--r--. 1 copettid mpb 1.6K Feb  6 18:13 log.ovl_asym_merge.mergesort.memtime
(ipa_env) bash-4.2$ du -hs Hda_i3/*
3.0K    Hda_i3/01-generate_config
184G    Hda_i3/02-build_db
22K     Hda_i3/03-ovl_asym_prepare
1.1T    Hda_i3/04-ovl_asym_run
1.5K    Hda_i3/05-ovl_asym_merge
512     Hda_i3/assembly-results
512     Hda_i3/config.yaml
512     Hda_i3/input.fofn
512     Hda_i3/ipa.log
512     Hda_i3/qsub_log

Do you think that if I resume the run again I will end up in the same space issue? Are there files that I could delete from folder 04?
More in general, after more than a month (vs 3 days with hifiasm), I wonder if I should still continue with this assembly for this genome.
Any thoughts of if and how to continue?

Hi @Dcopetti,
I'm glad to hear that the overlap stage completed!
There is a chance I can make if faster in the future by skipping some obvious local alignments. These repetitive plant genomes are slow to overlap because of those.
Based on my previous experience, the phasing stage should hopefully be faster than overlapping for genomes like this.
I'm surprised that HifiAsm managed to complete the entire assembly in only 3 days. That's very impressive.
On the other hand, an advantage of IPA is that it's designed to run on a cluster where it can leverage larger concurrency.
Also, memory consumption of IPA should be an order of magnitude lower than HifiAsm for genomes like the one you're currently running (even when running the job on single small local nodes).

Now I found that the job stopped because there was not enough space:

The Linux sort command uses a temporary directory to sort files, and by default it uses the /tmp directory. Sometimes the /tmp can run out of space (for example, other users/processes may have consumed it, or someone hasn't cleaned up after themselves).

do you think that indeed step 05 took all that space?

No, that's fine. The /tmp is probably not mounted on your /scratch, otherwise you'd have plenty of space.

Do you think that if I resume the run again I will end up in the same space issue? Are there files that I could delete from folder 04?

You can definitely resume it, but it's important that you specify a different temporary directory first to prevent the same issue from happening again.
For example, when running an assembly, use the following option:
ipa --tmp-dir "./" --resume ...
This will set the temporary directory to the current running directory of the task instead of the default /tmp that the standard sort command uses.
Before you run this, I just want to verify that your RUN/04-ovl_asym_run/*/ folders still contain overlap files (those are marked as temporary in our Snakemake workflow and should get removed by Snakemake only after the 05-ovl_asym_merge stage is successfully completed.
For example, any folder in RUN/04-ovl_asym_run should look something like this:
$ ls RUN/04-ovl_asym_run/0/
log.ovl_asym_run.pancake.s0.b0_0_1.memtime log.ovl_asym_run.sort.memtime ovl.sorted.m4

(with a ovl.sorted.m4 file in it.)

More in general, after more than a month (vs 3 days with hifiasm), I wonder if I should still continue with this assembly for this genome.

If possible, I would definitely be interested to see this genome being fully assembled by IPA.
Based on your feedback I see at least two improvements we can make so far:

Set the default temp directory to "./".
Improve overlap speed for very repetitive genomes by prefiltering some clear local alignments.

Thank you!
Best regards,
Ivan.

Hi Ivan,
Thank you for the feedback,
Regarding step 04, inside 04-ovl_asym_run/*/ I have 40 ovl.sorted.m4 and log.ovl_asym_run.sort.memtime (same number as the folders) files, but 101 *memtime files (there are from 1 to 10 such files in each - is that OK?)
Do I need to delete fold 05?
I can definitely change the tmp directory when I resume this assembly, however I have to wait a bit since I have jobs with more priority going on now. I will get back to you with the final assembly.
As for the comparison with hifiasm, I have to correct myself: the assembly took 3 days on a machine with 96 CPUs (vs 48 here) - equivalent to 6 days on the machine where I am running IPA. Still a clear difference though.
Cheers,
Dario

V1.5.0 is publicly available now.


Unable to perform several customization

Trying to customize the view of calendar. Unable to change as simple as height and width of calendar. It takes whole of my page. Tried applying css width to few divs on DOM, it works but height still do not work. Can't a general object be added to directive which would define its layout like:
{
height:
width:
color:
}
It was easier in original code to change tmpls directly. Seems a bit harder here. Also will it be possible to show events in month view similar to how they are shown in week view with strips. Kindly guide the way to customize... I have a question for the same here. would appreciate  an answer..
http://stackoverflow.com/questions/38636447/how-to-customize-angular-based-bootstrap-event-calender

Wrapping the mwl-calendar component in a simple <div style="max-height: 200px; overflow: scroll"> allowed me to easily change the height of it: http://plnkr.co/edit/E5vArN9FKysU9bXGd1YB?p=preview
For custom templates there is an example on how to change them here: https://mattlewis92.github.io/angular-bootstrap-calendar/#?example=custom-templates
The source templates are available here: https://github.com/mattlewis92/angular-bootstrap-calendar/tree/master/src/templates

Thanks Matt,
I  Tried the suggested approach and was able to customize my year view. However, now on click the month in year view the slide window do not appear. I wanted all the months in a straight row, instead of default view. So I removed the ng-repeat ="rowOffset......", and got the desired view. I assume it has nothing to do with slider. The ng-click of div is still intact. Below is my code, can u have a look and suggest what might have caused slider to disappear.

I think it's because openRowIndex isn't set, try removing vm.openRowIndex === $index from the is-open logic on the slidebox

Great..that works, but how did it get affect?

Is its logic based on number of rows created or something

Yup, as you've removed the rows it's no longer relevant: https://github.com/mattlewis92/angular-bootstrap-calendar/blob/master/src/directives/mwlCalendarYear.js#L47

Okay..will it be safe to assume that if I customise my views for single row (month views too), removing this condition will suffice ?

Yeah probably

On 29 Jul 2016, at 10:45, Saurabh<EMAIL_ADDRESS>wrote:
Okay..will it be safe to assume that if I customise my views for single row (month views too), removing this condition will suffice ?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

Great...thanx for the guidance..and BTW great job !!


Leaked FileInputStream

https://github.com/ConsenSys/IPFS-Store/blob/987558dffac4fc38751d2e4c5b629902a4f827b8/ipfs-store-client/ipfs-store-client-java/src/main/java/net/consensys/tools/ipfs/ipfsstore/client/java/IPFSStore.java#L81
A FileInputStream is created but never closed.

Thanks for this, good catch! It's fixed with PR #40 (release version: 0.1.7)
Feel free to reach out if you have questions or feedback.


Enable direct request, e. g. per Environment variable

When building the docker image a feature was enabled per
sed -i 's/#directrequest:<value>/directrequest:enable/' /opt/OCR4all_Web/submodules/LAREX/Larex/src/main/webapp/WEB-INF/larex.config

This is not done anymore while building the docker image, but seems to be neccessary for certain functionality.

I'm currently working on it.
The feature is included and I'm currently about to test everything.
The change should be live in the next hour.

Larex now supports an environment variable to define define another path to the larex.config.
Larex automatically tests for a "LAREX_CONFIG" variable and checks if it points to an existing file.
It will, if the file exists use that configuration file instead of the default file in "LAREX/src/main/webapp/WEB-INF/larex.config"
The changes are active starting with cc86b40


Clee/edxwebview

PR exploring comment made in https://github.com/edx/edx-app-android/issues/900.
Seems like the EdXWebview is working fine for Find Courses.

Current coverage is 17.33% (diff: 50.00%)

Merging #901 into master will increase coverage by <.01%

@@             master       #901   diff @@
==========================================
  Files           467        467
  Lines         19619      19620     +1
  Methods           0          0
  Messages          0          0
  Branches       2881       2881
==========================================
+ Hits           3401       3402     +1
  Misses        15832      15832
  Partials        386        386

Powered by Codecov. Last update a7b66e0...8f434b2

👍 Looks got but has merge conflicts


Snapping allows placement of invalid positions

Description
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

drag one snake over the drop zones of another animal
drop the snake, while it is still touching the drop zone, but with the dragged snake being on the outmost part of a grid square
snake will snap to outer grid square (outside of the drop zone), but will remain there and not snap back

Expected behavior
Snake snaps to outer grid square and resets to original position, as the snapped position is not valid.
Screenshots
https://github.com/mango-gremlin/arch-enemies/assets/104799849/6de7d2c6-f273-405a-911f-02209e5831e5
Solution Proposal
The problem arises, because the snap to grid happens in the _process function, so the variables is_inside_dragable and is_inside_forbidden are not updated, as the dragged animal hasn't moved. Everything occurs in the same frame, so no update is possible. To fix this, maybe use signals emitted by events to handle the snap to grid, so it occurs in different frames (or rather, before _process is automatically called)

probably this happens because of your grid snapping, which rounds to the nearest 10's coordinate right? and if its on the rim it thinks the nearest coordinate is outside of the dropzone?
maybe we can edit the snap to grid function to include this case, which should get around the problem of the stuff happening at the same time?


Wiki - Overall architecture page outdated

This page:
https://github.com/codalab/codabench/wiki/Overall-Architecture
seems to be obsolete. For now I simply remove the link to it from the wiki's homepage.

Deleted.


[OSD-19745] - Make OCM Agent Highly Available

What type of PR is this?
(feature)
What this PR does / why we need it?
Update the OCM agent deployment configuration to tolerate disruptions, such as infra nodes being terminated to make OCM Agent Highly Available
Which Jira/Github issue(s) this PR fixes?
https://issues.redhat.com/browse/OSD-19745

Codecov Report

Merging #102 (89532f9) into master (c83dbdd) will increase coverage by 0.40%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #102      +/-   ##
==========================================
+ Coverage   66.28%   66.69%   +0.40%
==========================================
  Files          19       19
  Lines        1234     1249      +15
==========================================
+ Hits          818      833      +15
  Misses        349      349
  Partials       67       67

Files
Coverage Δ

pkg/ocmagenthandler/ocmagenthandler_deployment.go
90.97% <100.00%> (+0.49%)
:arrow_up:

/hold - Need to add PDB in operator's deployment

Pretty much LGTM if tested!

Local test succeed: https://gitlab.cee.redhat.com/-/snippets/7462

/unhold - PDB added and tested against a staging cluster and works as expected

/unhold

/lgtm
/hold

/lgtm

/lgtm
/unhold


When using flakes, jetpack-nixos ignores kernelPatches in top level config

I built a standalone example here where I create a nixosConfiguration that add's an extraConfig kernel patch but it doesn't get applied when I build the kernel.
To reproduce this do the following:
git clone git@github.com:therishidesai/test-nixos.git
cd test-nixos
nix build .#nixosConfigurations.test-jp5.config.system.build.kernel.configfile

This build should fail because the kernelPatch describes a file that doesn't exist, but this patch is never being applied.

https://github.com/anduril/jetpack-nixos/pull/56
Fixes this issue


Create N-queens.py

As you requested, link:-https://leetcode.com/submissions/detail/749548614/

Closes #8


Reload after dismiss

Hi all,
Sorry for my funny english, i'm french and i don't speak very well…
In my website, i wanted the page reloads after accepting, but i'm not a big develloper…
Well, i just added this followed inline into the cookieconsent-min.js window.location.reload();
Before s.setCookie(n,"yes",this.options.expiryDays,this.options.domain,this.options.path);
After s.setCookie(n,"yes",this.options.expiryDays,this.options.domain,this.options.path);window.location.reload();
I think is not a good parctice, but it's work ;)
But i think this option can added to the script, maybe, if you want ;)

That should work, but can I ask why you'd want to reload after dismissing?

In my case, it's for displaying my customer login form, after refresh and cookie verification.
The form should be displayed if cookies are enabled and accepted.
It's a rule for my customer, and this script is the more simple i have found who correspond.
It's not the original task, but for me, it's good.
My submission is just an information ;)

In CookieConsent V3, this functionality could be easily added in a callback. I.E.
cookieconsent.initialise({
  onStatusChange: function(status) {
    if (this.hasAnswered() && this.hasConsented()) {
      window.location.reload();
    }
  },
});


[main] Upgrade to latest dependencies

Cron -knative-prow-robot
/cc knative-sandbox/networking-wg-leads
/assign knative-sandbox/networking-wg-leads
Produced by: knative-sandbox/knobots/actions/update-deps
Details:
bumping knative.dev/hack 0456e8b...e178598:
  > e178598 Update community files (# 157)
  > cf8cbc0 Fix the regex used for matching GitHub Actions files (# 156)
  > a8ef7b6 Skip presubmit tests for .github updates (# 154)
bumping knative.dev/networking 48ac02f...1934422:
  > 1934422 upgrade to latest dependencies (# 622)
  > 17e1877 upgrade to latest dependencies (# 621)
bumping knative.dev/pkg d481724...708dc1c:
  > 708dc1c Update community files (# 2443)
  > 062ca30 upgrade to latest dependencies (# 2433)
  > 4fcbc1b Update actions (# 2434)
  > 80c511a Wait for reconciler/controllers to return prior to exiting the process (# 2437)

/retest

/retest
strange error.
failed to try resolving symlinks in path "/var/log/pods/test-pods_5f159fb8-9907-11ec-a174-921801d91a0f_6d9e63b0-1632-46ac-9bf4-a2495aeb7bfc/test/0.log": lstat /var/log/pods/test-pods_5f159fb8-9907-11ec-a174-921801d91a0f_6d9e63b0-1632-46ac-9bf4-a2495aeb7bfc/test/0.log: no such file or directory

/lgtm
/approve


Hide not owned repos from sidebar and repo list

closes #971 , closes #1454
closes #1436
fixes: https://codeberg.org/Codeberg-CI/feedback/issues/28
Notes for testing:
check if the user gets pipeline updates (not only the sidebar, but most importantly new pipelines and updates to the workflows and steps)

for own private projects
for org private projects
for public projects the user does not directly own, but is allowed to see
for public projects as unauthenticated users

Tested:

[x] for own private projects
[x] for org private projects
[x] for public projects the user does not directly own, but is allowed to see
[x] for public projects as unauthenticated users

Findings:

Org private projects do not appear in the feed
Woodpecker admin sees all repos/pipeline updates

@anbraten

when starting on route /org/repo the feed is empty and doesn't receive updates. When starting on /repos the feed shows pipelines and receives updates.

Strange. It should be have a pipeline entry to add the step updates to, but I will investigate on that.

Woodpecker admin sees all repos/pipeline updates

Are you sure that is the case? The filter I've added only shows pipelines which match the names from GetRepos and that function uses RepoList from the store which is not checking for admin permission as far as I can tell.
@lukashass

@lukashass Kindly reminder

Are you sure that is the case? The filter I've added only shows pipelines which match the names from GetRepos and that function uses RepoList from the store which is not checking for admin permission as far as I can tell.

@anbraten Well, just tried this again and can't reproduce. I guess I messed something up before.
The other thing is still an issue though!

@lukashass Thanks for testing. The problem was that the repo list was not loaded on some pages. Should be fixed now.


Improve caching Gradle state on CI

https://slack-chats.kotlinlang.org/t/12918775/i-m-wondering-what-i-m-doing-wrong-when-it-comes-to-caching-

Please don't wonder, I replaced the Slack-link by a linen-link, as it then is not bound to the 90 day limit and also can be looked at without starting / using Slack.

Let's try enabling configuration cache as well.

Configuration cache problems found in this build.

4 problems were found storing the configuration cache, 2 of which seem unique.
- Task `:library:validateDuplicatedVersion` of type `org.gradle.api.DefaultTask`: cannot serialize Gradle script object references as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:library:validateDuplicatedVersion` of type `org.gradle.api.DefaultTask`: invocation of 'Task.project' at execution time is unsupported.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution

Removing references to project (at least the explicit ones) is easy. However, I have a problem understanding the other issue regarding serializing Gradle script object.

You access $version in the task action.
This has an implicit reference to the script instance as that has the property you are reading.
IF you add a val version = version within the task configuration closure but outside the doLast { ... } should be enough to fix the problem as then only the value is captured, not a reference to the script instance.

I started adopting the project to work with Gradle configuration cache. Some still issues appear - let me wait the full release cycle (2 weeks) to give a chance of all relevant workflows to run, and fix the build logic if needed.

Publishing is broken: https://github.com/typesafegithub/github-workflows-kt/actions/runs/6987650312/job/19014417902
15 problems were found storing the configuration cache, 6 of which seem unique.
- Task `:closeSonatypeStagingRepository` of type `io.github.gradlenexus.publishplugin.CloseNexusStagingRepository`: cannot serialize object of type 'org.gradle.api.DefaultTask', a subtype of 'org.gradle.api.Task', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:task_access
- Task `:closeSonatypeStagingRepository` of type `io.github.gradlenexus.publishplugin.CloseNexusStagingRepository`: cannot serialize object of type 'org.gradle.api.internal.project.DefaultProject', a subtype of 'org.gradle.api.Project', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:initializeSonatypeStagingRepository` of type `io.github.gradlenexus.publishplugin.InitializeNexusStagingRepository`: cannot serialize object of type 'org.gradle.api.DefaultTask', a subtype of 'org.gradle.api.Task', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:task_access
- Task `:initializeSonatypeStagingRepository` of type `io.github.gradlenexus.publishplugin.InitializeNexusStagingRepository`: cannot serialize object of type 'org.gradle.api.internal.project.DefaultProject', a subtype of 'org.gradle.api.Project', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:disallowed_types
- Task `:releaseSonatypeStagingRepository` of type `io.github.gradlenexus.publishplugin.ReleaseNexusStagingRepository`: cannot serialize object of type 'org.gradle.api.DefaultTask', a subtype of 'org.gradle.api.Task', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:task_access
- Task `:releaseSonatypeStagingRepository` of type `io.github.gradlenexus.publishplugin.ReleaseNexusStagingRepository`: cannot serialize object of type 'org.gradle.api.internal.project.DefaultProject', a subtype of 'org.gradle.api.Project', as these are not supported with the configuration cache.
  See https://docs.gradle.org/8.4/userguide/configuration_cache.html#config_cache:requirements:disallowed_types

We either fix till the next release, or ignore it: https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:usage:ignore_problems

I also noticed that on CI, the cache looks like it isn't preserved. I see this all the time:
Calculating task graph as no configuration cache is available for tasks: build

Workdlows that usually run periodically work fine:

https://github.com/typesafegithub/github-workflows-kt/actions/runs/6988312072
https://github.com/typesafegithub/github-workflows-kt/actions/runs/6988312448

We either fix till the next release

You cannot, until you wait for Gradle 9 and the release of the nexus publish plugin following that one: gradle-nexus/publish-plugin#221

or ignore it:

Don't do that. This setting is not something you should use in production. It is just to find some more issues while the build is not yet fully CC compatible. Also, setting it will in many cases not mean that the build suddenly will work, it often just fails later and harder. Just don't try to use CC on publishing currently, on GHA it is not reused anyway.

I also noticed that on CI, the cache looks like it isn't preserved. I see this all the time:

Correct, the Gradle build action does not preserve the configuration cache.

Requested adding support for configuration cache state storing: https://github.com/gradle/gradle-build-action/issues/966

The code is already there, just the call is commented out. I guess it does not yet work properly as the cache is not yet relocatable

I pushed it as far as possible right now. Let's resolve it for now.


PR gets stuck when there are only document changes

When PR has only document files changed, the CI gets stuck.
"build"-s are not supposed to be triggered but they are.
I am not sure if it related to the recent change with "licence/cla" action.

Two examples can be found here:

https://github.com/shader-slang/slang/pull/5659
https://github.com/shader-slang/slang/pull/5672

It is because that CI ignores changes restricted to the docs directory, this is safe to ignore. Or we can unconditionally run such workflows

Why does the build workflows are triggered in the first place?
The label checking workflow seems to be doing fine as expected in a way that it is not triggered at all.
They seem to have the same path-ignore settings.

The tests are showing up because the repo is configured to expect these tests to pass, when in fact they are not triggered. This is just a github display issue, and no tests are actually stuck.


Bigint and bitfield-as-integer

Adds bigint and bitfield support to SSZ.
Possible alternative definition for Bitfield:
o = [0] * (len(value)//8 + 1)
for i in range(len(value)):
    o[i//8] ^= value[i] << (i%8)
o[len(value)//8] ^= 1 << (len(value) % 8)
return bytes(o)

Is there a good reason for why we want to support Bigints? Otherwise I would tend to favour the alternative definition over introducing Bigints just for defining Bitfields. While the definition is more elegant with Bigints, I think in reality all implementations will use this.

If we want to use SSZ as an ABI later on, then there will definitely be use cases for bigints, so there's arguably a reason to have them in there. But you're right, it's not necessary for consensus layer.

I think it makes sense to add this, we've already opted for union types on similar grounds. Also, I quite like the bigint definition like this because where it will come very natural is in MPCs where it is very natural to operate on Bitfields in this way.
I tried adding the Bigint and Bitfield to the other sections of the document. Clearly they should no be a basic type as they are variable length. Not entirely happy to classify them as "composite", if someone has a better name ("Helper type" to indicate that it is only a thin layer on top of "bytes"?) maybe we should split this section.
I suggest capitalising these types, in fact I would suggest capitalising all non-basic types (at the moment this is inconsistent).

Another argument for the bigint types was to be able to handle deeper SSZ objects in generalized merkle indices ( https://github.com/ethereum/eth2.0-specs/issues/842#issue-425567448)

Oh, and this request would basically only support fixed size Bitfields, right? Some of the potential applications in the spec seem to be fixed size, should we add support for this too?

This looks good. We should modify the justified_bitfield type before merging

As per our discussion yesterday, added a fixed length Bitfield. Not really sure about the notation I'm using; some alternatives:

Abuse the list/vector notation and use ["bool"] and ["bool", N] for variable/fixed length bit fields. Then there would be no way to do get them in the "one byte per bool" version anymore (but do we even need that?) -- I guess this is very close to what @JustinDrake suggested on the call, at least in terms of notation
Alternative names for variable/fixed length bitfields if not abusing notation:

VariableBitfield and FixedBitfield[N] (my current suggestion)
Bitlist and Bitvector[N]
Bitfield and Bitifeld[N] (I found this a bit "dirty")

Now that #1077 is merged and we have Vector[123] and BytesN[123], I don't think Bitfield[123] is very strange. The bad part however is that we don't want Bitfield to be of the same type as Bitfield[42]. So I prefer the Bitlist + Bitvector[N] naming.
My recommendation would be to not push it to the SSZ layer. And instead, just implement the data types as subclasses of bytes and BytesN[X]. And then make the item lookup function and concatenation deal with the i / 8 to map it into bytes. This way neither the spec nor the ssz implementation will be affected :)

Just a quick nitpick: the code is full of off-by-1 errors (missing the partially used byte). E.g. log2(x)//8, should be (a.bit_length() + 7) // 8, but yes, it's less pretty. It is probably a good idea to just write the conversion function once, and re-use that.

@dankrad Can we close this in favor of your coming PR?

Yes, please :)


How to exclude a folder?

How would I exclude a folder in the scan?

You can add an "allowed" pattern that excludes specific directories via the regular expression. The lines scanned when filtering out allowed patterns is formatted as follows:
filename:line:text

Where filename is the matched file (e.g., ./foo/bar.txt), line is a line number (e.g., 124), and text is the line of the file that matched one of your prohibited patterns (e.g., "this is the text").
You can add an allowed pattern that can filter out directories and files using regular expressions. This will exclude all matches in "somedirectory/name":
git secrets --add --allowed '^\./somedirectory/name:.*'

I'm a bit stuck here. I have a dependency in the node_modules directory which is causing a lot of false positives. However I change or update your example it doesn't work for me. Using your regex adapted to ^\./directory/relative/to/root:.*' still gives errors in the specified directory.
Ideally I want to exclude all  all node_modules directories and all subdirectories occurring anywhere in my project. A simple regex such as '*node_modules/*' looks like it should do the job but it doesn't work either. Anyone an idea of how to solve this?

@olbrew Did you solve the issue?


Feature/notification

Codecov Report

Merging #138 (a6ee144) into master (6508437) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6508437...a6ee144. Read the comment docs.


Run CI on PRs

My bad for not testing #185, I'll see if I can figure out why it breaks.

Nice, thanks!


Minor update pe-format.md

Corrected link to IAT section (not delay IAT)

@j04n : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Sparkle.Framwork missing as a dependancy

I tried adding this, but cant seem to get it to link (Its been a while since Ive used Xcode though)

The README says this:
If you don't see the Sparkle submodule, do a git submodule update --init --recursive.
Did you do that?

nope... again sorry, I should have read the readme


Improve push origin command

Be sure your local code will be pushed into your remote.
someday ☀.

+1 to this feature. The git pushdemont should be enforced.


Adding and then sorting an array breaks repeat binding

When adding (pushing) to an array and then sorting the result, a repeat binding doesn't update correctly.
I have created a repro here. I have added a repeat binding over a collection to the Welcome view model. There are a couple buttons below, one to add then sort and one to add without sort. If you use add with sort, you will see the unexpected behavior.

i am facing the same issue

Ye, this is a huge problem for me as well :/ I can't to this.somearray[0] = something, since it is not reflected in the views. On the other hand, pushing works just fine. this.somearray.push(something) works... I hope this will be fixed quickly

@BruceL33t your issue is different than what @simonfox reported.  Aurelia cannot observe indexed assignment of an array- at least not without Object.observe, which is no longer being championed in TC39.  As long as you use methods on the array instance to mutate the array it will work as you expect.  In your case, change this.somearray[index] = something to this.somearray.splice(index, 1, something).

I was updating my project to the latest release and also faced this issue (made a simple plunkr repro: http://plnkr.co/edit/k3SrDHFKt9x0Wf0O5Ect). Is there an easy workaround for this, and is there a fix planned?

This is what I'm experiencing when doing push and sort on the underlying array:
The repeat gets two change records. One for adding one view (with index inside the bounds of the array). Another for removing one view, on max index +2.
view-slot before making any changes

Second splice index off by +1?

@jdanyow / @EisenbergEffect Any ideas? Potentially looks like the splices are calculated wrong somewhere in calcSplices.

I think @simonfox found a case where this is needed.

@jdanyow Concatenating  changeRecords and records did fix the error. But the sorting in the DOM is off.
On this line I added this:
if (changeRecords) {
  records = changeRecords.concat(records);
}

Here is a failing repeat integration test.
Let me know if you have any ideas. I can continue on this tomorrow.

@jdanyow @martingust I wanted to push out a release today or tomorrow since we've built up a number of fixes. Any idea if this could be fixed by tomorrow? I just want to know if I should wait or not.

@EisenbergEffect I will continue look into this tomorrow, but I have no idea right now what the issue would be so I can't say if it will be fixed or not by then.

No problem. The master is safe to release as is though, correct?

yep- all tests passing (other than this one)

Ok, great. I'll get a release out and then we can get this one in as soon as it's done in a later patch.

@EisenbergEffect @jdanyow  I continued trying to trace this down. This is what I have found so far:
The bug happens when doing push and (certain) sort on arrays when the repeat's views require lifecycle. As in earlier comment I tried to concat records with existing records here that fixes the error but the existing records gets duplecated in the viewSlot.
 // initial array
 var foo = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'];

 // doing push and sort
 foo.push('10');
 foo.sort((a, b) => {});

console.log(foo); // ["0", "10", "2", "3", "4", "1", "6", "7", "8", "9", "5"]

The repeat gets 3 splices, where each splice have 1 add and 1 delete. First the views get deleted. The first two splices correctly deletes at index 1 ('1') and index 5 ('5') third wants to delete at index 10 (outside array), and error is thrown.
If doing an ugly hack, to skip deleting if index is outside of array, the repeat processes and adds all views to be added, correctly.
In other words; the problem (as I see it) is with the third splice, which should not contain any deletes, just 1 add. I have looked through array-change-records.js without any luck figuring out if something goes wrong here or not. Wild guess is that something is going wrong in there, when calculating the edit distances, perhaps something to do with this comment. I'm not very familiar with this code and not making much progress from here, would need you guys to take a look.

@martingust I merged in the integration test you put together and added logic to the array observation to force a flush of the change records prior to doing an array operation that requires a "reset" (sort or reverse).  Downside is this will cause DOM changes outside of the task queue but I couldn't think of a way to get around it.

@jdanyow Awesome. I think we can live with that, repeat performance is pretty good in any case.

@EisenbergEffect any chance of getting a patch release for this fix this week?

Yes, I'm waiting to hear about a couple of other issues.

Awesome thanks @EisenbergEffect

I'm coming across a similar issue and can't get it to work. In an array (this.steps): I need to move an item from a position to another (index previousPosition to newPosition). This array is used in a repeater in the view.
Here are the relevant parts of the code I've tried:
steps.js (VM)
// Option 1 (using splice)
this.steps.splice(newPosition, 0, this.steps.splice(previousPosition, 1)[0]);

// Option 2 (using sort)
let index = 0;
for(let step of this.steps) {
  step.position = (index < newPosition) ? index : index+1;
  if(index == previousPosition) {
    step.position = newPosition;
  }
  index++;
}
this.steps.sort((a,b)=>{
  return a.position - b.position
});

steps.html
<li repeat.for="step of steps">...</li>

Both options have problems when the view gets updated. Sometime the moved item gets duplicated, sometimes the array gets an empty new item, sometimes an item gets missing, ...
Is this stil a bug in the binding system or am I doing something wrong when updating the array ?

@jdanyow @martingust Can you guys look into this?

Experiencing the exact issue @martingust described in detail here earlier.

I'm seeing this also. More generally I'm seeing Aurelian fail to maintain the array order with the order on screen if I do many splices and pushes in a row. :(

Oh and latest version here too.

My perspective on the importance of this bug is that Aurelia has never been able to maintain order of my lists in the view due to various bugs, two of which have been fixed but this one remains.
My website is all about sorting things by score. In the year I've been using Aurelia it's never been fit for this simplest of purposes. That coupled with dealing with so many bugs and a lack of documentation has left me feeling extremely regretful about my choice.


[stable/sentry] Add some values in .Values.email as environment to pod

values.yaml provides varibles email.from_address and email.use_tls but they hasn't been added as environments to pod yet. This PR fix that.

/assign @unguiculus


Add Caesar Cipher Card

This PR adds a new card as described in this issue #28

Good work @tr1ten!


[added] IndexRedirect

Closes #1966
I removed the example because it would be no longer relevant/correct if this component is added.

De-staled the PR (fixed merge conflict).

Thanks, @taion! :D
I rebased and committed in ca9e3b74a9be05b8b887a03b41c92d6ef1877298

BTW, my apologies for taking so long to take a look at this. You're welcome to be a collaborator if you'd like a little more sway in getting things through :)

I'd love to help if you think I can. Thanks for merging this.


⚠ Bofrost.lu has degraded performance

In c71b9ba, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1203 ms

Resolved: Bofrost.lu performance has improved in 27ce857 after 6 minutes.


Set near/far camera clipping distance

🎉 New feature
Closes #308
Summary
Added ability to configure the user camera's near/far clipping distance in the MinimalScene plugin from xml. If none is set, the default near 0.01 and far 1000 is used.
Test it

Add <camera_clip> to the MinimalScene plugin config. For example,
<camera_clip>
  <near>0.1</near>
  <far>5000</far>
</camera_clip>

to minimal_scene.sdf
Checklist

[x] Signed all commits for DCO
[x] Added tests
[ ] Added example and/or tutorial
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[x] codecheck passed (See contributing)
[ ] All tests passed (See test coverage)
[ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge

windows CI seems unhappy


Task - Update Auth related documentation to be more generic in regards to the Auth provider

Currently, in some of the docs we are very specific about using JWT auth provider or using the Basic auth provider. As we are starting to add more providers we need to be more generic in our description of using the auth in Amplication with the auth provider of the user choice.

Hey @GreenMachine01, I'm taking care of these updates in #315, the PR related to Adding and Deleting a User Entity. I've also included the necessary updates to take care of #385. Please let me know if I can update anything.


[RKOTLIN-986] K2 Support

This adds support for Kotlin 2.0. This branch does not have code specific to K2 compared to main/release but is for compiling and testing the library with Kotlin 2, as distributables are not binary compatible across Kotlin 1.9/2.0.
This is targeting release/k2 that will be used to publish named (<x>-K2) releases corresponding to normal releases when see fit.
Closes #1614

This PR was targeting release/k2 as a -K2-suffix release. Superseded by #1764 that targets main for 2.0.0. The version bump comments are addressed in #1764.


File dependency conflict

We have an issue with latest version of Flutter:
import 'dart:io';
import 'package:flutter_cache_manager/flutter_cache_manager.dart';

flutter_cache_manager overrides dart:io, cause in flutter_cache_manager declared:
export 'package:file/file.dart' show File;
Look at this for details:
https://github.com/Baseflow/flutter_cache_manager/issues/320
This could be fixed by:
import 'package:flutter_cache_manager/flutter_cache_manager.dart' as fcm;

  Future<void> loadNetworkPDF() async {
    final fetchedFile =
        await fcm.DefaultCacheManager().getSingleFile(widget.networkURL!);
    await (writeCounter(fetchedFile.readAsBytesSync()));
    path = (await _localFile).path;
  }

Could you please fix it?

@akadatsky Check this PR, it is working.
https://github.com/erluxman/pdf_flutter/pull/34


Fixed Pollen.com bugs with ZIP codes and invalid API responses

Description:

Fixes a bug wherein ZIP codes starting with 0 would fail.
Adds some guards against bad API calls.

Related issue (if applicable): fixes #12787
Pull request in home-assistant.github.io with documentation (if applicable): https://github.com/home-assistant/home-assistant.github.io/pull/4784
Example entry for configuration.yaml (if applicable):
sensor:
- platform: pollen
  zip_code: "00544"
  monitored_conditions:
    - allergy_average_forecasted
    - allergy_average_historical
    - allergy_index_today
    - allergy_index_tomorrow
    - allergy_index_yesterday
    - disease_average_forecasted

Checklist:

[x] The code change is tested and works locally.

If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass

Awesome 🎉