Cluster View: Present complete information on the Role Section for HANA

The cluster monitors HANA Application and collects the cluster attribute role as a delimited string containing different pieces of information about the HANA State, like if the node is primary or secondary, if the landscape is in a healthy state, among others.
On the screen, this information should be visualized in an easier way by the user, where he immediately identifies the HANA status collected by the cluster. Currently, a partial view of the information is presented, what can be misleading, like on the screenshot below where there are 2 HANA masters, while the correct is possible that both nodes are running the HANA Nameserver process as master.

A full description of each field can be found on the SAPHanaSR main pages. I am pasting an extracted session here for reference:
roles - actual and configured roles for HANA on that node

The roles field in the Hosts section has six sub-fields for scale-up (A:B:C:D:E:F). It has four sub-fields for
scale-out (C:D:E:F). The sub-fields C:D:E:F are primarily useful for HANA scale-out. For scale-up that
four fields C:D:E:F should be master1:master:worker:master.

Field A: return code of landscapeHostConfiguration.py
Value: [ 4 | 3 | 2 | 1 | 0 ]
This field contains the return code of landscapHostConfiguration.py. The parameter does not tell you if the
secondary system is ready for a takeover. The meaning is different from common Linux return codes.
4 = OK - Everything looks perfect on the HANA primary.
3 = WARNING - A HANA Host Auto-Failover is taking place.
2 = INFO - The landscape is completely functional, but the actual role of the host differs from the config-
ured role.
1 = DOWN - There are not enough active hosts.
0 = FATAL - Internal script error, the state could not be determined.
See landscapeHostConfiguration.py.

Field B: HANA SR role
Value: [ "P"rimary | "S"econdary ]

Field C: NameServer Config Role
Value: [ master1 | master2 | master3 | worker | slave | standby | shtdown ]

Field D: NameServer Actual Role
Value: [ master | slave | standby | shtdown ]

Field E: IndexServer Config Role
Value: [ master1 | master2 | master3 | worker | slave | standby | shtdown ]

Field F: IndexServer Actual Role
Value: [ master | worker | slave | standby | shtdown ]

@lee-martin @diegoakechi how do we want to show this information to the user?
Should we add a column for each Role or should we show badges such as IndexServer: Master?
Also, should we show the actual or the configured role?

@fabriziosestito I would recommend we use badges, as depending on the Cluster config, this attribute might have more columns. Maybe we should have the man page explanation as alt text on the rollover of each badge to link to the old CLI tool info and educate users.
I recommend to show current and config info for now (maybe using some color code for that). This allows an visual verification if HANA is working as configured or if something forced runtime changes.
In the future we can re-evaluate the info hierarchy based on feedback if needed.

@fabriziosestito
Lets break this down:

The return code part
This is currently shown at the top of the cluster details screen as "SAPHanaSR health state:".
Here I think we should start by adding some kind of "i" button or box which appears when you mouse over the return value - I'm open to ideas how to best & easily represent this. The purpose is only to help users understand that definition of the value 0, 1, 2, 3 or 4.

The Role Column for SAP HANA Scale-Up (where we are now)
Here I would only display "HANA Primary" or "HANA Secondary" (Field B) in the column and not display the "HANA Master" box.
The full details can still be seen when the user hits the "details" button

The Role Column for SAP HANA Scale-Out (future music)
Here we will need to show much more details in the Role column.
As per the man page "The sub-fields C:D:E:F are primarily useful for HANA scale-out." Therefore I would park these scale-out questions for now and concentrate on scale-up, which is basically return code and field B.
Scale-out will also be easier when we have Terraform for scale-out

@lee-martin @diegoakechi great thanks for the clarifications
I will fix the issue number 2. and keep the rest open for the future

@fabriziosestito ok... Lets do as suggested by @lee-martin. Feel free to split into multiple issues if needed.


Sdk Package Names should be tolower'd

While merging in new templates from https://github.com/dotnet/sdk we took into CLI  elements that looked like this:
  <Project Sdk="Microsoft.NET.Sdk" ToolsVersion="15.0">

Notice the capitalization of NET. The actual on-disk layout for these SDKs contain instead Net. This capitalization change did not cause issues on Windows as that is a case-insensitive file system but did cause issues on other platforms which are case-sensitive. The experience of tracking down this issue was frustrating because a capitalization problem in a file path is not one that immediately jumps out.
Given that NuGet already tolower's package names to avoid this class of issues I think we should do the same in the Sdk attribute. CLI and VS will tolower the package name during layout generation and MSBuild will tolower whatever string is passed in by the user.
@AndyGerlicher @srivatsn @dsplaisted @rainersigwald thoughts?
Note: NET was actually the right casing, so we changed the on-disk layout. However, this issue is still real as anyone typing an Sdk value may mix up caps and end up in the same broken state.

See discussion of this issue in #1421


When to switch on / off online mode for enrichers ?

I did a refactoring of the GitEnricher since he constantly throws errors because trying to contact a running Kubernetes Cluster.
IMO the default should be offline, which could be switched on by a CD. The reason is, that enriching the resource descriptors with service urls coming from random clusters where one is currently logged in, and which end up in an jar-artefact, too, is not really good for versioning and reproducability (you can't recreate an artefact later if you are not running in the very same environment).
It's ok when running in a CD in order to store meta data about the current build because this artefact gets pushed around the pipeline (even then, one need to give up the dream that a single GIt tag is enough to reproducably recreate a piece of software).
So my idea is to start in offline mode and let the CD switch it on with a property. This is currently possible by adding a -Dfabric8.online to the build scripts (if #156 gets merged in). Of course this can be set within the pom.xml, too for a permanent setting.
I introduced an AbstractLiveEnricher which does all the on / offline handling and is supposed to be the single entry point for contacting a cluster.
I also renamed the GitEnricher to a BuildEnricher (since it adds also Jenkins meta data).
Is this ok that we start offline by default ? This also avoids nasty errors and pauses when people start from scratch.

+1: I just came up with that (closed) PR for changing the log messages because people were saying "I cannot just execute the mvn spring-boot:run to see the quickstart running locally, it requires some kubernetes stuff".
IMHO, the f-m-p should not even try to contact the local kubernetes cluster, unless it is going to deploy the app to a cluster.

I wonder whether there is fabric8 specific CD environment variable, which we could check for an online mode ? BUILD_ID is probably not good enough as it is set by any Jenkins build.
@rawlingsj maybe you have an idea how we could uniquely detect from within the plugin whether it is running in a CD run so that K8s service are accesible and we can extract the proper URL ?

I think ${env.JOB_NAME} or ${env.BUILD_ID} should be ok

@rawlingsj Yeah, but isn't BUILD_ID set for any Jenkins environment, whether it is running in Kubernetes or not ? The problem is, when running online then the enricher tries to detect the Service URL for Gogs (also a fabric8 special thing) and Jenkins.
What I really look for is an env which is specific to our CD which runs on top of Kubernetes (so that one can be somewhat sure that service lookup succeeds)

Ah I see - as we're only talking for maven jobs here maybe we can use one of the env vars that are used to annotate resources that create UI linksGIT_URL, GIT_COMMIT or BUILD_URL

We don't have a specific env var used to identify a fabric8 pipeline build in Jenkins but perhaps one of the three above can help.

@rawlingsj yes, that could work.
Wouldn't it be a good idea to add an env FABRIC8_CD or so, that the detection is more bulletproof ? (or we could call it VOLKSWAGEN_MODE, too. or any car manufactor's label nowadays really :) We could add this to the Jenkins images (but wonder how slaves could inherit this env). But I guess it similar how GIT_URL et al are added.

@rhuss : I don't think this issue is still valid as we have moved fabric8 and fabric8-ui specific enrichers into a deprecated module. WDYT?

Yes, I think we have moved that already out to the deprecated enrichers. Let's close this issue for now.


how to custom fields and render from json data in new version?

I have developed the react form builder using Formeo 1.0.3.
import { FormeoEditor } from 'formeo'

const editor = new FormeoEditor()

How should I do to custom fields and render from JSON data?

kevinchappell
Help me.


Better directory structure for project

I feel like the current structure of files and directories is inefficient and not clean. There should be better management of the source files and the build files, rather than all of them being in the root directory.
I am comparatively inexperienced with how larger C++ projects handle directory management. I would a better management to be suggested, and better yet, I'd welcome all PRs that implement such a structure. Thanks.

I can take this one :+1:

Great, thanks!


feature: base64 encode

i don't know if this would best be handled internally or through a 3rd party, but it would be nice to be able to base64 encode icons without having to visit a 3rd party website

This was implemented in the latest version (3.2.0)

great - thanks!


Adding the updateIRIS mechanism with interface and new mapping

Codecov Report

Merging #97 into master will decrease coverage by 7.69%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master      #97     +/-   ##
=========================================
- Coverage   94.85%   87.16%   -7.7%
=========================================
  Files           5        6      +1
  Lines         136      148     +12
  Branches       29       32      +3
=========================================
  Hits          129      129
- Misses          7       19     +12

Impacted Files
Coverage Δ

contracts/mock/ERC20Mock.sol
72.72% <ø> (ø)
:arrow_up:

contracts/LinniaRecords.sol
86.84% <0%> (-11.67%)
:arrow_down:

contracts/mock/IrisScoreProviderMock.sol
0% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4efe4fb...fef56c6. Read the comment docs.


digikam build is broken, via librsvg

Describe the bug
$ nix-build -A digikam https://github.com/nixos/nixpkgs/archive/c75a18fea6f25dd5dfcc7d76c91cf4
9e3c53c6c5.tar.gz  # master @ today
[...]
   Compiling cssparser-macros v0.3.4
error[E0506]: cannot assign to `self.input.cached_token` because it is borrowed
   --> /build/librsvg-2.44.15/vendor/cssparser/src/parser.rs:572:17
    |
547 |     pub fn next_including_whitespace_and_comments(&mut self) -> Result<&Token<'i>, BasicParseError<'i>> {
    |                                                   - let's call the lifetime of this reference `'1`
...
560 |             Some(ref cached_token)
    |                  ---------------- borrow of `self.input.cached_token` occurs here
...
572 |                 self.input.cached_token = Some(CachedToken {
    |                 ^^^^^^^^^^^^^^^^^^^^^^^ assignment to borrowed `self.input.cached_token` occurs here
...
584 |         Ok(token)
    |         --------- returning this value requires that `self.input.cached_token.0` is borrowed for `'1`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0506`.
error: Could not compile `cssparser`.

To learn more, run the command again with --verbose.
make[2]: *** [Makefile:1934: /build/librsvg-2.44.15/target/release/librsvg_internals.a] Error 101
make[2]: Leaving directory '/build/librsvg-2.44.15'
make[1]: *** [Makefile:1438: all-recursive] Error 1
make[1]: Leaving directory '/build/librsvg-2.44.15'
make: *** [Makefile:931: all] Error 2
builder for '/nix/store/ijs8ng5xc14dc7jl0sbqbfqm8nv8i3l1-librsvg-2.44.15.drv' failed with exit code 2
cannot build derivation '/nix/store/1509dxsd0c53g9pbnjfbqmyr4hkbjic9-hook.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/sg92idbw8rapv774g1za1r90i8wcnci8-digikam-6.2.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/sg92idbw8rapv774g1za1r90i8wcnci8-digikam-6.2.0.drv' failed

So digikam is broken because one of its dependencies, librsvg, is broken.
Expected behavior
digikam should build (and run).
Additional info
git bisect points to https://github.com/nixos/nixpkgs/commit/adb15c3a63fd96edcf7585394b8ccb as the first bad commit.
Metadata
$ nix run nixpkgs.nix-info -c nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.75, NixOS, 19.09beta439.b30f86ffc6a (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3`
 - channels(root): `"nixos-19.09beta596.77b5a1965fc"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

This should've been fixed 02585db25bc4d5d623ce083b41f727c65709ad3f

Yes, librsvg + digikam builds on master now. Thanks.


benchmark the range approach to utf-8 validation

Consider switching to the approach preconised by Yibo Cai.

Yep, it's been pretty thoroughly benchmarked at this point!

Indeed. :-)


Replace wwrelay-utils with pe-utils

Move in config and template files into snap-pelion-edge from wwrelay-utils
Remove wwrelay-utils part from the snap
Add in pe-utils part for identity.json generation
Note: Configuration files and templates have been moved around in these commits
DeviceDB is disabled to start, with a commit that we can revert later to turn back on

Wait for v1.18.1 tag before merging to dev

Note that PR #276 was incorrectly merged. I rebased and removed that commit from this PR, and opened a new PR that we will keep closed for now. The icon says merged in this PR - that is NOT the case. Ignore it


Twitch integration

Jinux refuses to respond.

Needs debugging.

All commands work except for -twitch add username. If I try to add the an username that is already in the list, this error gets thrown in console:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/discord/client.py", line 307, in _run_event
    yield from getattr(self, event)(*args, **kwargs)
  File "Jinux.py", line 168, in on_message
    active, Cmd_char)
  File "/home/adam/Jinux_Bot/cmds/twitch.py", line 14, in ex
    await c.send_message(c, '{}, `{}` is already in the list!'.format(m, u))
  File "/usr/local/lib/python3.5/dist-packages/discord/client.py", line 1145, in send_message
    channel_id, guild_id = yield from self._resolve_destination(destination)
  File "/usr/local/lib/python3.5/dist-packages/discord/client.py", line 289, in _resolve_destination
    raise InvalidArgument(fmt.format(destination))
discord.errors.InvalidArgument: Destination must be Channel, PrivateChannel, User, or Object. Received Client

Welp. It was a simple human error. All good.


RFC: LAFF On Programming for Correctness

Problem: This course in Advanced Programming is too irrelevant, too specific, too research-heavy, full of repetition and tedium, extremely light on actual programming, and of somewhat low quality in its materials.
Duration:
July 20, 2020
Background:
I wrote a lengthy review of LAFF here.  I have read through all the electives of the Knowledge Areas in the CS 2013 document and this course does not really cover any of them in my opinion (except for Logic, which is covered elsewhere in the curriculum).
Proposal:
Remove it from the curriculum.

I don't see any clear connection to the topics in CS 2013 either.
As an aside, I think there is an open question as to what exactly should be in "Advanced". Should Core be all topics in Tier 1, with Advanced covering topics in Tier 2 and Extras covering electives?
Either way, this course does not seem to target our curricular goals. I support removing it.
@spamegg1 can you set the duration to July 20, 2020? That will leave a month long comment period.

I agree with removing it 👍

@waciumawanjohi Updated the date, it was a typo.

I think there is an open question as to what exactly should be in "Advanced".
I have some ideas, stay tuned for my next proposal!

Making a pull request

@spamegg1 @waciumawanjohi  In the LAFF series (see http://www.ulaff.net/), there are four courses. The first one is the foundational course.
I read spamegg1's review of the 2nd course, On Programming for Correctness , and their recommendation is NOT to take it, and this was removed from OSSU. However, this is intended as a course for research students (i.e. 6000 level graduate class), and is meant to be intentionally hand-wavy, and the instructor explicitly says so in the accompanying textbook and notes (see the ULAFF website).
On the other hand, the first course has some good reviews. This course is meant as a show-piece course, and is funded by the US National Science Foundation (grant ACI-1148125). The authors are world-famous researchers in the field, working at TACC.  I work in the HPC area, and have met the TACC researchers and course leaders a few times at major conferences.  Linear Algebra is their research area, and they have written several major libraries in the field.
I suggest a review/examination to include the 1st course in the series.

Linear Algebra: Foundations to Frontiers (LAFF) used to be the recommended Linear Algebra course. It was removed about a year ago. You can read more in the RFC here: https://github.com/ossu/computer-science/issues/693

@krishnakumarg1984 , you note that you work in the HPC area. While I don't think that LAFF is appropriate for our Linear Algebra course, I could see a plausible place for it in an advanced section on High Performance Computing. Part of the challenge would be to determine what courses are appropriate for an undergraduate introduction to HPC. The ACM and IEEE wrote in their Computing Curricula 2020:

High-performance computing (HPC) refers to processing data and performing complex calculations at quadrillions of calculations per second, orders of magnitude faster than ordinary high-speed computers [Raj1 p2]. HPC is essential for today’s popular computing areas such as artificial intelligence (AI), cyber analytics (CA), data science and engineering (DSE), and the internet of things (IoT). The diversity of HPC domains makes teaching HPC difficult. HPC education is commonly needed from undergraduate to post-graduate levels because of HPC’s importance, from computing to non-computing disciplines. A variety of practice based HPC teaching and talent training methods have been widely recognized and implemented in recent years [Che1,Che2,Che3]. But there are still many challenges for incorporating HPC into computing and engineering curricula [Che3 p2,Raj1 p5]. Creating competencies that incorporate the wide range of HPC knowledge, skills, and dispositions may further serve as a useful guide to curriculum development [Raj1 p7].

I could imagine an RFC for 3-5 courses that would make up a section on HPC in Advanced CS. If that RFC quoted relevant sections from the papers cited in the CC2020, it would enable many OSSUnians to participate in the discussion in an informed way. Does that sound like something you would like to take on?

Linear Algebra: Foundations to Frontiers (LAFF) used to be the recommended Linear Algebra course. It was removed about a year ago. You can read more in the RFC here: #693

Precisely my issue here. #693 deals with removing the 2nd course in the series On Programming for Correctness based on spamegg1's detailed review.  I am wondering if the first course in the 4-part series should be the Linear Algebra course here?

#693 deals with removing the 2nd course in the series On Programming for Correctness based on spamegg1's detailed review.

That's not the case.
You can read the commit that resolved #693.


Add DCO Sign-Off requirement to Contributing doc

Description of changes:
Add a note in CONTRIBUTING.md indicating that all commits require a DCO Sign Off, with an explanatory link to https://wiki.linuxfoundation.org/dco
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This change inspired by https://github.com/awslabs/soci-snapshotter/actions/runs/3734827806/jobs/6337344576 where my first PR failed make check-dco's use of https://github.com/vbatts/git-validation, indicated by - FAIL - does not have a valid DCO on each commit.


app tries load bundle from disc root

GET file:///D:/main.bundle.js net::ERR_FILE_NOT_FOUND
tried change 'relativePaths' and 'scope' settings, they has no effect

Hi @Zuzon
How do you run it? From server/webpack-dev-server/direct from disk?

Also, if it's possible, can you show what SW/AppCache plugin generates for you?

appcache:
`CACHE MANIFEST
#ver:v1
CACHE:
../0ac1be82bd79e6be47a32376d286480f.png
../polyfills.bundle.js
../vendor.bundle.js
../main.bundle.js
...
`

How do you run it?
simply open index.html as file

SW is not supposed to work this way. AppCache, even it can, there is not reason for it to work. I tested path generation only with usage on servers, so may fail with file:.
Btw, what is the problem running/testing it with server? I use http-server which is pretty easy to work with:
npm install http-server -g # install
http-server # run in output folder

my project is a bit specific.
i need to create a desktop .net app, on this app i integrate gecko browser engine and also that engine runs my web app in offline without webserver

@Zuzon hmm.. okay, I will check file: support. But, are you sure AppCache is working with file: protocol?

NekR, i found a reason of path problem, because i have baseUrl with slash set (/), with empty string all works perfectly.
I'm sorry to have kept spending your time.

@Zuzon no problem! Glad that it works for you know. Feel free to open another issue if something :-)


Approach to sniff ESP32 packets using Wireshark

I am using ESP32 to send some data from the esp to some server.
My laptop is connected to the same router as my esp and the router is using WPA2 as security. With my little reading, I understand that under such security wifi clients will not be able to see the traffic of each other. and therefore I am not able to see the packets through Wireshark on my laptop.
So the question: is there exists a way that allows me to be able to sniff the packets sent from the esp to the server using Wireshark on my laptop?
I hope I was able to explain my question in an obvious way
Thank you for your help

Your question doesn't seem to relate to this project in any way... but anyway here is a guide to decrypting WPA traffic with wireshark: https://wiki.wireshark.org/HowToDecrypt802.11 - it should work with ESP32 or any other wifi station.

@anszom Thank you so much for your replay.
I am sorry what happened is I was searching for this problem and found another issue with a similar question here, so I thought this is the right place to place my question.


Infinite cuffing glitch

Description
If you put two pairs of handcuffs on a person at the same time, that person can have at least nine total pairs of handcuffs added to them from that point forwards. After the first two cuffs are put on, the character's sprite appears to have two handcuffs on. Once you go higher than that, their sprite is covered by an error. I'm convinced being able to add even two cuffs is a bug, as if you attempt to cuff an already-cuffed person normally, it states something about their hands being full.
An interesting thing to note: The amount of "hands" the person is considered as having goes up with each subsequent handcuffing.
Steps to replicate:

At the same time, have you and another person attempt to handcuff a third person.
Add as many cuffs as you want to that person.

Screenshots

Additional context
Nothing to add here, but I'll update this if I learn more

what bodysystem does to a mf


Filter results by moeity with Checkmol or SMIRKS/SMARTS

Codecov Report
Attention: Patch coverage is 84.67433% with 40 lines in your changes are missing coverage. Please review.

Project coverage is 74.88%. Comparing base (91b1ae1) to head (c01d79d).

Additional details and impacted files

Need to look into the discrepancy between implementations possibly exposed here: https://github.com/openforcefield/yammbs/issues/43#issuecomment-2099158032

Thanks for the review!
Files with special character names make me nervous, so I'm going to leave the current solution in place


[Syncer] Fix flaky node sync

Description
This PR makes the flaky and insane legacy syncer implementation a little more readable.
The only reason it's not rm -rf'd is because we're providing backwards compatibility support, otherwise this legacy logic needs to be rewritten completely.
Changes include

[x] Bugfix (non-breaking change that solves an issue)
[ ] Hotfix (change that solves an urgent issue, and requires immediate attention)
[ ] New feature (non-breaking change that adds functionality)
[ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)

Checklist

[x] I have assigned this PR to myself
[x] I have added at least 1 reviewer
[x] I have added the relevant labels
[ ] I have updated the official documentation
[x] I have added sufficient documentation in code

Testing

[x] I have tested this code with the official test suite
[x] I have tested this code manually

Manual tests
Tested manually with older versions of the client.
Additional comments
Because the original issue that started this PR was resolved by simply notifying peers of new blocks asynchronously , much of the existing tests in syncer_test.go started failing due to not waiting for this dispatch to execute. To alleviate this until syncer_v2 is implemented, a test helper SyncBroadcast was introduced with the same behaviour as the original Broadcast - synchronously waiting for peers to be notified.
Fixes EDGE-557
Fixes EDGE-564
Fixes EDGE-563
Fixes EDGE-561
Fixes EDGE-565

https://github.com/0xPolygon/polygon-edge/blob/7d6cd800c75e489ecb02f0b8d480215eae5606b6/protocol/syncer.go#L114 Personally like more
if !ok{
//error or print
}
//do stuff


Get request body

One of example code from https://docs.socketify.dev/routes.html:
async def home(res, req):
    body = await res.get_json()     #<------
    user = await get_user(body)
    res.cork_end(f"Hello World! {user.name}")

app.post("/", home)

What's the meaning of 'res' and 'req', is it response and request ?
why do we get request body content from 'res' not from 'req' ?
thx

Is response and request the API reflects the uWS API with I also don't agree with, this will be change with the next API


merge from arlac77/template-github,arlac77/template-kronos-component

.github/workflows/auto_approve.yml

chore: add ${{ secrets.GITHUB_TOKEN }} (jobs.auto-approve.steps.with.github-token)

Codecov Report

Merging #793 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master      #793   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            1         1
  Lines           30        30
  Branches         7         7
=========================================
  Hits            30        30

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0009fc2...436ec64. Read the comment docs.


Fix(background): If wallet not connected, display default icon

Solves #505

Extension builds preview

Name
Link

Latest commit
da5cc85a9109c6dd679049c8cc3dea5af2402a03

Latest job logs
Run #10418248921

DownloadDownload


AssertionError: Low or high out of range

I'm trying to use this module on enwik5 data (10 000 bytes). But I encounter this error:
AssertionError: Low or high out of range

Are there any additional limitations in the implementation? Or do I do something wrong?
The script below works ok with enwik4 data (1 000 bytes).
I count statistics myself and then use StaticModel, but I encounter either this Low or high out of range error, or ValueError: Symbol has zero frequency error.
enwik5.zip
fn = 'enwik5'

print(fn)

def read_bytes(path):
    with open(path, 'rb') as f:
        return list(f.read())

data = read_bytes(fn)
nsyms = 256
stats = [0] * nsyms
for c in data:
    stats[c] += 1

from arithmetic_compressor import AECompressor

from arithmetic_compressor.models.base_adaptive_model import BaseFrequencyTable
from arithmetic_compressor.util import *

SCALE_FACTOR = 4096

class StaticModel:
  """A static model, which does not adapt to input data or statistics."""

  def __init__(self, counts_dict):
    #vals = (v for k, v in counts_dict.items())
    #counts_sum = sum(vals)
    #probability = {k: v / counts_sum for k, v in counts_dict.items()}
    #print(probability)
    probability = counts_dict

    symbols = list(probability.keys())

    self.name = "Static"
    self.symbols = symbols
    self.__prob = dict(probability)

    # compute cdf from given probability
    cdf = {}
    prev_freq = 0
    self.freq = freq = {sym: round(SCALE_FACTOR * prob)
                        for sym, prob in probability.items()}
    for sym, freq in freq.items():
      cdf[sym] = Range(prev_freq, prev_freq + freq)
      prev_freq += freq
    self.cdf_object = cdf

  def cdf(self):
    return self.cdf_object

  def probability(self):
    return self.__prob

  def predict(self, symbol):
    assert symbol in self.symbols
    return self.probability()[symbol]

  def update(self, symbol):
    pass

  def test_model(self, gen_random=True, N=10000, custom_data=None):
    self.name = "Static Model"
    return BaseFrequencyTable.test_model(self, gen_random, N, custom_data)

freq_map = {
    sym: freq for sym, freq in enumerate(stats)
    if freq > 0
}

model = StaticModel(freq_map)
coder = AECompressor(model)

N = len(data)
compressed = coder.compress(data)

Oh, sorry i am just seeing this. I have no idea why i wasn't notified 🤔
About the issue, this usually happens when theres a symbol in the data that is not present in your frequency map, this is most likely a unicode symbol. So setting nsyms = 256 is not enough, you probably need to go through the data to get all possible symbols and build a frequency map out of that.
Something like this:
data = read_bytes(fn)
stats = {}
for c in data:
    stats[c] = stats.get(c, 0) + 1

...

freq_map = {
    sym: freq for sym, freq in enumerate(stats)
}

Again, sorry for the late reply

Hmm. I checked the maximum symbol value in enwik5.dat after the read_bytes function and it is 237. So it seems that there is no unicode symbols after fopen(..., "rb").
From Python 3.11 docs:

Files opened in binary mode (including 'b' in the mode argument) return contents as bytes objects without any decoding.

I just tested this code and it works fine for me? 🤔
fn = 'enwik5.zip'

print(fn)

def read_bytes(path):
    with open(path, 'rb') as f:
        return list(f.read())

data = read_bytes(fn)
nsyms = 256
stats = [0] * nsyms
for c in data:
    stats[c] += 1

from arithmetic_compressor import AECompressor

from arithmetic_compressor.models.base_adaptive_model import BaseFrequencyTable
from arithmetic_compressor.models import StaticModel
from arithmetic_compressor.util import *

SCALE_FACTOR = 4096

freq_map = {
    sym: freq for sym, freq in enumerate(stats)
    if freq > 0
}

model = StaticModel(freq_map)
coder = AECompressor(model)

N = len(data)
compressed = coder.compress(data)

print(len(data))
print(len(compressed))

I am also on version 3.11

I should also point out that the StaticModel takes in a map of probabilities, not frequency, so it should be this:
...
total_count = sum(stats)
prob = {sym: freq/total_count for sym, freq in enumerate(stats) if freq > 0}

model = StaticModel(prob)

...


module.FindDocuments

set cpp breakpoint  is ?

The file was not loaded when the breakpoint was set？
Test LocalComponent.cs
IDkmSymbolDocumentCollectionQuery.FindDocuments


Change FactoryGirl::Generators::Base#explicit_class_option to output 1.9 Hashes

This changes the generator to use the 1.9-style Hash syntax for the class option. So, where previously the generator would output:
FactoryGirl.define do
  factory :buttery_biscuit_base, :class => 'Buttery::BiscuitBase' do
    #...
  end
end

it will now output:
FactoryGirl.define do
  factory :buttery_biscuit_base, class: 'Buttery::BiscuitBase' do
    #...
  end
end

I totally understand this is a very minor change and is ultimately down to personal preference - but it is something I would like :). FactoryGirl requires Ruby > 1.9 (https://github.com/thoughtbot/factory_girl/blob/master/factory_girl.gemspec#L17) so this shouldn't cause any issues in practice.

Thanks! Merged in 3a050736f3ea0b84e389caef5ac1bc8dc7b7b679


Why is React hydration data within a script tag, commented out?

I'm working on an issue to optimize open source library for server rendering: github.com/shakacode/react_on_rails/pull/660.
We're considering breaking away from placing react hydration data in a meta tag and moving into maybe a script tag.
Why does Airbnb puts react hydration data inside of an <script type=“application/json” data-key=“foobar”><!--{json}--></script> per the screen shots below? Is this related to XSS or performance reasons?
view-source:https://www.airbnb.com/s/Panorama--BC--Canada?guests=10&adults=10&children=0&infants=0&checkin=12%2F18%2F2017&checkout=12%2F25%2F2017&ss_id=19nvfbwh&ss_preload=true&source=bb&page=1&s_tag=pMJm3GqR&allow_override[]=

CC: @goatslacker

Some older browsers will execute any script block, even if it's type is "application/json".
Best practice is to always comment it out to ensure nothing can ever execute.

Hi @ljharb! Big thanks for the explanation! In terms of avoiding XSS, it seem that you escape & and >, and not <. Any other characters?

The only thing that needs escaping is to prevent the literal string </script>, because if browsers see it in any context, they instantly close the script tag.

@ljharb there might be the chance of some white space in the </script> and it being accepted, so probably you would not just strip that out. We're planning on using the Rails json_escape.
In terms of performance, did you ever compare placing the JSON data inside of meta tag? the react-rails gem does that. It results in lots more escaping when that's done that way.
ProductHunt.com uses an alternative approach where they use an inline script to set compressed, uuencoded, likely JSON data, directly to a global variable (see image below). Possibly the data is compressed to avoid XSS. If the page data were gzipped, double compression would do little. Still, one might need to strip out the '>' before placing directly in the script tag. The web client would read the global value and then decompress and parse the JSON to an object.
Did you ever try the performance of that technique? One downside I see is that until CSP 2.0 is widely supported (the nonce feature), a site that wants to use CSP would not be able to use this technique.

CSP is very important; inline JS would be a very bad practice. Performance is irrelevant until other concerns are addressed. A meta tag would have to go in the <head> which would delay the page from loading.
Is there something slow about a JSON script block that warrants investigation?

Is there something slow about a JSON script block that warrants investigation?

Some users of https://github.com/shakacode/react_on_rails have complained that the use of a div's meta data element to hold the JSON data is too slow.
This is the source for https://nootrobox.com. There is too much quoting here.
Thanks again for the advice!

@ljharb, thank you for explanation.  I have one comment:

The only thing that needs escaping is to prevent the literal string </script>, because if browsers see it in any context, they instantly close the script tag.

In our testing, preliminary closing happens for everything which conforms to HTML end tag syntax (https://www.w3.org/TR/html5/syntax.html#end-tags).  That means that the string can be case-insensitive and it can have spaces between script and <.
That is, <ScRiPt     /> will also prematurely close.
I guess that you are actually handling that by escaping >, right?

script isn't self-closing, but sure, < / sCrIpT> would probably be a concern too. Indeed handling either < or > covers any possibility of a closing tag.

I was sooo not awake when I wrote this.   Of course it's </ScRiPt    >.  Sorry!

@squadette brought up the point:

This is an explanation on why HTML comments are non-trivial if we do care about "older browsers": http://www.howtocreate.co.uk/SGMLComments.html#doubledash (via http://softwareengineering.stackexchange.com/questions/198481/why-cant-an-xml-comment-contain-two-hyphens)

Basically it means that we have to somehow escape dash-dash (even though we do escape lt/gt). That's why I'm really concerned about this idea.

@ljharb, any thoughts on if you'd need to escape double dash in the JSON?

@justin808 that seems like a potential concern; a PR with test cases would be appreciated.
However, my reading of that is that -- merely changes how > is interpreted; so as long as > is always escaped, the issue won't come up.

@ljharb no, it's not about escaping the >.  Here is an example of JSON content in HTML comment:
<!--{"foo": "bar"}-->

Here is theoretically problematic JSON content in HTML comment:
<!--{"foo": "bar -- baz"} -->

The second (inner) -- will, according to spec, turn on termination on >, but it will be again disabled by the third, final --.  So, > will not be interpreted as the rest of comment, and browser will ignore everything until the next -- and > (possibly with other content between the two).
Both Chrome and Safari handle this correctly, and this is one of the worse parts of HTML spec (deprecated and removed in the new spec).  My concern is that if we really do that comments thing due to compatibility with older/broken browsers then we should think about what other problems this approach could lead to in the same older/broken browsers.
FTR: as for me, properly encoded JSON within the script tag, without comments, is perfectly ok.

Interesting - that does sound like something we want to handle, then.

Just in case, relevant part of HTML4 spec: https://www.w3.org/TR/html4/intro/sgmltut.html#h-3.2.4
HTML 3.2 says that it is SGML Application: https://www.w3.org/TR/REC-html32#sgml
HTML 2.0: https://www.w3.org/MarkUp/html-spec/html-spec_3.html#SEC3.2.5
Sometimes they only talk about whitespace, and it's unclear what happens if there is anything else between the pairs of dash-dash.

Here is historical evidence that this is real issue in old versions of Firefox: http://weblog.200ok.com.au/2008/01/dashing-into-trouble-why-html-comments.html


Update version after scaling

When an app is scaled with existing tasks, the version of the new and of the old
tasks was different. This looked confusing in the UI for the user.
This patch updates all the old tasks at the end of the scaling deployment step.
Backend part of the fix for #1025

For @drexin and @ConnorDoyle: is there a better way to update many tasks at once than calling store n times?

I'm not convinced that rewriting the version history is the right thing to do.  What problem does this solve?

I am not convinced 100%  either, but IMO we have some kind of inconsistency in our concepts about apps:

there is an app definition,
there is the app version and
there is the app instances value.

In my world view, version is a revision of the target cluster state regarding the app, where transitions are modeled with deployments. Therefore the version shall be strictly increasing with every deployment.
For the app definition we have basically two options:

it describes the target state of the cluster regarding an app, i.e. all its instances
it describes only how a task looks like and how it is handled within its life-cycle (health, upgradeStrategy, etc.).

Right now we have a mix: the app definition does the second and the last ("current") app definition does the first. This is strange.
This patch tries to implement the first by updating the AppDefinition for old tasks.
From my experience using the Marathon API in a continuous deployment setup, it would make more sense to separate app state and app definition, i.e. to have something like:
case class AppState(
  app: AppDefinition,
  version: Timestamp,
  instances: Int
)

The underlying question is "what does the task version mean?"  Right now an AppDefinition models a request for some state.  Tasks that are launched to achieve that state are tagged with the version of the request.  If we stick with that definition, then the current behavior is internally consistent.  IMO updating the version of already-launched tasks makes this a much harder concept to explain.

@ConnorDoyle downscaling an app marks all running instances after deployment as out of date. That doesn't look like very consistent to me.
I like AppState from @sttts. Can anything else except instances be changed that would not require task rescheduling? Looks like ports are ephemeral an can be changed without restarts. This way version changes, app definition changes, tasks are not rescheduled but out of date yet again. I think this scenario should be considered too.

@bobrik We are talking about adding moment.js to handle the display of relative times for the version field: "Just now", "An hour ago" and "Three days ago": https://github.com/mesosphere/marathon/issues/1025
Will this not solve the problem without changing the backend?

@mlunoe just like I said: https://github.com/mesosphere/marathon/issues/1025#issuecomment-70313812

I'm ok with relative time if you could get absolute values in tooltip or with click (like in mesos ui).

Yes, tooltip is definitely part of that solution. Sorry I didn't call that out.

I guess with the relative dates this pr is obsolete.


Animation tracks at runtime not in same order as within editor.

Tested versions
v4.2.rc.custom_build [d34f1ec4b]
System information
Windows 10  v4.2.rc.custom_build [d34f1ec4b]
Issue description
Not sure if anyone noticed this or if it's already fixed in newer versions but the order of animation tracks does not persist when the scene is loaded in my project. The animation tracks that have to do with materials get pushed to the bottom making it hard to get/set the keyframes in these tracks by their index.
E.g. Here is an animation in my project, I marked the keyframe values of the first section/group. The following groups have the same keyframe values except for rotation:

Then I wrote this code to get the order at runtime:
		var anim : Animation = anim_player.get_animation("Impact").duplicate()

		var test_val_arr = []
		var i = 0
		while i < anim.get_track_count():
			var j = 0
			var track_vals = []
			while j < anim.track_get_key_count(i):
				track_vals.append(anim.bezier_track_get_key_value(i,j))

				j+=1
			test_val_arr.append(track_vals)
			i+=1

		print(test_val_arr)

This prints out:

As you can see the [1,0] tracks are all pushed to the back of the array.
Steps to reproduce
In an AnimationPlayer node:
Create an animation for several mesh nodes that modifies their materials and their transforms.
take note of the order of the tracks in the editor.
See that they are different at runtime using the code above.
Minimal reproduction project (MRP)
should be easy enough to test.

Can you try on an official build or a more recent branch? You're on a custom branch that's some 6 months old


FileNotFoundError importing from HipChat - emoticons.json

Attempting to migrate from HipChat Server (Atlassian HipChat 2.2.9 (2<IP_ADDRESS>010) ) to Zulip

Fresh Zulip install using the production install guide - https://zulip.readthedocs.io/en/stable/production/install.html
Followed the Import from Hipchat instructions (including upgrading Zulip server to the latest master branch) - https://zulipchat.com/help/import-from-hipchat
Encountered the following error:

zulip@xxxxxxxxxxxxx:~/deployments/current$ ./manage.py convert_hipchat_data hipchat.tar.gz --output converted_hipchat_data
Converting Data ...
2018-12-27 20:41:31.433 INFO [] input data was untarred to /home/zulip/deployments/2018-12-27-20-25-42/hipchat.gz
2018-12-27 20:41:31.559 INFO [] Starting to process emoticons
Traceback (most recent call last):
  File "./manage.py", line 56, in <module>
    execute_from_command_line(sys.argv)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zerver/management/commands/convert_hipchat_data.py", line 77, in handle
    masking_content=options.get('masking_content', False),
  File "/home/zulip/deployments/2018-12-27-20-25-42/zerver/data_import/hipchat.py", line 835, in do_convert_data
    output_dir=output_dir,
  File "/home/zulip/deployments/2018-12-27-20-25-42/zerver/data_import/hipchat.py", line 351, in write_emoticon_data
    with open(data_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/zulip/deployments/2018-12-27-20-25-42/hipchat.gz/emoticons.json'

zulip@xxxxxxxxxxxxx:~/deployments/current$ find hipchat.gz/ -type f -exec grep -H 'emoticons' {} \;
hipchat.gz/users/18/history.json:      "message": "[...]",
hipchat.gz/users/1/history.json:      "message": "[...] :-D",
hipchat.gz/users/3/history.json:      "message": "[...]  ",
hipchat.gz/users/3/history.json:      "message": "[...]",
hipchat.gz/users/6/history.json:      "message": "[...] :-D",
hipchat.gz/users/6/history.json:      "message": "[...]",
hipchat.gz/metadata.json:    "includes_emoticons": false,
hipchat.gz/rooms/24/history.json:      "message": "[...]  ",
zulip@xxxxxxxxxxxxx:~/deployments/current$

zulip@xxxxxxxxxxxxx:~/deployments/current$ ls -al hipchat.gz/
total 100
drwxrwxr-x  4 zulip zulip  4096 Dec 27 20:41 .
drwxr-xr-x 28 zulip zulip  4096 Dec 27 20:40 ..
-rw-r--r--  1 zulip zulip   301 Dec 25 15:28 metadata.json
drwxrwxr-x 32 zulip zulip  4096 Dec 27 20:40 rooms
-rw-r--r--  1 zulip zulip 18518 Dec 25 15:20 rooms.json
drwxrwxr-x 50 zulip zulip  4096 Dec 27 20:40 users
-rw-r--r--  1 zulip zulip 59296 Dec 25 15:20 users.json

Hipchat Export:

@cruscio thanks for the report!  I think we're just not properly handling the includes_emoticons: false state.  @showell can you investigate?

Managed to get an export with an emoticons.json file using hc-migrate - instructions here
And encountered the following error during conversion
zulip@xxxxxxxxxxxx:~/deployments/current$ openssl aes-256-cbc -d -in /home/xxxxxxxx/20181228_migrate.tar.gz.aes -out hipchat.tar.gz -md md5 -pass pass:xxxxxxxx
zulip@xxxxxxxxxxxx:~/deployments/current$ ./manage.py convert_hipchat_data hipchat.tar.gz --output converted_hipchat_data
Converting Data ...
2018-12-28 13:45:37.689 INFO [] input data was untarred to /home/zulip/deployments/2018-12-27-20-25-42/hipchat.gz
2018-12-28 13:45:38.387 INFO [] Starting to process emoticons
Traceback (most recent call last):
  File "./manage.py", line 56, in <module>
    execute_from_command_line(sys.argv)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/zulip/deployments/2018-12-27-20-25-42/zerver/management/commands/convert_hipchat_data.py", line 77, in handle
    masking_content=options.get('masking_content', False),
  File "/home/zulip/deployments/2018-12-27-20-25-42/zerver/data_import/hipchat.py", line 835, in do_convert_data
    output_dir=output_dir,
  File "/home/zulip/deployments/2018-12-27-20-25-42/zerver/data_import/hipchat.py", line 359, in write_emoticon_data
    for d in data
  File "/home/zulip/deployments/2018-12-27-20-25-42/zerver/data_import/hipchat.py", line 359, in <listcomp>
    for d in data
TypeError: string indices must be integers

Contents of emoticon.json file below.
jq can parse it... maybe ujson ran into an issue?
Also, should I open a separate issue for this error?
{
  "Emoticons": [
    {
      "ID": 211,
      "user_id": 1,
      "group_id": 1,
      "path": "1/caaando-1453996070.png",
      "shortcut": "caaando",
      "width": 23,
      "height": 28,
      "audio_path": ""
    },
    {
      "ID": 213,
      "user_id": 3,
      "group_id": 1,
      "path": "1/xxxx-1454020172.jpg",
      "shortcut": "xxxx",
      "width": 21,
      "height": 28,
      "audio_path": ""
    },
    {
      "ID": 214,
      "user_id": 1,
      "group_id": 1,
      "path": "1/allthethingssad-1454360927.png",
      "shortcut": "allthethingssad",
      "width": 28,
      "height": 29,
      "audio_path": ""
    },
    {
      "ID": 215,
      "user_id": 3,
      "group_id": 1,
      "path": "1/partyon-1462569075.jpg",
      "shortcut": "partyon",
      "width": 23,
      "height": 29,
      "audio_path": ""
    },
    {
      "ID": 216,
      "user_id": 1,
      "group_id": 1,
      "path": "1/vbscript-1462644197.gif",
      "shortcut": "vbscript",
      "width": 30,
      "height": 21,
      "audio_path": ""
    },
    {
      "ID": 217,
      "user_id": 1,
      "group_id": 1,
      "path": "1/annoyedpicard-1464893521.jpg",
      "shortcut": "annoyedpicard",
      "width": 29,
      "height": 22,
      "audio_path": ""
    },
    {
      "ID": 223,
      "user_id": 1,
      "group_id": 1,
      "path": "1/itsworking-1486252514.jpg",
      "shortcut": "itsworking",
      "width": 27,
      "height": 12,
      "audio_path": ""
    },
    {
      "ID": 224,
      "user_id": 1,
      "group_id": 1,
      "path": "1/fingerscrossed-1493839213.png",
      "shortcut": "fingerscrossed",
      "width": 30,
      "height": 30,
      "audio_path": ""
    },
    {
      "ID": 225,
      "user_id": 3,
      "group_id": 1,
      "path": "1/punchingbag-1519856497.jfif",
      "shortcut": "punchingbag",
      "width": 30,
      "height": 30,
      "audio_path": ""
    }
  ]
}

@cruscio I just merged 44f117ac72e46c6e204f372075e888b3c7f1c0d6, which should fix the original issue you reported.  Usually, I'd recommend a separate issue, but I just went ahead and fixed the second issue in 41b7d9a4c881f7df14b721b0f75f04c31f5373a8.
Please open a new issue for anything else you find.  It appears that hc-migrate uses a different export format than the HipChat exports we've seen in the past, so it's possible that as soon as you test with the emoticon.json issue fixed, you'll run into something else.
If you're willing to share with us the export file you have, I'd be happy to iteratively test with it to get it working fully, since I imagine other folks will end up using hc-migrate.

Thanks @timabbott - After updating zulip, the conversion ran without error on an export from the hipchat web server admin page. Not too surprisingly, the hc-migrate-based export still fails further along; looks like it puts files in a different path. Unfortunately I can't share one of my exports. I may be able to stand up a clean instance, fill in some chatter, and share an export of that. If I do, I'll get it into a new issue.
For reference:
FileNotFoundError: [Errno 2] No such file or directory: '/home/zulip/deployments/2018-12-30-13-18-21/hipchat.gz/1/caaando-1453996070.png'

zulip@xxxxxxxxxxxx:~/deployments/current$ ls hipchat.gz/1
ls: cannot access 'hipchat.gz/1': No such file or directory

zulip@xxxxxxxxxxxx:~/deployments/current$ find hipchat.gz/ | grep caaando
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>hipchat.gz/files/img/emoticons/1/caaando-1453996070.png

OK, sounds good.


Improve error messages and testing in bases.jl

Coverage increased (+0.1%) to 98.885% when pulling c9fa27b7a1e45ff7607ef42e15567aba5b91a2d8 on bases into b89e216ec61575541c2d67a3c1b0278813e32511 on master.


Add zhtw Traditional Chinese locale

Add zhtw Traditional Chinese locale 🇹🇼

Hey, thank you very much for the contribution!


[PCM-2140] - Fix issue causing SDK to not find existing session.

Please add a unit test(s) to test the cases we care about.

Please add a unit test(s) to test the cases we care about.

We actually had unit tests but they were flawed (my fault). I altered them to be more thorough.


typo: EBSC table major version has a footnote for "EBLT"

On https://docs.microsoft.com/en-us/typography/opentype/spec/ebsc the footnote to the majorVersion is "Note that the first version of the EBLC table is 2.0." - this should be "Note that the first version of the EBSC table is 2.0."

Thank you for the feedback. Fixed.


Module.createRequire is not a function, 2.18

this is for documentation
if u encounter this error when doing
npm run config:check

after upgrading to MagicMirror 2.18, it is because the eslint package requires node 14 or above.
please check your node and npm versions by doing
node -v
npm -v

the easiest way to upgrade node is to use the npmjs package called n (node version manager written in node js)
do
sudo npm install n -g

then to upgrade node (and npm) do
#upgrade node to version 16
sudo n 16
#fix path for this terminal session, it will be ok for new started sessions
PATH="$PATH"

n is pretty cool, it will let you switch between node versions too. we are only using a tiny bit of its function.
do n --help
to see all it can do

thanks a lot. i'm a beginner and do not understand this or what i have to do exactly:
#fix path for this terminal session, it will be ok for new started sessions
PATH="$PATH"
Can You explain this? Thank You very much !

you just cut and paste the text
leading # is a comment

Thanks a lot.
My npm version is 8.1.2 - i need 8.3? i can't install because: error: eacces: permission denied, rename xxxxx
You know the solution?
My screen is still black ;-)
i can delete this post if he is not for here...
thanks you very much!

no, whatever comes with node 16 is fine..
black screen is another problem ..
see
https://forum.magicmirror.builders/topic/15778/fix-for-black-screen-in-2-16-and-later

sudo n 16 gives me
Error: no version found for '16'

what pi is this?

no direct route to node 16 on pi 0w


Submodule instructions are missing from the README

(Copied over from archived anms-ace repo) The Development section of the README does not include instructions for git submodules, which must be checked out to get unit tests to pass.

This is addressed in issue #7 with the associated PR #8


PGP: Expose S2K packet

To know if we deal with a key that contains no secret key (GNU extension) we would like to expose the S2K packet. This pull requests adds a method to PGPSecretKey to get the packet.

Added


The cora sites graph is not connected?

@tkipf
When I displayed the graph I got a graph with many components.  Is this correct?  pdf attached.  Maybe you included extra info in the edge list?
cora.pdf

@tkipf
When I displayed the graph I got a graph with many components. Is this correct? pdf attached. Maybe you included extra info in the edge list?
cora.pdf

How did you plot the graph?

My apologies for not responding sooner!  I used mathematica for this but there are many other ways.  Gephi is a free open source code that installs quickly.  You could probably use python...
M := Import["cora", "Table"];
Ev := EdgeList[M];
g := Graph[Ev];
A := AdjacencyMatrix[g]
G := AdjacencyGraph[A];
GraphPlot[G]


Coda Daemon crashed

I'v started node and eveything was good at the beginning. But in some time daemon crushed.
terminate called after throwing an instance of 'std::bad_alloc'
what():  std::bad_alloc
2019-08-14 13:43:25 UTC [Error] Prover failed: $error
error: "((rpc_error (Connection_closed ("EOF or connection closed")))\n (connection_description ("Client connected via TCP" (233648.local 38979)))\n (rpc_tag rpc_parallel_plain_3) (rpc_version 0))"
2019-08-14 13:43:25 UTC [Info] Generating crash report
coda_crash_report_2019-08-14_13-43-41.739273.tar.zip

Thanks for the report! This seems to be another instance of https://github.com/CodaProtocol/coda/issues/2978. I'll close this issue in favor of that one...


Update map.ts

Update map latitude or longitude by string variable doesn't center it. But for markers are parsing input string to a number. I Copied code from Marker.ts file to update this map.ts for auto parsing string to number

it's better to only allow a number, then to try different type versions.


Feat/eco 183 live preview

Title: JSON preview changes
Changes: Live preview text changes
JIRA: https://contentstack.atlassian.net/browse/ECO-183

Closing this PR need to resolve a conflict will reassign this


WithOnlyOneExpandedItem and nested Expanded lists

Hi!
I try to implement some logic, when need show list with 2 lvls of lists:
Root 1

Category 11

Item 111
Item 112

Category 12

Item 121
Item 122

Root 2

Category 21

Item 211
Item 212
...

So, all work fine if not use WithOnlyOneExpandedItem for adapter. If I set the flag to true then categories subitems unable to open.
But i need only one expanded root item. So, what I can do it ?
Thanks.

@gromiloff this is not possible with this flag as it will prevent any additional hirachies of being opened, no matter about the level.
In this case you will have to check the selected items count when a new element is opened and perhaps close other parents.

https://github.com/mikepenz/FastAdapter/blob/develop/library-core/src/main/java/com/mikepenz/fastadapter/FastAdapter.java#L882
might help.
Or also this:
https://github.com/mikepenz/FastAdapter/blob/develop/library-extensions/src/main/java/com/mikepenz/fastadapter_extensions/utilities/SubItemUtil.java


Bug: fix describe.only to work

The describe.only seems not to be working currently. After researching something codes from the repository, I learn only is based on the awesome option grep. But the set to grep from only describe does after runner.run(done)(Essentially it's grepTotal), but the current source code attach grep at https://github.com/yorkie/mocha/blob/b0ce87c78e84d32008dcbc93fa32fd0cd3129845/lib/mocha.js#L465, so the grep from describe.only doesn't apply at this case.
I added a trigger beforeSuite and an once listen for doing re-grep. Now this change works for me :-)

Anyone of maintainers can take a look at this one? Thanks

Ping @dasilvacontin @travisjeffery?

Hmm it'd be nice if we could find another way, cause that event will get ran a lot.

cause that event will get ran a lot

This would cause what problems and why you think this way is not good, thanks :-)

Primarily performance problems, maybe it's not too big a deal. I'll try it on some different test suites tomorrow and see how big of a difference it'll be.

Ok, thank you @travisjeffery

@travisjeffery If you have some secret benchmarking sauce you're hiding from us, please share

@yorkie do you have a small reproducible example of your issue? Just tried with
var assert = require('assert');

describe('suite not run', function () {
  it('should not run', function () {
    assert(fail);
  });
});

describe.only('suite to run', function () {
  it('should run', function () {
    assert(true);
  });
});

describe('suite not run 2', function () {
  it('should not run', function () {
    assert(fail);
  });
});

and it worked.

I guess you should use nested describe to reproduce :-(

@travisjeffery Try it like this:
describe.only('suite to run', function () {
  it('should run', function () {
    assert(true);
  });
});

describe('not a suite to run', function () {
  it('should run', function () {
    assert(true);
  });
});

This runs both blocks even though it should only be running the first one...

@saschagehlich that particular case is expected because /suite to run/ ~= /not a suite to run/

Is it expected? The docs read "The exclusivity feature allows you to run only the specified suite or test-case by appending .only() to the function.". For me, this means that only the block I flagged with .only should be run.

@saschagehlich thanks for pointing out the issue, but it seems to be another issue that I thought about, which is relevant to the regular expression to filter cases.

Yeah since .only uses .grep internally, so the docs could be improved there.

This functionality changed in v3.0.0 - closing for inactivity.


progress discussion

In configuration file, do we have the LST range as hour integers only?
Currently we have gaincal and bpcal -- what to do with the other calibrators, fluxcal and polcal
Do we keep the delaycal tag?
Should you visit the bandpass calibrator first if cadence is specified?

Decimal float seems better.
Not sure (will need to think about it).
Do not know.
Yes, bpcal first, target after (or alternatively bpcal immediately after target).

I think there is some confusion over the use of tags in the target.
First the observation configuration file should define how the targets are observed. But the tags are part of the target string that is passed through CAM and to the Datafile and the SDP backend. And are interpreted as the INTENT of the target, like if it is a calibrator and solutions need to be calculated with it.

Thanks for the feedback, I think I got most things added now:
You can give it a run and see what it does
Currently limit to the simulation side, will go to work on devcomm next week to get the dry-run system and sensor values in
You need a system that has katpoint and ephem installed
If you clone the repository
cd astrokat/scripts
Some example
python observe.py --profile ../config/targets.yaml
python observe.py --profile ../config/image_sim.yaml
python observe.py --profile ../config/image.yaml
python observe.py --profile ../config/OH_periodic_masers.yaml

Additions after progress discussion

Better description of usage and implementation

Adding type key for drift scans

Updated the instrument subkeys -- they are now similar to the CAM sensors, which makes addition of more keys easier, also since PTUSE is a special case, it will make the addition of this USE system easier
NOTE: this is the only USE system that will be added because currently this is the only beamformer pipeline that exist. Other USE software will be ignored, better design and implementation is required.

More discussion needed

target_list and calibration_standard unify? I really do not want to do this. If you use only the cadence tag to separate them then you require processing upfront to generate 2 lists behind the scenes. I do not always want to process all targets in the list, since this is wasteful if you have a long list of targets, but only a few calibrators with cadences. Explicit representation in the configuration file makes implementation clear and simplify scripts.

On the last item. I say you move the responsibility to the user (i.e. implementation in the configuration file). Two benefits:

you give user more control at a price of (small, but non-negligible) risk of things going awry,
less work for you,

About the pol-/flux-cal, especially in the beamformer (bf) case: currently we have only adopted single dish approach to pol/flux calibration. For fluxcal:

slew the telescope to on-source (flux calibrator) and record it with noise diode firing,
slew the telescope to an offset position of +/-X deg. in declination (that is to be decided) of flux calibrator and record it with noise diode firing (so 2 obs in total),

The flux calibration observations should happen at least once per observing session, in general the observations of flux cal targets are done at least once per week if not more often.
For polcal:

slew the telescope to an offset position of X deg. in declination (that is to be decided) of target of interest (pulsar) and record it with noise diode firing,
this is done before or immediately after target of interest of observation,

This is working fine: my rev. 1 report on SD calibration will demonstrate it, hopefully will be released before the "BRP-odyssey" ;-). Given improved calibration pipeline (removing cross-hand leakages, as per BenH/Ludwig latest feedback) we will AFAIK apply the same approach for large subarray for now.
So the catalogue2config.py should have (for bf case):

offset in deg from target for both flux and pol cal,
a strategy of pointing,

OK, there are other issues lurking around the bushes which I think, we should winkle out. Shall we make a brief meeting about that? Tomorrow, before lunch looks fine for me...

Update to observation script:

drift scans are now included, but katpoint does wierd things, need to look at the success of this in more detail
python observe.py --profile test_targets.yaml
User logger now displays type of observation as well as first tag name, e.g. target, bpcal, etc
python observe.py --profile ../config/two_calib.yaml

Please do not comment yet, still busy working, not ready for action

Closing this repository, follow development https://github.com/ska-sa/astrokat


test(app): Protocol Upload Revamp e2e test case

Overview
Add the ability to drive the installed run app for end to end testing of the feature Protocol Upload revamp(PUR).

Use python, selenium(4), and pytest to execute tests against the installed run app.
Roughly use the Page Object Model
See Notes in README

TestCase Scenarios:

Happy path scenario for PUR - connecting to robot->landing on protocol page->uploading a protocol -> verifying "setup for run page"
Multiples of a module - verifying that Moam link is available when a protocol with MoaM is uploaded
Gen 1 pipette - Links related to Gen 1 pipette and no modules are uploaded.
No modules uploaded testcases
All new code with no dependencies outside of the new app-testing directory.

Risk assessment
There is no risk to any other module in the repository. The app-testing folder is self contained.

Now, the test checks all the labware offsets in the PR


Implement IEmbeddingGenerator (Microsoft.Extensions.AI)

With #112 support for the IChatClient came to be, but there's a companion, IEmbeddingGenerator, which is used for getting embeddings.
It'd be beneficial for that to be supported too for full feature parity with Microsoft.Extensions.AI.

Damn, that was quick!
I'll get that into the next version of the Aspire OllamaSharp integration too 🤩


Beer

Not really an issue, just if you are up for a beer drop me a note at<EMAIL_ADDRESS>:-)

🍻


example/centos: qcow2 for x86 and aarch64

Initial setup and dedup to build the rest on.

@achilleas-k All commits have been split out separately.

We had a chat on Google Meet and I've resolved the outstanding comments related to SELinux and test(ing|ability).


use of no_aug_train set

Hi what is the use of no_aug_train_10p set ?
When i tried to refine with imagenet, it eexpected a no_aug train set. What is its purpose?

Hi, the purpose is to get a metric of representation quality already during training.
We use 10% of imagenet (the imagenet1k_10percent_simclrv2 dataset is a subset of ImageNet containing only 10% that was also used in SimCLRv2) to train a k-NN classifier.
We find this to provide a relatively good proxy for downstream performance during training.
"Noaug" indicates that no augmentations are applied, as the k-NN classifier is typically trained on non-augmented images.
You can create the dataset with this repo, or you can simply remove the "train_noaug_10p" dataset from the yaml and then also remove the "offline_knn_metrics_callback".
Appendix G contains some additional information.
As an example: for MAE-Refined-L/16 the k-NN accuracy trained on the 10% subset looks like this:


keeping it simple: don't declare queues and exchanges

I don't see the point in making the worker declare queues and exchanges. All that should be defined in RMQ config.
If we want to keep the worker simple, then just indicate queue and vhost to subscribe to and leave configuration to RMQ. What d'y'all think ?

I agree.


Chore/Target net 7.0 in tests

My bad, Updated workflows to include .net 7 aswell , should be working now


Evaluate and possibly integrate an IPython notebook environment in the single machine package

Currently, our single machine deployment package does not include any notebook or SQL environment. Instead we provide the Thrift server and clients can connect to it through JDBC. Once we start to rely more on FHIR views for query needs, having a python/notebook environment becomes even more important. We should evaluate different options and possibly integrate one. There are some options here.

To clarify: The image we add here should include required JDBC drivers (and Java deps), FHIR-views as a packaged library (with Spark support), and possibly other useful libraries that do not come with the upstream image. That said, we should try to limit the overhead for maintaining this image.


version  0.26 missing  in terraform snowflake provider repository

Provider Version
terraform-provider-snowflake_v0.26
The provider version you are using.
Terraform Version
The version of Terraform you were using when the bug was encountered.
Terraform v1.0.3
Describe the bug
We need this version as it has fixed bugs which we are currently facing.
Looks like 0.26 version was removed recently.
A clear and concise description of what the bug is.
Expected behavior
A clear and concise description of what you expected to happen.
Code samples and commands
Please add code examples and commands that were run to cause the problem.
Additional context
Add any other context about the problem here.

Can  you please help us with an update ?

We are closing this issue as part of a cleanup described in announcement. If you believe that the issue is still valid in v0.89.0, please open a new ticket.


Creation of list of Heavy loading tasks

Problem description
Lots of tasks are dependent on each other
Requirements
Need to create a list of heavy loading tasks to be made.
Definition of done
Complete list of tasks to be made
Acceptance criteria
What are the criteria by which we determine if the issue has been resolved?

@anvabr @Pyatakov @simvalery Need a detailed discussion on finalizing the list.

`Create new token (TokenCreateTransaction)
newToken
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:68
Get balance account (AccountBalanceQuery)
balance
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:130
Get associate tokens and balance (AccountInfoQuery)
accountInfo
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:146
Minting fungible token allows you to increase the total supply of the token (TokenMintTransaction)
mint
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:325
Minting a non-fungible token creates an NFT with its unique metadata for the class of NFTs defined by the token ID (TokenMintTransaction)
mintNFT
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:357
Wipes the provided amount of fungible tokens from the specified account (TokenWipeTransaction)
wipe
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:393
Transfer tokens from some accounts to other accounts (TransferTransaction)
transfer
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:428
Transfer non-fungible token from some accounts to other accounts (TransferTransaction)
transferNFT
guardian-service/src/hedera-modules/hedera-sdk-helper.ts:463
mintProcessing
guardian-service/src/policy-engine/blocks/retirement-block.ts:141
retirementProcessing
guardian-service/src/policy-engine/blocks/mint-block.ts:168`

@simvalery @anvabr @Pyatakov To have a discussion on the above list.

This ticket was QA approved and closed on March 30th


Feature/fi 1756 direction

Summary
Adds incoming icon to requests list.

Notes
Axe Deque may have issues with empty table headers. This may need to be resolved before moving forward.
Testing Guidance
Ensure that any test with incoming requests show the new icon and tooltip on hover.
One test that works:
Run (g)(10) against the reference server and check test 1.3.05.

Do you want me to run Axe Deque against this?  I don't think I have the extension set up anymore but probably need to have that ready anyway ...

@chris-at-work You can definitely do that, the issues I was seeing were sort of intermittent and sometimes "needs review" instead of actually being errors so might be good to do a sanity check there.

Codecov Report
Base: 74.45% // Head: 64.03% // Decreases project coverage by -10.41% :warning:

Coverage data is based on head (6f86d8e) compared to base (6914298).
Patch coverage: 19.04% of modified lines in pull request are covered.

:exclamation: Current head 6f86d8e differs from pull request most recent head 4f8e121. Consider uploading reports for the commit 4f8e121 to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #285       +/-   ##
===========================================
- Coverage   74.45%   64.03%   -10.42%
===========================================
  Files         186       72      -114
  Lines        9019     5976     -3043
  Branches      367      370        +3
===========================================
- Hits         6715     3827     -2888
+ Misses       1941     1783      -158
- Partials      363      366        +3

Impacted Files
Coverage Δ

client/src/components/Header/Header.tsx
73.87% <0.00%> (-9.62%)
:arrow_down:

...ite/TestSuiteDetails/TestListItem/RequestsList.tsx
71.84% <36.36%> (-2.79%)
:arrow_down:

lib/inferno/jobs.rb

lib/inferno/dsl/tcp_exception_handler.rb

lib/inferno/dsl/suite_option.rb

lib/inferno/entities/test_suite.rb

...nferno/repositories/validate_runnable_reference.rb

lib/inferno/repositories/headers.rb

... and 108 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Yeah it doesn't like it.  Header text must not be empty.  I pushed a fix.  Not sure if aria-role none is really needed.  It semantically hides the element which is true because it's just the table header?

Looks like the error is resolved in any case so probably fine?

@arscan Yeah I couldn't find anything better, let me know if you have any lightbulbs. Merging.


Unbork MapView docs

Our markdown renderer seems to not handle code blocks inside blockquotes.
This works around the issue by removing the blockquote.
** Briefly describe changes and the motivation behind them here **
This PR contains:

[ ] Changelog N/A
[X] Documentation
[ ] Tests N/A

This seems to be a duplicate of #104...


fix: dropped unavailable video from the resources page

Closes: SUP-7034

:tada: This PR is included in version 1.24.1 :tada:
The release is available on:

1.24.1
GitHub release

Your semantic-release bot :package::rocket:


Debug Logging started after Restart of HA

You can Start/Stop debug logging in the settings, which is working.
But every time i restart HA it is set to started again.
This results in a warning message in the core log that max. log is reached and attribute logging is cancelled.

Are you start/stopping the Debug logging in the Configuration Wizard or on the Event Log > Actions. In the Wizard, set it to info in the Other Parameters or Format Parameters screen depending on the version.

I start/stop it in the Configuration Wizzard.

I just updating my note (above) after remembering what was going on.

changed in b16

installed already and is working


Delete old branches

[ ] manual-bot-rerun: Previously used to re-run EIP-Bot. Now handled by a different system.
[ ] rerun-pull-request-target-on-review: Previously used to re-run EIP-Bot. Now handled by a different system.

@alita-moore could you give some guidance on what all the other branches are and if it's safe to delete them?

Don't delete delete-failed-on-review it's for the marking stale bot

Also mark-eips-stale

Should be good to clear everything else, but verify the bots being used on EIPs


bash-completions breaking macOS

Is bash-completions required to use goto? I couldn't seem to get it working until I added bash-completions and set that up via Homebrew. But that broke the default BASH completions in iTerm2. Sorry, I didn't test with the default Terminal before removing bash-completions and goto. I really like using goto but I can't accept how bash-completions changes the default behavior under iTerm2. Is it possible to use just goto without bash-completions?

Hi @runeimp.
The script has to be sourced in order to make the goto function available because it's not an executable. In the same script though there is the code for the bash completion which is an important feature of the program.
Unfortunately, I don't see a solution at the moment unless you manually source it without the bash completion (ex. copying the script to a file and remove the bash completion code and then add a line in your .bashrc to source it). I don't suggest it though.
I didn't have any problems when using the formula in macOS under iterm2 though. Can you provide more information on what breaks after installing the bash-completions?
Thank you for reporting.

Hey @iridakos,
I  did eventually try to just source the file without bash-completion installed and it was working fine. The issues I was having with bash-completion was with path completion for cd or ls and file path completion for text editors such as vim and subl (Sublime Text 3). Basically all other completions that I was used to using were broken.
Hm, sourcing the file also seems to be allowing the completion of goto shortcuts without needing to install bash-completion.

@runeimp ok, then  it could be something iTerm does with the auto-completion. Good news that it works though. Thanks for reporting ;)

Just a note that the auto-completion works in Terminal as well even thought I've do not have bash-completion installed.


[SDK] Fix: Disconnect smart wallet when signer is disconnected

PR-Codex overview
This PR focuses on improving error handling in the AccountProvider and DetailsModal components, enhancing the connection manager for wallets, and updating tests to ensure proper functionality when no address is provided or when wallets disconnect.
Detailed summary

Added error handling in the AccountProvider for missing addresses.
Modified DetailsModal to close when the active account is falsy.
Updated tests for AccountProvider and DetailsModal to check for error scenarios.
Improved connection manager to handle wallet disconnections.
Cleaned up unnecessary comments and code in DetailsModal.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

Merge activity

Dec 6, 7:21 PM EST: Graphite disabled "merge when ready" on this PR due to: a merge conflict with the target branch; resolve the conflict and try again..


should have committed ages ago. So close...just need to fix submit bu…

…tton
getting there...

one box won't empty upon submission...
and there's SOME css

have to break for a bit, but here's a start


Unable to change the hostname of the ethernet interface using tcpip_adapter_set_hostname(TCPIP_ADAPTER_IF_ETH, ... )

Hello, i have a project that connects to the ethernet using the ethernet example in the examples directory. After calling esp_eth_init() i call tcpip_adapter_set_hostname(TCPIP_ADAPTER_IF_ETH, "HelloWorld"). Connection to the router works and all internet functionality works correctly, my only problem is that when i go on my router in the DHCP lease table i see "espressif" as a hostname, instead of "HelloWorld".
If i connect to the router using WiFi and then setting the hostname for the WiFi sta, the dhcp lease table shows the correct hostname ("HelloWorld").

Update: Nevermind, the set_hostname call is working correctly. Apparently plugging and unplugging the cable does not update the, just the leash time for the dhcp. After i removed the address from the dhcp table and plugged the cable in again everything worked correctly. Closing this


When playing some songs the note splashes dont look right

Describe your bug here.
on some songs i play, the note splashes sort of bug out and turn red and green..

i dont really get how the note/note splash coloring system works at all but i know its not really supposed to do this..
it happens even with other note splashes aswell

its not a game breaking bug but id like for my note splashes to actually look right.. any help??
Command Prompt/Terminal logs (if existing)
No response
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows
Did you edit anything in this build? If so, mention or summarize your changes.
No.

It looks like the noteSplash rgb shader is turned off.
Try using this:
function onCreate()
for i = 0, getProperty('unspawnNotes.length')-1 do
if getPropertyFromGroup('unspawnNotes', i, 'mustPress') then
setPropertyFromGroup('unspawnNotes', i, 'noteSplashData.useRGBShader', true);
end
end
end
Hope this helps!


parsl server: Close slurm jobs when workflow has finished

I've been processing some test datasets on the new server config (the server-newtasks branch) and, after a whole workflow has finished processing, parsl still keeps the slurm executor running for >15 min later (and still ongoing). In that way, we block a whole cluster node even though now actual processing is happening anymore.

Note that this problem becomes very clear in the monitoring. I recall seeing a progress bar where all tasks were completed but then there was a long idle period. I'll put a screenshot here if I find it.

The current example (which only runs on fake data) is
# Define/initialize empty folder for project-related info
# (and also for the output dataset -- see below)
TMPDIR=`pwd`/tmp-proj
rm -r $TMPDIR
mkdir $TMPDIR

# Set useful variables
PROJECT_NAME="myproj"
DATASET_IN_NAME="input-ds"
DATASET_OUT_NAME="output-ds"
WORKFLOW_NAME="My worfklow"

# Create project
poetry run client project new $PROJECT_NAME $TMPDIR

TESTDATA=../../../tests/data

# Update dataset info
poetry run client dataset modify-dataset $PROJECT_NAME "default" --new_dataset_name $DATASET_IN_NAME --type image --read_only true

# Add resource to dataset
poetry run client dataset add-resource $PROJECT_NAME $DATASET_IN_NAME ${TESTDATA}/png/ --glob_pattern *.png

# Add output dataset
poetry run client project add-dataset $PROJECT_NAME $DATASET_OUT_NAME --type zarr
poetry run client dataset add-resource $PROJECT_NAME $DATASET_OUT_NAME ${TMPDIR}/$DATASET_OUT_NAME --glob_pattern *.zarr

# Create workflow
poetry run client task new "$WORKFLOW_NAME" workflow image zarr

echo "{\"__PROVIDER_ARGS__\" : {\"max_blocks\": 10}}" > /tmp/args_wf.json
poetry run client task modify-task "$WORKFLOW_NAME" --default_args /tmp/args_wf.json

# Add subtasks (with args, if needed)
poetry run client task add-subtask "$WORKFLOW_NAME" "Create OME-ZARR structure"

echo "{\"parallelization_level\" : \"well\", \"rows\":1, \"cols\": 2}" > /tmp/args_yoko.json
poetry run client task add-subtask "$WORKFLOW_NAME" "Yokogawa to Zarr" --args_json /tmp/args_yoko.json

poetry run client task add-subtask "$WORKFLOW_NAME" "Replicate Zarr structure"

echo "{\"parallelization_level\" : \"well\"}" > /tmp/args_mip.json
poetry run client task add-subtask "$WORKFLOW_NAME" "Maximum Intensity Projection" --args_json /tmp/args_mip.json

echo "{\"parallelization_level\" : \"well\"}" > /tmp/args_labeling.json
poetry run client task add-subtask "$WORKFLOW_NAME" "Per-FOV image labeling" --args_json /tmp/args_labeling.json

# Apply workflow
poetry run client workflow apply $PROJECT_NAME $DATASET_IN_NAME "$WORKFLOW_NAME" --output_dataset_name $DATASET_OUT_NAME

For this example, running on SLURM, the parsl monitoring looks "wrong", with plenty of time where everything is complete but the worfklow is not over:

I don't know whether this is the same problem as in this issue (since I don't see these long tails in squeue), but it may be related.

Here is the parsl.log file, as a future reference:
parsl.log


[Restore] Restore with seed phrase

The function of restore with seed phrase work well in Android development version vDebug.
The demo video attached in [Restore] Restore with google drive
#50


6 dropdown menu for users

Added a script called select.py to allow users to search for genetic diseases and select one for the API search. Also modified requirements.txt.
Warning: may have merge conflicts as venv was created locally and path to XLSX file may differ.

https://github.com/isabelrem/2023-Software-Development-Assessment/issues/6

This code imports the national testing directory from the url, might be better than a set path stored locally. I was specifically cleaning it up to search for the clinical indication via the R code, but should be able to use pandas to pull out any information, more or less.
import pandas as pd
import xlrd
pip install openpyxl --upgrade
def import_directory():
url = "https://www.england.nhs.uk/wp-content/uploads/2018/08/Rare-and-inherited-disease-national-genomic-test-directory-version-5.1.xlsx"
# import second sheet of genomics test directory - first contains version information, second contains info
data = pd.read_excel(url, sheet_name = 1)

# set first row as header
data.columns = data.iloc[0]

# get rid of the first row which contains the text which are now the headers
data = data.iloc[1:]
data.reset_index(drop=True, inplace = True)

return data

def Rcode_to_Clin_Imp(Rcode):
# import directory data
data = import_directory()

# cleaning the input:

# remove any dashes
Rcode = Rcode.replace('-','')

# Add R prefix to code
if not Rcode.startswith("R"):
    Rcode = "R" + Rcode

CI = data.loc[data['Clinical indication ID'] == Rcode, 'Clinical Indication'].iloc[0]

print(CI)

# should add errors excepts here too

return CI

Rcode_to_Clin_Imp(input("Please enter the Rcode"))

Logging and Testing will need to be added in future


Suggestion: separated starter script form codebase

Initially based on the script you have that runs after installing the dependencies, I've created a script that simplifies the process in only one step (it clones the repo and runs the script) and has also the benefit of being outside the codebase.
You can run the script with:
npx github:gimenete/create-typescript-library <dirname>

Repo here: https://github.com/gimenete/create-typescript-library
Besides the library name it also asks for a few other things:

What do you want the library to be called?
Is this a web app or a Node.js app? Node.js / Webapp
Is this a library? Standalone application? Yes / No
Do you want to use travis? Yes / No
Do you want to use commitizen? Yes / No

Maybe we can remove the embedded script you have in the project and promote using this one. My suggestion is to put this script in your repo in a different branch. Then it will be run like this:
npx github:alexjoverm/typescript-library-starter#script <dirname>

Where script is the branch name.

Screenshot here:

Hi @gimenete!
That would be great. Currently all scaffolding scripts and deps are in the repo itself, so this idea separates the setup from the working code.
Some thoughts:

I'd do it progressively: let's focus first on splitting the setup-related stuff in a separate branch, then we can talk about adding more options.
Although npx is great and  the preferred option, it could be a good idea to provide instructions to do it just with npm as well.

I'd be happy to work with you or someone else on this. Any PR is more than welcome :)

It would be nice to separate the starter code from the deps so any update to the deps could be translated to all libraries started with this starter.
I took this idea from create-react-app that uses react-scripts to hold all its starter logic, then I saw @kentcdodds talking about how PayPal is taking this approach for their starters in this blog post.
I've started to do this in the TypeScript Node Starter and the results are awesome 🙂.
I would love to see this starter going in that direction so my existing libraries could benefit from updates to this project.
Thanks for the great work!

Thanks @hrajchert for the links and the suggestions. This is indeed an exciting feature. I've added the help wanted tag in case someone wants to take care of this issue, otherwise I'll try to have something working when I find some time.


Add mongo alias

As shown on xeol, habing mongo would also be useful

Thanks guys :ok_hand:

https://github.com/endoflife-date/endoflife.date/discussions/2611


drop addresses from fraud (#2938)

Coverage remained the same at 90.274% when pulling d5c644d10d5250d3fc1e1d66be0112355d71ea07 on release into 73578024cc456721919fa78ee7b05ba35d09989e on preprod.


Integration with conditional loader

It would be nice to have a simple integration with yepnope or require.js or something of the sort for a drop-in caching solution.

closing all my old issues


Utilize updated buildpack metadata types

Only write diffID in the layers metadata
Write buildpack stack & api in layers metadata

Codecov Report

Merging #292 into master will increase coverage by 0.04%.
The diff coverage is 93.33%.

@@            Coverage Diff             @@
##           master     #292      +/-   ##
==========================================
+ Coverage   67.17%   67.21%   +0.04%
==========================================
  Files          77       77
  Lines        3107     3099       -8
==========================================
- Hits         2087     2083       -4
+ Misses        772      771       -1
+ Partials      248      245       -3

Impacted Files
Coverage Δ

pkg/cnb/remote_buildpack_metadata.go
100% <ø> (ø)
:arrow_up:

pkg/apis/experimental/v1alpha1/store_types.go
0% <ø> (ø)
:arrow_up:

pkg/apis/experimental/v1alpha1/buildpack_types.go
0% <ø> (ø)
:arrow_up:

pkg/cnb/remote_store_reader.go
71.42% <100%> (+1.42%)
:arrow_up:

pkg/cnb/cnb_metadata.go
66.26% <100%> (ø)
:arrow_up:

pkg/cnb/builder_layers.go
100% <100%> (+26.66%)
:arrow_up:

pkg/cnb/store_buildpack_repository.go
80.7% <100%> (+1.85%)
:arrow_up:

pkg/cnb/create_builder.go
70% <100%> (+7.5%)
:arrow_up:

pkg/cnb/builder_builder.go
79.09% <80%> (-0.22%)
:arrow_down:

pkg/apis/experimental/v1alpha1/store_validation.go
87.5% <0%> (-4.17%)
:arrow_down:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea7713b...305a08b. Read the comment docs.


Add parameters for plugin commands

Is your feature request related to a problem? Please describe.
Sometimes it's required for the user to input some parameters to help with API request, (e.g. to get a specific section for video transcript for youtube).
Describe the solution you'd like

plugin sends a response code of something (not 200) and a dialog pops up for the user to input extra information for the command to run
Then the metadata is populated with this extra information & sent back as the same request

User feedback (https://mail.google.com/mail/u/4/#inbox/FMfcgzGsnBXcwdzcznwnFxTFLTMrsXfn) :

Can I directly paste and upload images, insert Markdown syntax in notes, instead of generating a new note with the image? Can I use only the PicGo API to support uploading to my own image hosting service?


style: auto fix clippy warnings and formatting

Issue being fixed or feature implemented

To increase code quality and readability we can apply auto-formatting and clippy fixes
What was done?

cargo clippy --fix
cargo fmt

How Has This Been Tested?

None
Breaking Changes

None
Checklist:

[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated relevant unit/integration/functional/e2e tests
[ ] I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
[ ] I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

[ ] I have assigned this pull request to a milestone

cargo fmt


For a ASP .NET Core (.net core 3.0 preview7)  Web API project via CLI, get the error "HTTP 404 error That’s odd... Microsoft Edge can’t find this page".

Describe the bug
For a ASP .NET Core (.net core 3.0 preview7)  Web API project via CLI, get the error "HTTP 404 error That’s odd... Microsoft Edge can’t find this page".
To Reproduce
Steps to reproduce the behavior:

Install .NET core sdk 3.0.100-preview7-012424
Create a new .net core ASP .NET Core Web App API project via CLI, dotnet restore and build succeeded.
Dotnet run succeeded, and display two listening ports. . Run https://localhost:5001/api/values, get the error "HTTP 404 error That’s odd... Microsoft Edge can’t find this page."

Expected behavior
Run https://localhost:5001/api/values succeeded with no errors.

Environment data
dotnet --info output:
.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview7-012424
 Commit:    aafcce302b

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.0.100-preview7-012424\

Host (useful for support):
  Version: 3.0.0-preview7-27816-04
  Commit:  c36106d9b5

.NET Core SDKs installed:
  2.1.800-preview-009696 [C:\Program Files\dotnet\sdk]
  2.2.400-preview-010219 [C:\Program Files\dotnet\sdk]
  3.0.100-preview7-012424 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview7.19315.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview7-27816-04 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0-preview7-27816-04 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Note: For a ASP .NET Core(.net core 3.0 preview7) Empty web/Web App (Model-View-Controller) project can work well.

Same issue here (with preview 6): I need to perform a clean before launch debug, otherwise i got a 404.

@YunWeiaa does doing a clean either git clean or dotnet clean help?

Thank you for filing this issue. In order for us to investigate this issue, please provide a minimalistic repro project that illustrates the problem. We can't repro this issue.

Here is the repro project - cs30webapi.zip， hope to help you.

Thanks @YunWeiaa.
Looks like this was an intentional change on our end and the ValuesController is not there any more. We're going to fix this in the next update for the templates, which we hopefully squeeze into preview7 milestone (tracked elsewhere).

https://github.com/aspnet/AspNetCore/issues/11699


No internet while using DNS crypt

Hi, I get no internet connection when I am connected to a DNScrypt server. Applications such as steam, teamspeak still works fine but I can not visit any website.

Emm...Which DNS server do you use? Some servers may have poor performance in some network environments, especially in some mutiple-NAT networks.


MM-27688 - Implement OpenId Connect in Cloud

Documentation for: https://mattermost.atlassian.net/browse/MM-27688
Updated:

Cloud Admin Guide > Authentication > Office 365 Single Sign-On

Added H2s to break up steps & supplemental images
Updated procedure for creating a new connection via OpenID Connect

@wiersgallak - We have product documentation for setting up Office 365, Google, and GitLab with OpenID Connect. Does it make sense to add another page that details how to setting up a generic OpenID application?

@cwarnermm Yes, I think a page for generic OpenID connect makes sense. Thanks!

@wiersgallak @justinegeffen The Cloud-specific product documentation updates for OpenID Connect are now ready for final PM and Editor review. Thanks for your patience on this one!


Use MKLocalSearchCompleter instead of MKLocalSearchRequest

https://medium.com/@ishabazz/location-search-using-mapkit-works-ae94f3c3dbd7
Can localsearch request be replaced by local search completer, it offers better completion even Apple uses MKLocalSearchCompleter in their maps according to the answer for the SO question. https://stackoverflow.com/questions/43249382/mklocalsearch-doesnt-provide-the-same-results-as-search-in-native-apple-maps-ap

https://medium.com/@ishabazz/location-search-using-mapkit-works-ae94f3c3dbd7
Can localsearch request be replaced by local search completer, it offers better completion even Apple uses MKLocalSearchCompleter in their maps according to the answer for the SO question. https://stackoverflow.com/questions/43249382/mklocalsearch-doesnt-provide-the-same-results-as-search-in-native-apple-maps-ap

that would be great !

MKLocalSearchCompleter doesn't provide coordinates, or anything else besides title/subtitle as a matter of fact