[Feature Request]: Want to Update Ui of Seller Registration form

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
Want to Update Ui of Seller Registration form
Use Case
It  provides better readability and look to website
Benefits
No response
Add ScreenShots

Priority
Medium
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

@nishant0708 go ahead


reordering doesn't works with csrf protection

Hello,
By default the reordering of todos is blocked by the csrf token, the solution to this problem is here: https://docs.djangoproject.com/en/dev/ref/contrib/csrf/#ajax
Since I'm not very experimented in javascript I won't send a pull request because I don't know what is the best way to implement this.

I've currently just temporarily disabled csrf (using the csrf_exempt decorator) for the reorder view since I'm only using it on a local site for testing purposes anyway.

So sorry I haven't gotten the csrf fix rolled out - I do have one here, just need to integrate it. Will do soon!

Hi guys - this has been fixed. Thanks for your patience.


Create test_computetools.py with BorisPush tests

Pull Request
Description
Create test_computetools.py and create test for BorisPush __init__ method and push method
This pull request addresses #97

[x] make sure you are pulling from a dev branch on your fork
[x] check for code linting

I'm not sure how to proceed given that the values from the calculation are not saved anywhere nor are they attributes.

Although implemented, this test returns false, stating where False = <function allclose at 0x10ddd8f80>(array([[      0.        ,       0.        , 2999849.80105991]]), array([[0.  , 0.  , 0.03]])) E        +    where <function allclose at 0x10ddd8f80> = np.allclose

I think that's because the time step for your clock is 0.1. Try changing the dt parameter in the dictionary that you pass to the Simulation constructor to 1e-9 instead of 0.1.

Now the assertion seems to be caught up between [[0, 0, 0.299985]] and [[0, 0, 0.03]]

you can change the tolerance for the allclose function, should help with passing the check

@arichar6 Whoops, misunderstanding on my end!

Thanks for the contribution! Sorry this one was kinda troublesome.


only update if context exists

fix for #542
fixes this error that comes if a user is not authenticated yet.

closing... problem was in custom context processor. https://github.com/jazzband/django-debug-toolbar/issues/542#issuecomment-920894735


[Suggestion] Configureable Crooks

A white-list of items that can be used as valid crooks and the durability they should have. This feature could also be paired with the ability to adjust the durability of existing crooks.

 Changing existing items is impractical.


feat: add plugin social wallets

Relates to:
New plugin for sending TX to social accounts
Risks
Low - this is a new plugin and PR is mainly around adding code in the new /packages/plugin-social-wallets folder.
Note:

there are two changes made to the Twitter client, adding a simpler Base client that doesn't auto start and adding avatar to fetchProfile
for handlerCallback, type void was added as possible return

Background
What does this PR do?
Adds a way to send transactions to someone's social handle. Built to support twitter for now.
Leverages Privy as the account repository and Privy import feature to add users social account. This creates a wallet on behalf of the user that they can claim by logging in with twitter on that Privy instance.
In the future, the data source could be replaced with an onchain social registry that anyone can opt into themselves to link their socials to a wallet, using TEE-verified oauth. With a reserve vault smart contract layer on top of that, this could hold tokens sent to users that haven't signed up yet and then easily be claimed on signup.
What kind of change is this?
Features (non-breaking change which adds functionality)
Why are we doing this? Any context or related work?
To enable agents to send tokens directly to twitter handles without having to know the users wallet address. This could spark entirely new forms of meta games, stories, and economies.
Documentation changes needed?
My changes do not require a change to the project documentation.
Testing

Where should a reviewer start?
/packages/plugin-social-wallets
Detailed testing steps
Add env values for:

PRIVY_APP_ID
PRIVY_APP_SECRET
EVM_PRIVATE_KEY
TWITTER_USERNAME
TWITTER_PASSWORD
TWITTER_EMAIL

Run sh ./scripts/start.sh
Ask agent send 0.001 ETH to @<twitter_username>
In addition some adjustments may need to be made to plugin-evm that triggers the TX flow. Last time we tested, that plugin was giving us some issues and we had to resolve them to get the plugin to trigger.
Discord username
@djdabs_

@djdabs

I just tested the code, your build was failing for client-twitter.
so i checked it more , why it's failing
I think you should initlize twitter client with runtime , rather than exporting SimpleTwitterManager form client-twitter
export class SimpleTwitterManager {
    client: ClientBase;
    constructor(runtime: IAgentRuntime) {
        this.client = new ClientBase(runtime);
    }
}

you can try importing TwitterClientInterface , initlize it with runtime and then use twitter details you wanna use.
Can you fix this issue and make a PR again.
Feel free to take a latest Pull request from develop branch from eliza code.


Create zip for each file

Allow fetching data in parallel

Codecov Report

Merging #91 into master will increase coverage by 0.05%.
The diff coverage is 70%.

@@            Coverage Diff             @@
##           master      #91      +/-   ##
==========================================
+ Coverage   60.82%   60.88%   +0.05%
==========================================
  Files          18       18
  Lines        1838     1874      +36
==========================================
+ Hits         1118     1141      +23
- Misses        610      616       +6
- Partials      110      117       +7

Impacted Files
Coverage Δ

api/diagnostics.go
73.92% <70%> (-0.67%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 278a996...d0df646. Read the comment docs.

:wave:  @mrnugget @drewvanstone Can you review, please?

btw. did you try running the code with more than 1 worker?

btw. did you try running the code with more than 1 worker?
Nope, I'll do it in a minute

@mrnugget

btw. did you try running the code with more than 1 worker?

More then one workere test https://github.com/dcos/dcos/pull/4659/commits/c969619c1863bdf73d786c7e0e3fec30cd075a3c

Ah! And it looks like all the tests are passing, right?


Can I use without vue-router?

I'm using gtag for a small SPA without Vue Router, but when I NPM run build I get this error:
node_modules/vue-gtag/vue-gtag.d.ts:3:31 - error TS2307: Cannot find module 'vue-router' or its corresponding type declarations.

3   import type { Router } from "vue-router";

Seems like this is just a types issue, but it's an annoying one. Do I have to have vue-router types to use this package?

After digging around a bit, I'm not sure if this package is usable without vue-router (assuming typescript). Happy to be told otherwise, I don't want to bloat my project without need!

@petegriffith I'm using it without vue-router and it's working as expected.

Are you using typescript? The inbuilt types import types from vue-router...

Yes, using typescript.
I now checked, I have vue-router installed to the project but I'm not using it, I guess that's the reason for it working.

It would be helpful if you guys used the template I put on issues when you create one. There are multiple versions of this package with different versions of Vue, and they are very different.
Could you please be more mindful of this?
What version are we talking about?


input_slider set state problem

Home Assistant release (hass --version):
0.30.2
Python release (python3 --version):
Python 3.5.2+
Component/platform:
Ubuntu Linux
Description of problem:
In my scene.yamlfile I have this code:
- name: ALL OFF
  entities:
    input_slider.livingroom_dimmer:
      state: 5
    group.all_switches:
      state: off

And I get the following error when I activate the scene.all_off:
16-10-20 20:01:17 homeassistant.helpers.state: reproduce_state: Unable to reproduce state <state input_slider.livingroom_dimmer=5 @ 2016-10-20T20:00:06.498240+02:00>

Expected:
I expect that the input_slider would be set to the value 5. When I use the http://IP:PORT/dev-state it works well.
Problem-relevant configuration.yaml entries and steps to reproduce:
input_slider:
  livingroom_dimmer:
    name: Dimmer
    initial: 0
    min: 0
    max: 15
    step: 1

Instead of using a scene I turned to a script, which fixed the problem for me:
all_off:
  alias: 'ALL OFF'
  sequence:
    - service: input_slider.select_value
      data:
        entity_id: input_slider.livingroom_dimmer
        value: 0


Add oidc_provider_arn output

Needed for downstream consumers of the module to get the OIDC provider for use with IRSA.

Verified this works as needed for downstream use by the terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks module.


/regexp format to load multiple segments from precomputed skeleton

I have created a script to create a precomputed skeleton with unsharded format, I have ~ 10k files numbered from 0
to 12519 along with the info file in a locally served directory.
The example shows how to display 1 segment:
https://github.com/google/neuroglancer/blob/master/python/examples/example_skeletons.py
I have used the Rendering panel to add segments 1 by 1 from the web browser.
But how can I use the regexp syntax to load all or part of them at once?

You should use the neuroglancer precomputed segment info format to create a list of 12519 objects .  neuroglancer was designed with dense EM data with millions or billions of objects and so there isn't a built in way to load "all" of them, but there are usually a subset of them that are of greater interest, which is what the segment info file is designed to do.
I will pitch that Casey Schneider-Mizell recently added python functionality for creating segment properties files from pandas dataframe to his package nglui.  See docs here .  Also note that you can store this segment_info file as a seperate source of a layer by pressing the + button at the bottom of the source tab of the layer.  This means you can use precomputed segment info even on an a zarr set of voxels.


Please provide a setting to override the default connection timeout

I am using the REST client to test a REST service that takes a while to process the request.
Unfortunately, the REST client times out when I call my service.
Please provide a setting that can be used to override the default connection timeout.
Thanks for your help.
Regards

Duplicate of #502


Fix TLS version negotiation based on sigalg availability

If a server is using a brainpool certificate and if we can negotiate either TLS1.2 or TLS1.3, but the necessary sigalg is missing, TLS1.3 will fail immediately, while TLS1.2 may work. So use the signature_algorithms extension of the ClientHello as a hint which protocol to choose.
Fixes #25697

Hmm. I haven't looked carefully at how you all implement this, but I suspect that either this does not actually work, or that OpenSSLs where is_tls13_capable checks are reachable are silently less secure than those without. I.e. this will make brainpool TLS 1.3 less secure than standard TLS 1.3.
TLS 1.3 added this downgrade protection signal to the ServerHello random. You're supposed to send it whenever the server is TLS-1.3-capable but picked TLS 1.2 for whatever reason. That will then cause a TLS-1.3-capable client to reject the connection.
https://www.rfc-editor.org/rfc/rfc8446#section-4.1.3
Now, this signal is a workaround for TLS 1.2 ServerKeyExchange failing to sign the entire transcript. The signature payload doesn't contain any context outside the two randoms. An attacker is free to arbitrarily modify the ClientHello and see what ServerKeyExchange signature they can get out. That means if any ClientHello can induce a server to behave as if it doesn't support TLS 1.3, the attacker can do that and bypass this downgrade protection check.
This is relevant if, for example, the client supports TLS False Start over TLS 1.2, because then it means the Finished check doesn't work.
(This is a bit unfortunate because it means that this TLS 1.3 hardening measure is kind of a hair trigger. It does not tolerate the existence of any non-version-related reason to pick TLS 1.2 over TLS 1.3. In hindsight, the WG probably should have made a sign_whole_transcript TLS 1.2 extension that could be negotiated more unconditionally than TLS 1.3, and then used the random hack for it instead. I suppose it's not too late to do that, though I dunno how much appetite there is to add it to TLS 1.2 now.)

Thanks for the background, this is meant to to illustrate a possible alternative to #25733
And Yes, this kind of relies on the Finished verify_data to authenticate the handshake.
The situation with brainpool is this:
Due to my #7485 which added the ability to use TLS1.3 with brainpool it is now possible for server and client
to use these certificates, by default, when both have the latest openssl version, that is 3.2.1 or newer, which
was released in january of this year.  That is of course nice.
BUT it is not interoperable with applications that are written for previous version, where only TLS1.2 was able
to use brainpool, when both client and server explicitly add brainpool to the groups.
The client has no problem, if can now support both TLS1.3 and TLS1.2, while it previously only supported TLS1.2,
that is good.
But if the server tries the support both TLS1.3 and TLS1.2 by adding brainpool to the groups, this does not work.
It will always negotiate TLS1.3, except with super old clients, prior to 1.1.1 where only TLS1.2 was supported.
The only way out for my application, is now to roll out an update, that removes the brainpool signatures
from the server's sigalgs, but keeps the brainpool signatures for the client's sigalgs.  That must be done
in a way that does runtime detect the openssl version, so use the return code of the SSL_CTX_set1_sigalgs_list
see if the :ecdsa_brainpoolP256r1_sha256 is understood so we have 3.2.1 or newer runtime, and if not
undo the damage by setting sigalgs list without brainpool.  The server on the other hand must explicitly
use a sigalgs list without brainpool signatures, and might try to use a  -client_sigalgs which includes brainpool.
I need to roll out that before O/S vendors start to roll out the openssl 3.2, and keep that for 4-5 years,
until openssl-3.2 is available everywhere.
So I am wondering if it is better to permanently disable TLS1.3 or to accept this theoretic downgrade issue
for brainpool curves.
In other words, do the signatures from the TLS1.2 ServerKeyExchange expose any threat to the TLS1.3 protocol?


Feat/torxakis configuration via config file

TorXakis can be configured by using a configuration file (.torxakis.yaml).
The configuration file is expected either in the working directory or in the home directory. The working directory has precedence over the latter.
Fixes #47.
Created a new issue #197 since the solvers are still hardcoded in the unit tests.

Code climate reports one issue with a TODO in the ci/setup.sh file, but we need to leave it till https://github.com/TorXakis/TorXakis/issues/197 is solved.


zpp header have lots of errors

I'm getting around 84 errors while executing the example you provided. It prompts to the header file, serializer.h while compiling.

Can you share the errors? I've successfully compiled this in gcc, clang, and visual studio.
Perhaps you are using an old compiler which does not yet support C++14 features?

I am using c++11 compiler

I wanted to serialize nested map with the vector of integer as value inside the inner map

It should work with a C++14 comliler, can you validate that if you move to C++14 it works?

Closing due to inactivity.


Is there an easy grunt file template for this somewhere?

Not trying to be picky. This is a great resource.
It would be nice to be able to have a quick and easy grunt file to plug in, like a template for grunt-init or something to get grunt going out the box without have to write a grunt.js file.

I think providing templates for all possible task-runners is outside of the scope of this project. This is a basic starting point and should be as unopinionated as possible as far as specific tooling is concerned.
There are already npm scripts for the core tasks (such as starting a local server, running unit and e2e tests etc). If you want to use a task runner on top (grunt, gulp, brocolli, ???), you should provide the configuration file yourself imo.

:( understood.  It makes sense.  I'll just make one myself! With blackjack and hookers!
In all seriousness this an awesome resource, thanks much for ya'lls work on this.


CrossDCTest: Unable to load keys for caches that use UUID as key

There are two options

Use HotRod protocol to access the remote store with JBossMarshalling
Use Dataset provider functionality to access remote caches: https://github.com/keycloak/keycloak-benchmark/blob/main/dataset/src/main/java/org/keycloak/benchmark/cache/RemoteCacheResource.java

@mhajas - is this still necessary / still a problem? Can you add more context here if it is still necessary?

Sure. Yes, this is still necessary in my opinion.
The issue here is that the CrossDC testsuite accesses Infinispan caches using ISPN's Rest API. This works well for some simple tasks, for example cache size but not for more difficult tasks.
For example, we are currently not able to list keys for client session cache because the cache uses UUID key type and the REST API do not have support for that. btw. it is also hard to list user session keys (we need to do some nasty magic to parse it from REST API response).
The solution to this is to use something that understands the cache structure for these calls. There are two options:

Using HotRod client (we would need to get KC Externalizers from KC codebase to CrossDC testsuite so this is probably too difficult)
Use Dataset provider functionality to access remote caches: https://github.com/keycloak/keycloak-benchmark/blob/main/dataset/src/main/java/org/keycloak/benchmark/cache/RemoteCacheResource.java

The second option is preferred as it is already implemented in dataset provider. The missing part is to implement https://github.com/keycloak/keycloak-benchmark/blob/main/provision/rosa-cross-dc/keycloak-benchmark-crossdc-tests/src/test/java/org/keycloak/benchmark/crossdc/client/InfinispanClient.java interface with calls to Dataset provider. As a nice to have would be to filter the sessions we are obtaining based on the realm as currently we need to do another nasty magic to remove session ids created by Keycloak Admin Client.
The advantage of 1. would be we are not dependent on dataset provider (which we are anyway with lb-check) and would be secured as dataset provider is not secured at all.

Revert https://github.com/keycloak/keycloak-benchmark/commit/b0a408a3f88104e4671ceabd1b3811fb026ef4f6 when this is resolved


how to parse binlog from mysql with charsets gbk

Is there an config to parse binlog from mysql with charsets gbk?

Got it~~


Add possibility to Filter resources by Resource Group

Preflight Checklist

[x] I have installed the latest version of Storage Explorer.
[x] I have checked existing resources, including the troubleshooting guide and the release notes.
[x] I have searched for similar issues.

Problem
Unable to filter Subscription resouces by Resource Group
Desired Solution
I'd like to see a sub menu item on the subscription to select Resource Group/s to limit the amount of resources i see.  Working in large organization with hundreds of storage accounts make it difficult to see the ones i work with.
Alternatives and Workarounds
Alternatively i've used Quick Access to set the storage accounts i am working with.
Additional Context
No response

Please be aware that there is a setting to group resources by resource group in the tree view:

Let us know if this is sufficient for your needs.

Hi @MRayermannMSFT, thanks for pointing out this feature, but unfortunately i just then replace this issue with a high number of resource groups that the organization has.
Good thing about your suggestion is that I can then pin the Resource Group or more in the Quick Access.


How do you manage multiple environments?

For example, say production has a different database name than development. How would you manage this? Two thoughts come to mind:
Either make .env.development and .env.production and copy the the correct .env file over. Or set environment variables in the production environment that sets DB_NAME. However, isn't great since you can't have that in version control.
A third option is to completely drop the custom environment handling and use something like convict.

Hi @et
Yes just setup a .env file for your local environment.
You should not have multiple config files on one environment. In addition you should not commit any sensitive data to a repository (e.g. DB user and password).
In my opinion a other party needs to provide configuration. For example if you use cloud services like Cloud Foundry or Heroku (https://devcenter.heroku.com/articles/heroku-postgresql#connecting-in-node-js) you will load this configuration from the environment variables and this configuration is provided by the cloud provider.
What we do on Cloud Foundry is to setup a manifest.yaml to hold the more static env vars.
And if you have a CI like Travis or Jenkins you would also provide the config over the environment variables.
If you like to add an other environment or change the configuration of an environment you don't have to change the version control. You just have to change the config in the cloud and restart the app.
Thanks for the link to convict. Seams like an awesome library perfect for frontend (E.g. Webpack webapp) configuration (as there the configuration has to be loaded on at build/compile time).

Thanks for response. Env vars are the way to go. Totally forgot that dotenv doesn't overwrite existing env vars so that should be a safe path.
Thanks!


About prepare data

Hi,
When I try to run prepare.py, I find there is no space on the device:
OSError: [Errno 28] No space left on device
May I ask how to solve the space issue on v3-32?
Best,
Lucas

The instructions could be better 😄 . I prepared the dataset on a separate worker instance. A few hundred gb of HD space should be enough. Then copy the .tfrecord files to a GCS bucket in the same zone as the v3-32. It's very important to make sure the bucket you're using is in the same zone as the v3-32 instance.
Hope that helps!

hi @jenkspt
Thanks for your reply. I would like to ask if you could share the GCS bucket of the .tfrecord files. Since I do not have a device with a few hundred gb of HD space. Thanks in advance for your help.
Best,
Lucas

Can you use a GCS VM? It won't cost much.

Thanks for your help! I can use GCS VM now.

Hi @jenkspt
I find I cannot run the code successfully now.
The code will stop at this point:
SSH: Attempting to connect to worker 1...
SSH: Attempting to connect to worker 2...
SSH: Attempting to connect to worker 3...
INFO:root:using config file at config/gpt2.yaml
INFO:root:using config file at config/gpt2.yaml
INFO:root:using config file at config/gpt2.yaml
INFO:root:using config file at config/gpt2.yaml
wandb: Currently logged in as: lucasliunju. Use `wandb login --relogin` to force relogin
wandb: wandb version 0.15.4 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.15.0
wandb: Run data is saved locally in /home/supercomputer_ai/gpt-jax/wandb/run-20230627_081458-f1ojv7yv
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run gpt2_0626_v3_64
wandb: ⭐ View project at https://wandb.ai/lucasliunju/owt
wandb: 🚀 View run at https://wandb.ai/lucasliunju/owt/runs/f1ojv7yv
INFO:absl:Found no checkpoint directory at gs://lucas_time/vit/gpt2_v3_32_0627/checkpoints/train_state
INFO:absl:Found no checkpoint directory at gs://lucas_time/vit/gpt2_v3_32_0627/checkpoints/train_state
INFO:absl:Found no checkpoint directory at gs://lucas_time/vit/gpt2_v3_32_0627/checkpoints/train_state
PARAMETER COUNT: 124,475,904
INFO:absl:Found no checkpoint directory at gs://lucas_time/vit/gpt2_v3_32_0627/checkpoints/train_state```

I try to wait 1 hour and I cannot find the training loss in wandb and output log.

Thanks a lot. That does work now.


Fix the -batch <file> handling

Triggered by a tweet of Stephan Preibisch, this developer tried to run a
macro in batch mode via
.../ImageJ-win32 -batch abc.ijm

and was surprised that it did not work. The reason it did not work was a
completely borked logic behind the -batch handling, which assumed that
.../ImageJ-win32 -eval ... -batch was the only use case.
This patch fixes that logic to work correctly.
Signed-off-by: Johannes Schindelin<EMAIL_ADDRESS>

Cool, thanks @dscho!


porfolio -> portfolio

Already fixed by PR #64

Already fixed by PR #64

You know what's really cool? Merging another person's PR with these changes when this one came first

Alas, I'll never get credit for my incredible spell-check commit 😞 .  No worries


Choose and Adventure Game - Javascript, HTML 5, CSS 3

**Choose an Adventure Game is a game developed using:
HTML
Vanilla CSS
Javascript
**

** This is a game which can be fun as it maintains a suspense while one chooses one of the different cards given as options. **

Screenshots (optional)

Additional context (optional )

Please assign me this issue. This is a beginner-level project and can be contributed to. I'm a GSSOC 2023 contributor.


[10.x] Fix the issue of HasMiddleware compatibility

By chance, I have read this section. So, I realized it might be why @taylorotwell closed this PR #51058.

Changes in interfaces should be pushed to the master branch.
I see your previous PR was sent to master, but the reality is that it takes a while to understand what this PR is improving. (I am not a native English speaker, so take my comment with a grain of salt).
The problem I see is that new Laravel apps, created from version 11, have a plain class as their base controller, aimed just to share common code between controller classes, and it does no longer inherits from the Illuminate\Routing\Controller class.
Thus, the Illuminate\Routing\Controller@middleware method does not conflict with the Illuminate\Routing\HasMiddleware@middleware method definition.
https://github.com/laravel/laravel/blob/3b3f9f13faab1753e7b7cad6a0e7098e54c8199f/app/Http/Controllers/Controller.php#L1-L8
The scenario you describe, on PR #51058, applies to apps being migrated to Laravel 11.
For those, they can keep declaring middleware on their constructor, if their base controller still extends the Illuminate\Routing\Controller class, and skip implementing the Illuminate\Routing\HasMiddleware interface. So, no conflicts between the interface and the base class would happen.
Maybe proposing marking Illuminate\Routing\Controller as deprecated or internal on the next release, could be a better middle ground instead. So people migrating existing apps can adapt to the new proposed methodology for defining middleware within their controllers.

@rodrigopedra Also, it's not my native language :)
My PR intends to solve the issue that appears in that comment.
You say that the problem does not exist in Laravel v.11 Apps and it is not completely true because what if someone makes the plain class extends Illuminate\Routing\Controller and then implements the Illuminate\Routing\HasMiddleware, the error will be presented again!

The PR targets the versions lower than v.11 in a large percentage because the newly created controllers extend Illuminate\Routing\Controller by default which fires the problem if they implement the Illuminate\Routing\HasMiddleware.

So, the problem will exist in all versions, therefore, I am surprised that Taylor closed the PR but, it may mark the Illuminate\Routing\Controller deprecated as you said.

They aren't meant to be used together. If you extend base controller you don't use HasMiddleware.


Writing to DOM blocking other writes?

I've been working with this project this week. My initial results were great and I was super excited about what could be done with the Pi and Resin/Electron.
While trying to build a more complex page to display with Electron I noticed something odd. I have one socket.io connection that gets data and updates a div 10 times per second, and I have a socket.io connection that gets data and updates a different div every few seconds.
When the every few seconds div gets data and writes to it's div, the 10 times per second div had a hiccup for roughly 3 tenths of a second.
This isn't a problem running on chrome on my laptops browser, or even running an electron app on my windows laptop. Only with the Pi/Resin/Electron combo.
Anyways, I figured I'd throw it out there and see if any of you had a bright idea. I've tried a handful of things to speed up writing the data to the div etc but I still have the same result.
I did end up getting it to run smoothly with a epiphany kiosk option, but I really wanted to stick with Electron since I use it on the windows side of things.
Thanks!

I event thought maybe I'd try two separate BrowserWindow and it still seemed to act the same as before.

@CodyBranch it can be very likely due to the limited CPU resources, since this project is not hw accelerated on the rpi (yet)

It seems like you're correct. After some more testing I noticed when doing the same things on a small portion of the screen it was fine. When I tried to use the full screen it had issues.
Hopefully someone far smarter than me eventually adds in some HW acceleration :)

@CodyBranch I dream about that too :)
Please feel free to close the issue if you agree this is a X11/Chromium limitation on the target architecture

I agree! Issue closed.


Fix a bug that observation is not in np.float32

What changes do you make in this PR?

Please describe why you create this PR

Checklist

[ ] I have merged the latest main branch into current branch.
[ ] I have run bash scripts/format.sh before merging.
Please use "squash and merge" mode.

Merge this PR when all tests passed!


Add PSF warping

Add psf warping to MakeShearWarpTask.run

https://github.com/LSSTDESC/descwl_coadd/blob/cb1f30c93387bf214476c50cef2a79ce27117daa/descwl_coadd/coadd.py#L627
https://github.com/LSSTDESC/descwl_coadd/blob/cb1f30c93387bf214476c50cef2a79ce27117daa/descwl_coadd/coadd.py#L248

Invalid issue. PSF does not get warped in Warping task.

Where does it get warped?
Will it use our warping code to prevent biases?
How can we test it for consistency with the image warps?  This is also needed to prevent bias.

It gets warped during the coaddition stage. This is why I had to refactor out the PSF warping in the first place.
It uses warp_psf method.
One way would be to paste the PSF image in the calexp with the appropriate sub-pixel offset as some kind of a noiseless star and see that that is consistent with the PSF image on the coadd.

Now that I look at it, it delegates all of that to make_coadd function from descwl_coadd.

https://github.com/LSSTDESC/descwl-coadd-task/blob/main/descwl_coadd_task/_coadd_task.py#L451-L462

OK, in that case we can test it when we add a test_coadds.py


Re-implement "no-escape parameter call restriction" as SIL pass

My previous PR https://github.com/apple/swift/pull/23878 exposed a correctness issue. The following code was formerly rejected as an escaping use of a noescape function:
func test(fn: (() -> ()) -> ()) {
   func foo() {
     fn {}
   }
   fn(foo)
}

However, there is no escaping violation here, and the new implementation of escaping capture diagnostics made the bogus error go away. On the other hand, this example does violate the "no-escape parameter call restriction" rule, but we failed to diagnose it because the implementation of that rule was not taking transitive captures of local functions into account.
Finally, the Sema implementation of the NPCR rule had a bug that could cause an infinite recursion.
The new implementation looks at SIL instead, which makes it a bit simpler and fixes these correctness issues.
Fixes rdar://problem/34496304.

@swift-ci Please test

@swift-ci Please test source compatibility

Build failed
Swift Test Linux Platform
Git Sha - e2cb0572fedcb415ccfc09c87183cd0481926ddc

@swift-ci Please test Linux

@slavapestov I'm a little worried that we don't have any verification to ensure that follows the uses of non-escaping function parameters to ensure that all possible SIL patterns are recognized, thereby proving that this diagnostic is complete. But this is still a big improvement!

@slavapestov Is there some intuition behind this new error? I see it's about overlapping variable modifications, but I can't quite wrap my head around what a pathological case would look like.
I came across this error in the real world while trying to de-duplicate some wrapper functions I made. A reduced example:
Given some library functions I don't control,
func foo(_ f: () -> Void) -> Error?
func bar(_ f: () -> Void) -> Error?

I wrapped them like so to make them more Swift-friendly:
struct DefaultError: Error {}

func betterFoo<T>(_ body: () throws -> T) throws -> T {
    var result = Result<T, Error>.failure(DefaultError())
    if let error = foo({ result = Result(catching: body) }) {
        throw error
    }
    return try result.get()
}

func betterBar<T>(_ body: () throws -> T) throws -> T {
    var result = Result<T, Error>.failure(DefaultError())
    if let error = bar({ result = Result(catching: body) }) {
        throw error
    }
    return try result.get()
}

I didn't like having two copies of the same code, so I tried to deduplicate them with a helper function...but now I get this error.
func betterFoo<T>(_ body: () throws -> T) throws -> T {
    return try betterize(foo, body)
}
func betterBar<T>(_ body: () throws -> T) throws -> T {
    return try betterize(bar, body)
}

func betterize<T>(_ original: (() -> Void) -> Error?, _ body: () throws -> T) throws -> T {
    var result = Result<T, Error>.failure(DefaultError())

    // error: passing a closure which captures a non-escaping function parameter 'body' to a call to a non-escaping function parameter can allow re-entrant modification of a variable
    if let error = original({ result = Result(catching: body) }) {
        throw error
    }
    return try result.get()
}

Is there a workaround (other than presumably sprinkling in some @escapings) or at least an example of what could go wrong, to motivate why this code isn't accepted?
Thanks! :)


Error when creating DataResidualArray in GPU

Hello I incur in the following error when creating a DataResidualArray with a cupy array:
  File "/home/lcopparo/miniconda3/envs/emri/lib/python3.12/site-packages/lisaanalysistools-1.0.5-py3.12-linux-x86_64.egg/lisatools/datacontainer.py", line 227, in data_res_arr     self._data_res_arr = np.asarray(list(new_out), dtype=complex)
my conda installation uses the following dependencies:

cupy-cuda12x       13.2.0
numpy              1.26.4
CUDA/12.1

There are more GPU capabilities added to the code now. Please let me know if this error persists. I am closing this issue for now.


Feature: template changes

Changes to templates, like fixing the map, and a few copy changes too.

Copy changes done. Merging.


Fixed loads from the PROV (SITE_ABB) site

Thanks for the pull request Casey. The Travis build says that there were rows with some unexpected constituents:
NH3,OP, SI
Could those rows be removed?


Zero downtime migration doesn't work well with queue

If migration is performed with queue enabled, the new index will be created and the alias is switched to the new index but the index may not be complete due to the queue.
I would suggest to disable queue when migration is performed.
Alternatively, queue the switching so that it is performed after all migration is finished.

The latter is probably easier by queueing the last two function calls in the ElasticMigrateCommand: deleteSourceIndex() and createAliasForTargetIndex()

Hi @aygee, during migration the old index is used for reading and the new for writing.
If you are using queues for indexing, all data goes to the new index, that will be available right after migration is finished. I don't see a problem here, can you give me an example?

Hi @babenkoivan what I noticed is that the writing to the new index is queued, however the deleting of the old index and switching the alias to the new index are not queued. You can replicate this by running the migration on a large data and will find the data are missing on the alias until the queue is finished.
Or by turning of the queue worker so that it won't process the queue:

Create a new index with some data
Turn on queue in the scout config
Don't run the queue worker (so that they stay in the queue)
Run the migration
You will find that the alias has been switched to new index, but does not have any data

Also on a separate note, the migration can be improved by:

Having delete alias and create alias in one request.
Delete the old index after switching the alias, not before.

Hope that makes sense. If not, happy to explain more.

@babenkoivan PR here: #139

Hi @aygee, thanks for the explanation and sorry for taking so long to response!
I see now your concern, you expect to see fully ready to use index after migration, but it's filled with data only after some time.
But do you think turning off the queue until migration is over is a good idea? It will make unpredictable load on some environments and can take a long time to run. Maybe we can make it as an option for the command. What do you think?

@babenkoivan I think the option to turn it off is a good idea. In my case, I would need the data to be available right after migration. But you're right, it may not be suitable for everyone.
I've created a pull request to address the above with some other improvements on #139 . I will add the flag --no-queue or something like that. What do you think?

@aygee sounds great!
But I want to have a closer look at PR and unfortunately I'll have time to do that only during this weekend.

@babenkoivan I've updated the PR to add the --no-queue option. Please let me know what you think. Thanks :)

Hi @babenkoivan, any reasons why my PR was closed without merged? I have made the changes as you requested and also added comments for readability. I think my PR made the zero downtime migration to be a truly zero downtime which is great when you have large dataset.

Hi @aygee!
Sorry, if I offended you, it wasn't my intention.
I've added a comment in the PR after the first review and you didn't reply for couple of weeks, that's why I closed it.

@babenkoivan No worries. I actually pushed the changes as you requested on 4 Oct and replied to your PR comment pretty much the next day after your comment. Maybe you missed it. Any chance you can review again please. See the reply in here #139. Thanks a lot.

Hi @aygee, sorry for the late reply, I think you missed my comment. I can't find your answer on it.

That is so weird! It says 'Pending', not sure what that means. When I clicked 'Resolve conversation' just now, it's gone! Not sure why you couldn't see it. Why is this so hard??
Anyway, my comments were:

I have updated to call the parent::getOptions - see the updated commit on https://github.com/babenkoivan/scout-elasticsearch-driver/pull/139/commits/cd15dc15a2846a9062e187b85bddd29e3edc3d0f

I feel the code is necessary to get a true zero downtime migration by doing a remove and add in a single transaction as described in elastic search documentation. Please let me know how I can make it better?

Should I create another PR?

Hi @aygee.
I faced the same problems as you.
I believe that doing atomic index replacements is the right way. And the option to turn off the queue is also convenient.
@babenkoivan, I have a question about zero downtime migration algorithm.
Why do you switch the writing to the new index and do the reading from the old index? If the migration process takes even a few minutes, users will not be able to find the data that has just been saved.

@netsnatch I did a PR #139 a while ago which implemented a truly zero downtime based on elastic search documentation by doing an atomic index replacement. Unfortunately the PR was rejected although I put a lot of effort updating the PR and answering the questions that @babenkoivan had. I'm no longer using this package so I'm not really up to date where this project is at. Feel free to have a look at the PR and maybe try to fix / get it into the project.

@aygee I'm trying to give some attention to the PRs during the covid period. I'll take a look on your PR as soon as I stand on desktop tomorrow.


Real-time user activity/session tracking not working as expected

Describe the bug
Butler includes a Log4Net appender XML file that when deployed to a QSEoW server is supposed to provide Butler with information about starting and ending sessions.
This has worked in the past but in recent (tested Nov 2020) versions of QSEoW it partly fails.
The file in question is https://github.com/ptarmiganlabs/butler/blob/master/docs/log4net_user-audit-event/LocalLogConfig.xml.
Correct session messages are still sent to Butler, but there are also many, many incorrect messages sent by that XML file. Possibly QSEoW has changed logging in recent versions and now log more than earlier.
The result is that Butler receives (and forwards to for example Slack) messages such as

Expected behavior
The log appender xml file should only send valid session events to Butler.

The easy solution is to add filtering to the xml file so that only session start/stop events are forwarded to Butler:
<?xml version="1.0"?>
<configuration>
    <appender name="NodeUserLogger" type="log4net.Appender.UdpAppender">
        <filter type="log4net.Filter.StringMatchFilter">
            <param name="stringToMatch" value="Start session" />
            <param name="stringToMatch" value="Stop session" />
        </filter>
        ...
        ...

Fixed in coming 5.0 release.


Add support for staging optional configuration files along with app.yaml in flex staging

This is the opposite of https://github.com/GoogleCloudPlatform/appengine-plugins-core/issues/347. The current flex staging copies only app.yaml, but it'd be nice to have an additional option to include other config files when staging flex.
I know we have separate goals for deploying individual config files, but until the Eclipse plugin comes up with a working UI to enable that, we need a temporary support for copying all config files along with app.yaml. (https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/1895)

Closing (https://github.com/GoogleCloudPlatform/appengine-plugins-core/pull/392#issuecomment-298918039)


Direction reconstruction (also) from Telescope Frame

ctapipe.reco.HillasReconstructor has been modified to account for the possibility of receiving parameters obtained also in the TelescopeFrame.
The unit-test test_reconstruction has been improved:

it cleans and parametrizes the image also in the TelescopeFrame,
it compares the reconstruction from both frames
it is required that all of the shower fields are the same within a relative accuracy of 0.001%
after adding new test files to ctapipe-extra I now test for on Prod3b files in both parallel and divergent case separately

I have recycled my test scripts to make an updated version of the stereo reconstruction example named stereo_reconstruction_core_and_direction.py.
I tested parallel and divergent pointings, on-axis and off-axis.

reconstruction vs simulation:

pure divergent pointing using gamma_20deg_180deg_run9___cta-prod3-demo-2147m-LaPalma-baseline.simtel.gz and
comparing the old approach (i.e. cleaning and parametrizing in CameraFrame) against the new one (do that in TelescopeFrame)

all together (parallel from CameraFrame, parallel from TelescopeFrame and divergent with same pointing from TelescopeFrame - in this case, parallel and divergent must coincide)

This depends on psi, which (for some reason that I am missing) has different values between the two reconstructions.

Rotated by 90 degrees by any chance?

This depends on psi, which (for some reason that I am missing) has different values between the two reconstructions.

Rotated by 90 degrees by any chance?

No, they seem to be all different
PSI (PARALLEL MODE) = [-44.39868058 -71.33149472 -23.94571637 -18.19066478 -13.61923109
 -30.90901905 -15.93794037  63.15665631 -59.21130757] deg
PSI (DIVERGENT MODE) = [ -45.60131942  -18.66850528  -66.05428363  -71.80933522  -76.38076891
  -59.09098095  -74.06205963 -153.15665631  -30.78869243] deg

Is there a nice way to compare this to the old implementation where it's done int he CameraFrame? Might be a good way to see why psi is wrong now (or it could even be a bug in the original code).

For the Psi problem, I would suggest plotting the ground frame using the ArrayDisplay, and see if the angles make sense (also plot the Hillas ellipses  in the NominalFrame to see if they make sense as well).  It's really hard to debug 3D reconstruction without some plots like that.

Is there a nice way to compare this to the old implementation where it's done int he CameraFrame? Might be a good way to see why psi is wrong now (or it could even be a bug in the original code). Basically, I see 4 cases:
* CameraFrame reconstruction for parallel mode

* CameraFrame reconstruction for divergent mode

* TelescopeFrame reconstruction for parallel mode

* TelescopeFrame reconstruction for divergent mode

All should somehow be compared. As we discussed, we can eventually drop parallel mode, since it's really just a special case of divergent mode, and with mispointing, we will likely never need it. It is however nice, since the implementation is simpler, so might be easier to debug.

Comparing the reconstruction from these two frames is now automatic, because in this version the reconstructor accounts for both parameters' origin.
Regarding the difference between parallel and divergent pointing, it is sufficient to provide or not the telescope pointings to switch between the two modes.

Not really a comment on your changes, but I just noticed that inside we use spherical_to_cartesian from astropy.coordinates in HillasPlane.__init__, which I just found is about 50x slower than using SkyCoordinate.represent_as(CartesianRepresentation). So we could change those lines to speed this up a bit:
from astropy.coordinates import CartesianRepresentatino

cf = CameraFrame(focal_length=2*u.m)
x = SkyCoord(0.0*u.m, 1.0*u.m, frame=cf)
xt = x.transform_to(TelescopeFrame)

In [19]: %timeit xt.represent_as(CartesianRepresentation)
22.9 µs ± 271 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

In [20]: %timeit spherical_to_cartesian(1,xt.fov_lat, xt.fov_lon)
924 µs ± 7.88 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Not really a comment on your changes, but I just noticed that inside we use spherical_to_cartesian from astropy.coordinates in HillasPlane.__init__, which I just found is about 50x slower than using SkyCoordinate.represent_as(CartesianRepresentation). So we could change those lines to speed this up a bit:
from astropy.coordinates import CartesianRepresentation

cf = CameraFrame(focal_length=2*u.m)
x = SkyCoord(0.0*u.m, 1.0*u.m, frame=cf)
xt = x.transform_to(TelescopeFrame)

In [19]: %timeit xt.represent_as(CartesianRepresentation)
22.9 µs ± 271 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

In [20]: %timeit spherical_to_cartesian(1,xt.fov_lat, xt.fov_lon)
924 µs ± 7.88 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

I checked on this.
You are right, but one passage is missing in your example, namely our assumption on the azimuth convention: we are going in the opposite direction of what astropy expects.
This means that we need to change the sign to that component of the SkyCoord instance (as we do in fact).
I modified your test code adding this part, which would look like this taking into account how astropy manages memory,
xt.data.lon[()] = -1 * xt.data.lon[()]
xt.cache.clear()
xt.represent_as(CartesianRepresentation)

With this addition, it wouldn't be 50x faster, but 2, which is still better.
My test code is the following,
import timeit

import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord, spherical_to_cartesian, CartesianRepresentation
from ctapipe.coordinates import CameraFrame, TelescopeFrame

def test_time():

    setup = """\

import astropy.units as u
from astropy.coordinates import SkyCoord, spherical_to_cartesian, CartesianRepresentation
from ctapipe.coordinates import CameraFrame, TelescopeFrame

cf = CameraFrame(focal_length=2*u.m)
x = SkyCoord(-1.0*u.m, 1.0*u.m, frame=cf)
xt = x.transform_to(TelescopeFrame)

    """

    code_1 = """\

spherical_to_cartesian(1, xt.fov_lat, -xt.fov_lon)

    """

    code_2 = """\

xt.data.lon[()] = -1 * xt.data.lon[()]
xt.cache.clear()

xt.represent_as(CartesianRepresentation)

    """

    number=10000

    print(f"Time 'a' : {timeit.timeit(stmt=code_1, setup=setup, number=number)}")
    print(f"Time 'b' : {timeit.timeit(stmt=code_2, setup=setup, number=number)}")

def main():

    cf = CameraFrame(focal_length=2*u.m)
    x = SkyCoord(-1.0*u.m, 1.0*u.m, frame=cf)
    xt = x.transform_to(TelescopeFrame)

    a = spherical_to_cartesian(1, xt.fov_lat, -xt.fov_lon)

    xt.data.lon[()] = -1 * xt.data.lon[()]
    xt.cache.clear()
    xt.represent_as(CartesianRepresentation)

    b = xt.represent_as(CartesianRepresentation)

    print(f"a = {a}")
    print(f"b = {b}")

    for i,j in zip(a,b.xyz):
        np.testing.assert_almost_equal(i, j)

    test_time()

if __name__ == "__main__":
    main()

which returns,
a = (<Quantity 0.77015115>, <Quantity -0.42073549>, <Quantity -0.47942554>)
b = (0.77015115, -0.42073549, -0.47942554)
Time 'a' : 5.885553086
Time 'b' : 2.810596889

A little update about the timing improvement:
when I ask pytest to give the time of the unit-testing with
pytest --durations=0 ctapipe/reco/tests/test_HillasReconstructor.py::test_reconstruction
I actually find that the unit-test with what we thought was a faster algorithm is actually 3 seconds slower....

I don't think we should merge this PR in this state.
There are several open questions and some code smells.

I think cleaning up the TiltedGroundFrame version of psi is the main issue. Also  Thomas G and I had agreed a while ago to drop completely all logic related to "parallel" pointing, which will never really happen: divergent pointing and mis-pointing are handled in the same way, and there is always mis-pointing (except in perfect MCs).
Therefore, once we are happy that the algorithm give the same answer in "parallel" and "divergent" mode, we should just delete all code related to "parallel" mode, which will never exist in real life.

Therefore, once we are happy that the algorithm gives the same answer in "parallel" and "divergent" mode, we should just delete all code related to "parallel" mode, which will never exist in real life. Then it should be much simpler. Any problem with that?

I would vote to apply this cleanup later one in order to not mix too much material in this PR

I would vote to apply this cleanup later one in order to not mix too much material in this PR

Ok, let's just not forget it as a future cleanup, since the code as is is more complex than necessary.

I would vote to apply this cleanup later one in order to not mix too much material in this PR

Ok, let's just not forget it as a future cleanup, since the code as is is more complex than necessary.

I think we could open a cleanup PR to both CameraFrame usage and parallel pointing at once.

Let's just resolve where to put (or if it is needed to even store) the psi_divergent quantity.  I do not want to change HillasParametersContainer, as that changes the data model.

Let's just resolve where to put (or if it is needed to even store) the psi_divergent quantity. I do not want to change HillasParametersContainer, as that changes the data model.

It needs to be stored, otherwise, it is not possible to plot the core direction reconstruction (it's true that we can already plot the reconstructed and simulated core position, but this limits studies and debugging).
I think the cleanest solution would be to save it together with the other shower quantities in the ReconstructedShowerContainer and add this to the methods of `ArrayDisplay.
The only "problem" I would like to raise (I actually thought it just now...) is that the current solution (store it in HillasParametersContainer) in my opinion makes more sense, because all the info to recover psi would be later store into the DL1 files (the orginal psi and the telescope pointings).
If we put it in a DL2 container (thus it will end up in a DL2 file), then to compare how the cleaning affects shower reconstruction performances, we need always both files.

Yes, that's another point. For what would this value be needed after the HillasReconstructor is done?
Is psi_corr the psi in NominalFrame?
If so, wouldn't calculating the hillas parameters in NominalFrame directly give the correct values and contain well-defined values for all HillasParameters ?

Yes, that's another point. For what would this value be needed after the HillasReconstructor is done?
Is psi_corr the psi in NominalFrame?

No, the corrected value of psi (when it needs to be corrected of course) is a quantity defined in the "fake" camera frame, which is alway needed to estimate the core position and than )optionally) plot this estimation

What is the "Fake" camera frame?

What is the "Fake" camera frame?

As I explained above (one of the sub-discussions where I referenced the thesis of Thomas Gasparetto) it is the frame in which divergent pointing information gets "normalized" to create the equivalent common pointing correspondent to the array pointing.
This is because the camera planes in that case are never parallel to the Titled frame where the core is estimated.

But shouldn't it be possible to define the planes based on the individual pointing direction and the psi? Why is this parallel camera frame needed?

the camera planes do not require psi, thery are simply the planes orthogonal to the pointing direction of the telescope (so they need only this information

I am talking of the hillas planes. The planes through the main shower axis.

But before doing this you need to project the angle onto the ground from the TiltedFrame.
In divergent mode, the single cameraframes (where psi is defined) are not parallel to the Titled frame.
This does not allow to directly project them on the ground, because angles are not conserved when projected between non-parallel planes.
So we restart from the sky (where the positions are "right") we go to the fake camera frame and we recalculate psi there (it is indeed a re-calculation than a correction), so we are back to the original case of no divergent pointing .

I see. I think the problem here is with using the tilted frame for something it was not designed to handle.
It should be possible to project each of the Shower axis to the ground and then do the intersection without recomputing this psi.

I am talking of the hillas planes. The planes through the main shower axis.

Those are not used in core estimation, only for the direction in the sky.
The planes which have to do with core estimation are the camera planes (also in 3D) which are parallel to the TiltedFrame

Shouldn't the Impact on the ground be the intersection of the Hillas Planes at z=0?

I see. I think the problem here is with using the tilted frame for something it was not designed to handle.

Why not?

It should be possible to project each of the Shower axis to the ground and then do the intersection without recomputing this psi.

Maybe we can do in a sub-sequent PR, maybe it will trigger bigger discussion and for now we have a versino of the algorithm that is good I would say.

We should not merge this in an unfinished state.
I think this corrected psi should not be unnecessary and formost it should not be stored in the hillas parameters container.

UPDATE

updated ctapipe-extra to 0.3.1
2 unit-tests

specific for parallel pointing using prod3b test file La Palma
specific for divergent pointingusing specific prod3b test file La Palma

Both tests demonstrate that the new approach (from TelescopeFrame) can reconstruct showers with all numeric variables with 0.001% from the ones calculated using the old approach (from CameraFrame).

Ok, regarding where to store the value of psi_core:

HillasParametersContainer seems now to be out of question,
ReconstructedShowerContainer could have been cleaner, but we cannot save telescope-wise information there.

A third alternative could be to create a specific container for this that will be only used when calling the plotting methods of  ctapipe.visualization.ArrayDisplay.

Maybe we don't need to store it at all.
Could you just make a general method to calculate this correction? Which could then be called both by the HillasReconstructor and the plotting code?

Maybe we don't need to store it at all.
Could you just make a general method to calculate this correction? Which could then be called both by the HillasReconstructor and the plotting code?

Not sure if this is a good idea - even though this quantity is only used in plotting, it still a property of each shower image on the ground, and we shouldn't leave information out.

Maybe we don't need to store it at all.
Could you just make a general method to calculate this correction? Which could then be called both by the HillasReconstructor and the plotting code?

Not sure if this is a good idea - even though this quantity is only used in plotting, it still a property of each shower image on the ground, and we shouldn't leave information out.

Though after thinking about it I am not completely in disagreement with this proposal.
I'll test it

Coming back to where to put this: this is really just another set of Hillas Parameters, except you only really need one of them: psi, for plotting purposes.  Basically you have HillasParameters in the TelescopeFrame, which you use to do point-of-origin reconstruction (using mainly the centroid position, and psi parameters), and then you also need TiltedFrame hillas parameters for the ground reconstruction (for which really you can use the same centroid position, but you need a new psi since the coordinate system has changed).  So either we:

make a transform function to go from psi_telescope to psi_tilted (similar to what @maxnoe suggests above)
compute and store both sets of hillas parameters per telescope (a bit of a waste of computation)
invent a new container that just stores the hillas parameter that is different (psi), which is only really needed for plotting.

I think I like 1 the best, since it's the simplest, but requires some thought how to implement correctly.

So there are 2 parts of the problem:

we need to calculate the correction before estimating the core position anyway
we need to migrate some information (which is not contained only in the hillas dictionary) from HillasReconstructor.initialize_hillas_planes to e.g. ArrayDisplay.set_line_hillas

In particular this method would need at least,

self.divergent_mode from HillasReconstructor
self.mode from HillasReconstructor

If this method doesn't calculate psi_core then this variable needs to be at least returned by the reconstructor.
If this method has to also re-calculate this variable at plotting time then this method needs to read also,

subarray,
array_pointing,
cog_coord & p2_coord which are the positions in the sky telescope-wise (these are used to initilize the Hillas planes and never recorded afterwards

You realize that at a cetain point I cannot copy-paste the entire HillasReconstructor.initialize_hillas_planes...

So for me, the best and cleaner way would be to make HillasReconstructor.estimate_core_position to return psi_core (which remeber is a dictionary based on tel_id)
Right now HillasReconstructor creates a ReconstructedShowerContainer which already containes the list of tel_ids which is therefore already a dictionary based on tel_id.
Would it be so bad to save psi_core there and at display time read only the HillasReconstructor.predict instance as an additional argument?

I mean saving psi_core into ReconstructedShowerContainer should have nothing to do with the DL2 format, am I right?
In the DL2 write you would simply select the Fields you want.

You can't put it in ReconstructedShowerContainer, as that is for per-shower (not per-telescope) information, and furthermore that is not an intrinsic property of the shower.    Fields cannot be dictionaries (they are not writable to a table).
If you store it in a container it must be in a per-telescope container, like the ImageParametersContainer, for which one is generated for each telescope. Right now inside that there is a Hillas field that stores the Hillas parameters for a given telescope.  So I'd suggest we just make a new sub-container in there parameters.tilted.psi_angle.  It's not pretty, but would work and be consistent with where data should go.  So you would need to define for example:
class CoreParametersContainer(Container):
     """ hillas-style paramteers computed in the Tilted frame for a given camera """
      container_prefix="core"
     psi = Field(nan*u.deg, "rotation angle of ellipse in the tilted frame", units="deg")

Then add that to ImageParametersContainer, and everything should work (it will even be automatically written out using the DL1Writer in #1499 )

event.dl1.tel[tel_id].parameters.core.psi = psi_in_tilted

this would require the reconstructor to read also the event and not only the hillas dict and pointing

this would require the reconstructor to read also the event and not only the hillas dict and pointing

I think you can just pass those from the event to the reconstructor when calling it, right?
That is going to need to change anyhow, as the way the reconstructor works is different from the other "large" Components like the CameraCalibrator or ImageProcessor,  or DL1Writer which each take an "event" structure in their __call__ method.  We could do that later by either adding another class, e.g. Reconstructor, that internally constructs a HillasReconstructor or other type, and has a __call__(self, event: ArrayEventContainer) method like those other classes.
We could even just add a __call__ to the existing constructors that extract the relevant info from the ArrayEventContainer and call reconstructor.predict() with the right options

@HealthyPear can you resolve the conflicts?
Anything else left to be done?

Ok, conflicts were simple, done.

No idea why the CI fails at ctapipe/instrument/tests/test_subarray.py::test_hdf while it works for me locally...

My main concern now is that this now mixes low and high-level algorithms. Now that we have a  ShowerProcessor (high-level), HillasReconstructor (low-level) shouldn't deal with events directly, only the data it needs. That was the design decision we came to when making the ShowerProcessor. Basically, it should be possible to use HillasReconstuctor without having the data in an ArrayEventContainer, like all other low-level algorithms.
But maybe it is ok to keep it this way, it's just a bit confusing compared to other algorithms. But I would rather see HillasReconstructor.__call__ return a ReconstructedGeometryContainer, so it can be used outside a normal event loop.

the call now (but even before) doesn't return directly the container but it writes it to the  right place in the ArrayEventContainer
what returns the container is the _predict method
since one can always use the low-level API I thought that it makes more sense for the shower processor to call the reconstruction generally so to write the data within the call

since one can always use the low-level API

A private method is not part of the API and we should't encourage using methods that start with underscores.

Should we then revert the class to return the container when called on an event?
Any idea what might be the cause of the CI failure? I can't reproduce this problem locally.


[BitSail][Deployment] Support local run without outside hadoop dependencies

Is your feature request related to a problem? Please describe
Current hadoop dependency in BitSail are in provided scope.
When in local deployment, there could be no hadoop dependency, which leads to failure.
So we want to support local running without outside hadoop dependencies.

already support local runtime


Install issue regarding Mac OS X and nokogiri

I have successfully installed vagrant-aws on a Mac many times in the past, but now this is failing consistently (10.9.3) on both Vagrant 1.5.4 and Vagrant 1.6.2, on multiple Macs:
$ vagrant -v
Vagrant 1.5.4
$ vagrant plugin install vagrant-aws
Installing the 'vagrant-aws' plugin. This can take a few minutes...
Building nokogiri using packaged libraries.
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:
An error occurred while installing nokogiri (<IP_ADDRESS>), and Bundler cannot continue.
Make sure that gem install nokogiri -v '<IP_ADDRESS>' succeeds before bundling.

I haven't noticed this one in a while. Something must have fixed it.

facing similar issue on OS X 10.12.5
➜  ~ vagrant -v
Vagrant 1.9.5
`Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:
ERROR: Failed to build gem native extension.
/opt/vagrant/embedded/bin/ruby -r ./siteconf20170518-9976-g6h5d2.rb extconf.rb

checking if the C compiler accepts ... yes
checking if the C compiler accepts -Wno-error=unused-command-line-argument-hard-error-in-future... no
Building nokogiri using packaged libraries.
Using mini_portile version 2.1.0
checking for iconv.h... yes
checking for gzdopen() in -lz... yes
checking for iconv using --with-opt-* flags... no
checking for iconv... no
pkg-config could not be used to find libiconv
Please install either pkg-config or the pkg-config gem per
gem install pkg-config -v "~> 1.1.7"

checking for iconv using pkg-config libiconv... no
libiconv is missing.  Please locate mkmf.log to investigate how it is failing.
*** extconf.rb failed ***`

I can confirm this isn't working with Vagrant 1.9.6 on macOS Sierra 10.12.5.
See errors at https://gist.github.com/perlporter/b070fceaa19893cc17fd1c96e3a9ba34


Parameterize balanceTransaction over era type parameter

[x] Mostly mechanical changes to parameterize balanceTransaction over era

Motivation

Simplifies error handling in various helper functions
Makes it easier to in the future use a single Cardano.UTxO era across the entire balanceTransaction function.

Issue Number
ADP-1642

bors r+

bors r+

bors r+

The current bors job appears to be stuck here:

However, the tests for that particular job have passed:
Network.Wai.Middleware.Logging
  Logging Middleware
    GET, 200, no query (502ms)
    GET, 200, with query (503ms)
    GET, 200, not json (504ms)
    POST, 201, with sensitive fields (503ms)
    POST, 400, invalid payload (not json) (503ms)
    DELETE, 202, no query (503ms)
    GET, 400 (504ms)
    GET, 500 (503ms)
    GET, 503 (503ms)

Finished in 2926.6633 seconds, used 2003.1041 seconds of CPU time
4348 examples, 0 failures, 43 pending

Apart from this particular job, all other jobs have passed.
Will merge this manually, as we've now tried to merge this with bors several times unsuccessfully.


Package.getStylesheetsPath is deprecated.

Store package style sheets in the styles/ directory instead of stylesheets/ in the terminal package
Package.getStylesheetsPath (/Applications/Atom.app/Contents/Resources/app.asar/src/package.js:460:9)
Package.getStylesheetPaths (/Applications/Atom.app/Contents/Resources/app.asar/src/package.js:471:32)
Package.loadStylesheets (/Applications/Atom.app/Contents/Resources/app.asar/src/package.js:453:38)
<unknown> (/Applications/Atom.app/Contents/Resources/app.asar/src/package.js:183:19)
Package.measure (/Applications/Atom.app/Contents/Resources/app.asar/src/package.js:163:15)
Package.load (/Applications/Atom.app/Contents/Resources/app.asar/src/package.js:177:12)

i have same problem

All repositories in the atom-archive organization are unmaintained and deprecated. Because of this, we are archiving this repository and closing all issues and pull requests. Thanks very much for your support and contributions!


Dataset chondro

Dataset chondro is removed from the devel version of hyperSpec thus this package is not compatible with hyperSpec 0.100.0 or newer as it extensively uses chondro in examples and vignettes. Thus, I added hySpc.chondro as a dependency (2ebd1ba, 9a67761).
If this is not the desired behavior, please do something about this. Otherwise, close the issue.

This is a duplicate of #39.


Supporting sending replies

This and #26 will make replying possible.

Ready for review.

Just those two comments. Looking good to me otherwise.

Ready for re-review.

:+1:


Not working with rasbian stretch os.

I tried to install this on stretch os but as stretch uses php7.0 so the php5.0 installation command didnot work. So i went to the script file and changed it to php7.0 and installed hsmm-pi and it installed successfully but when i opened the browser to configure the network interface and try to save the new configuration it started to show internal error message.
I went to the code and found out that error500.ctp is being executed and i was unable to fix it.
So i installed jessie and it worked fine but try to fix the error for php7.0

Not working with php7.0


Update README with correct npm link

The changed introduced in https://github.com/spatie/spatie-attachment-uploader/commit/a62d16e2ea8d45c56d43ef1a8a04ada641961fc4 was not reflected in the README badge.

Thanks!


Enhance Get-VSTeamProcess #300

PR Summary
Get-VSTeamProcess now:

Gets information from the /work/processes URI path rather than process/processes as the former has additional details
Can query for processes using a wildcard name.
Updates when it runs  the cached process information
Displays whether a process was disabled or is the default process.

PR Checklist

[X] [Write Help]
(Small updates, mostly reflecting the different API provides more information but doesn't support paged responses)
[X] [Write Unit Test]
(Fixed existing ones.)
[X] [Update CHANGELOG.md]

I do not want to change the build process in any way at this time.

I've changed my build so that

_Classes.json is as as you had it and vsteam.classes.ps1 gets built.
vsteam.psm1 dot sources vsteam.classes.ps1 as you had it
vsteam.psm1 is copied as is to the output directory, as you had it.

I have also added a switch to build module if specified

vsteam.psm1 in the output directory is appended to vsteam.classes.ps1
the now-extended vsteam.classes.ps1 is copied over vsteam.psm1.
vsteam.classes.ps1 will still be dot sourced so is replaced with  "#Empty"

So have we can development that little bit easier with a build-time switch and the "normal" build process is unaffected.

I do not want any testing flags of any kind.

I found one last stray one in Add-VSTeamBuild.ps1  they are all removed from my branches now (I think)

Would be nice to apply the Update -List pattern to all cache classes.

See mu other comment - that's definitely the direction of travel.

How are you merging the changes from upstream?  I see 235 files changed but they really didn't it is making it very time consuming to review this PR.

How are you merging the changes from upstream? I see 235 files changed but they really didn't it is making it very time consuming to review this PR.

It was some time back... I think I pulled upstream into to my master branch and then merged master into this branch which had already been created for the PR.

Switching Get-VSTeamProcess from Processes to Work breaks TFS2017. I am working switching the call based on the API version being used. I will be merging these changes from another PR because there are too many files changed in this PR to make sense of the actual changes.


Fix compiler warnings about pointer integer conversion

that is it.

Most recent version we have changed the ap_find_command to msc_remote_find_command (that is our own implementation of ap_find_command). That made easy the compilation of ModSecurity in different platforms. I am closing this Pull Request.


Sos flooding cluster login node

Since upgrading to 0.22.3 (and similarly with 0.22.4) I've had issues using my HPC cluster. I've kept my notebook and hosts.yml file the same, so maybe I need to change them to keep up with the recent changes.
 Also the cluster login node slows to a crawl. When I run top on the cluster login node I see
[pgc29@farnam2 ~]$ top

top - 03:47:34 up 83 days, 17:01, 48 users,  load average: 108.98, 68.53, 32.03
Tasks: 1169 total,   3 running, 1165 sleeping,   1 stopped,   0 zombie
%Cpu(s):  0.4 us,  2.0 sy,  2.3 ni, 91.6 id,  3.7 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 13118614+total, 96860144 free, 23155532 used, 11170472 buff/cache
KiB Swap: 16777212 total, 16334836 free,   442376 used. 10499411+avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 2236 root       0 -20   30.9g  18.6g   1.8g S  10.3 14.9  16069:10 mmfsd
 4058 pgc29     30  10   42720   3808   1404 S   1.6  0.0   0:00.05 lua
 4061 pgc29     30  10   42744   3828   1404 S   1.6  0.0   0:00.05 lua
 4083 pgc29     30  10   42856   3916   1404 S   1.6  0.0   0:00.05 lua
 4057 pgc29     30  10   43152   4284   1436 D   1.2  0.0   0:00.05 lmod
 4060 pgc29     30  10   42744   3828   1404 S   1.2  0.0   0:00.04 lua
 4064 pgc29     30  10   42744   3832   1404 S   1.2  0.0   0:00.04 lua
 4077 pgc29     30  10   43152   4248   1436 D   1.2  0.0   0:00.04 lmod
 4080 pgc29     30  10   42744   3828   1404 D   1.2  0.0   0:00.04 lua
 4094 pgc29     30  10   42744   3828   1404 S   1.2  0.0   0:00.04 lua
 4123 pgc29     30  10   42748   3820   1404 D   1.2  0.0   0:00.04 lua
 4161 pgc29     30  10   42744   3832   1404 D   1.2  0.0   0:00.04 lua
 1303 dbus      20   0   79204   2900   1900 S   0.9  0.0  24:10.72 dbus-daemon
 2855 pgc29     30  10  131636   7384   2808 D   0.9  0.0   0:00.11 conda
 3085 pgc29     30  10  128692   4704   2572 D   0.9  0.0   0:00.05 conda
 4072 pgc29     30  10   42724   3820   1404 S   0.9  0.0   0:00.03 lua
 4084 pgc29     30  10   42744   3828   1404 S   0.9  0.0   0:00.03 lua
 4093 pgc29     30  10   42744   3812   1404 D   0.9  0.0   0:00.03 lua
 4095 pgc29     30  10   42744   3828   1404 S   0.9  0.0   0:00.03 lua
 4106 pgc29     30  10   42744   3824   1400 D   0.9  0.0   0:00.03 lua
 4110 pgc29     30  10   42744   3828   1404 D   0.9  0.0   0:00.03 lua
 4117 pgc29     30  10   42744   3832   1404 D   0.9  0.0   0:00.03 lua
 4128 pgc29     30  10   42744   3828   1404 D   0.9  0.0   0:00.03 lua
 4129 pgc29     30  10   42744   3828   1404 D   0.9  0.0   0:00.03 lua
 4136 pgc29     30  10   42744   3828   1404 D   0.9  0.0   0:00.03 lua
 4138 pgc29     30  10   42744   3820   1400 D   0.9  0.0   0:00.03 lua
 4141 pgc29     30  10   42724   3820   1404 D   0.9  0.0   0:00.03 lua
 4142 pgc29     30  10   42744   3832   1404 D   0.9  0.0   0:00.03 lua
 4146 pgc29     30  10   42744   3832   1404 D   0.9  0.0   0:00.03 lua
 4176 pgc29     30  10   42744   3824   1400 D   0.9  0.0   0:00.03 lua
 8793 pgc29     30  10  114328   2700   1352 S   0.9  0.0   0:00.04 bash
 9052 pgc29     30  10  114320   2724   1352 S   0.9  0.0   0:00.03 bash
30940 pgc29     30  10  138476  11884   2328 D   0.9  0.0   0:00.41 conda
31555 pgc29     30  10  163220   3792   1848 R   0.9  0.0   0:00.29 top

This setup was working fine previously and would only spawn one or two sos tasks on the login node. I think the lua processes come from the lmod module system, and the module load miniconda part of my script. It's as if multiple jobs are trying to load modules directly on the login node, rather than the jobs being submitted to sbatch and then loading the modules on the compute nodes. Any ideas?

Named path is. better supported in the newer version so you could write /home/pgc29/scratch60/ as #scratch/ but this should not be the reason for the problem.
Where is the notebook being executed? Is it on the headnode? Do you see the same configuration is you run !sos config from the notebook?

@pgcudahy I will need more information to debug this.

@pgcudahy I will need more information to debug this.

Sorry to bring up an issue and then not follow up. With the COVID surge
I've been doing all clinical work. I'll see if I can get some time to work
on it a bit this weekend.
Patrick
On Fri, Jan 8, 2021 at 8:51 PM Bo<EMAIL_ADDRESS>wrote:

@pgcudahy https://github.com/pgcudahy I will need more information to
debug this.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vatlab/sos/issues/1424#issuecomment-756934267, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABMGASAV3O7VBRZKJW5IRNDSY5H2RANCNFSM4VLYFLIQ
.

Sorry to bring up an issue and then not follow up. With the COVID surge
I've been doing all clinical work. I'll see if I can get some time to work
on it a bit this weekend.
Patrick
On Fri, Jan 8, 2021 at 8:51 PM Bo<EMAIL_ADDRESS>wrote:

@pgcudahy https://github.com/pgcudahy I will need more information to
debug this.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vatlab/sos/issues/1424#issuecomment-756934267, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABMGASAV3O7VBRZKJW5IRNDSY5H2RANCNFSM4VLYFLIQ
.

I'm running the notebook on my computer with the university cluster as the remote target. Running !sos config gives me the same output as my hosts.yml
'yale_hpc_slurm': { 'address'<EMAIL_ADDRESS>                                  'kill_cmd': 'scancel {job_id}',
                                  'max_cores': 200,
                                  'max_mem': '1200G',
                                  'max_running_jobs': 200,
                                  'max_walltime': '72:00:00',
                                  'paths': { 'home': '/home/pgc29/',
                                             'scratch': '/home/pgc29/scratch60/'},
                                  'queue_type': 'pbs',
                                  'sos': '/home/pgc29/.local/bin/sos',
                                  'status_check_interval': 120,
                                  'status_cmd': 'squeue --job {job_id}',
                                  'submit_cmd': 'sbatch {job_file}',
                                  'submit_cmd_output': 'Submitted batch job '
                                                       '{job_id}',
                                  'task_template': '#!/bin/bash\n'
                                                   '#SBATCH --time={walltime}\n'
                                                   '#SBATCH --nodes={nodes}\n'
                                                   '#SBATCH '
                                                   '--ntasks-per-node={cores}\n'
                                                   '#SBATCH --mem-per-cpu={mem '
                                                   '// cores //<PHONE_NUMBER>}G\n'
                                                   '#SBATCH --job-name={task}\n'
                                                   '#SBATCH '
                                                   '--output=/home/pgc29/.sos/tasks/{task}.out\n'
                                                   '#SBATCH '
                                                   '--error=/home/pgc29/.sos/tasks/{task}.err\n'
                                                   '#SBATCH --requeue\n'
                                                   '#SBATCH '
                                                   '--partition=general,scavenge\n'
                                                   'cd {workdir}\n'
                                                   '{command}\n'}

I'm running the notebook on my computer with the university cluster as the remote target. Running !sos config gives me the same output as my hosts.yml
'yale_hpc_slurm': { 'address'<EMAIL_ADDRESS>                                  'kill_cmd': 'scancel {job_id}',
                                  'max_cores': 200,
                                  'max_mem': '1200G',
                                  'max_running_jobs': 200,
                                  'max_walltime': '72:00:00',
                                  'paths': { 'home': '/home/pgc29/',
                                             'scratch': '/home/pgc29/scratch60/'},
                                  'queue_type': 'pbs',
                                  'sos': '/home/pgc29/.local/bin/sos',
                                  'status_check_interval': 120,
                                  'status_cmd': 'squeue --job {job_id}',
                                  'submit_cmd': 'sbatch {job_file}',
                                  'submit_cmd_output': 'Submitted batch job '
                                                       '{job_id}',
                                  'task_template': '#!/bin/bash\n'
                                                   '#SBATCH --time={walltime}\n'
                                                   '#SBATCH --nodes={nodes}\n'
                                                   '#SBATCH '
                                                   '--ntasks-per-node={cores}\n'
                                                   '#SBATCH --mem-per-cpu={mem '
                                                   '// cores //<PHONE_NUMBER>}G\n'
                                                   '#SBATCH --job-name={task}\n'
                                                   '#SBATCH '
                                                   '--output=/home/pgc29/.sos/tasks/{task}.out\n'
                                                   '#SBATCH '
                                                   '--error=/home/pgc29/.sos/tasks/{task}.err\n'
                                                   '#SBATCH --requeue\n'
                                                   '#SBATCH '
                                                   '--partition=general,scavenge\n'
                                                   'cd {workdir}\n'
                                                   '{command}\n'}

Thanks. I will try to reproduce the problem.

Thanks. I will try to reproduce the problem.

I tried to make a smaller more reproducible example.
My new notebook cell is
%run test_hpc2 -q yale_hpc_slurm

[test_hpc2]
import os
import glob
kraken_fastq_files = sorted(glob.glob('/data2/helen_mixed_infection/data/kraken2/*.kraken2'))
remote_kraken_files = ["/home/pgc29/scratch60" + i.split("/data2")[1] for i in kraken_fastq_files]

input: remote(remote_kraken_files[0:10]), group_by=1
output: remote(f'/home/pgc29/scratch60/helen_mixed_infection/data/kraken2/mtb/\
{os.path.split(_input[0].resolve())[1].split(".kraken2")[0]}.test')
task: walltime='00:60:00', mem='5G', trunk_size=10, trunk_workers=10,
    workdir='/home/pgc29/scratch60/helen_mixed_infection/data/kraken2'

run: expand=True
    touch {_output}
    echo {os.path.split(_input[0].resolve())[1].split(".kraken2")[0]} > {_output}

With the same hosts.yml as above. This just churns away with no output on sos status but there's some weird activity when I run top on the cluster login node.  It's good that it completes, but not sure why it's taking very long to set up the job.

I tried to make a smaller more reproducible example.
My new notebook cell is
%run test_hpc2 -q yale_hpc_slurm

[test_hpc2]
import os
import glob
kraken_fastq_files = sorted(glob.glob('/data2/helen_mixed_infection/data/kraken2/*.kraken2'))
remote_kraken_files = ["/home/pgc29/scratch60" + i.split("/data2")[1] for i in kraken_fastq_files]

input: remote(remote_kraken_files[0:10]), group_by=1
output: remote(f'/home/pgc29/scratch60/helen_mixed_infection/data/kraken2/mtb/\
{os.path.split(_input[0].resolve())[1].split(".kraken2")[0]}.test')
task: walltime='00:60:00', mem='5G', trunk_size=10, trunk_workers=10,
    workdir='/home/pgc29/scratch60/helen_mixed_infection/data/kraken2'

run: expand=True
    touch {_output}
    echo {os.path.split(_input[0].resolve())[1].split(".kraken2")[0]} > {_output}

With the same hosts.yml as above. This just churns away with no output on sos status but there's some weird activity when I run top on the cluster login node.  It's good that it completes, but not sure why it's taking very long to set up the job.

This might be caused by #1411
Basically, before this release sos trusts that the remote file exists and it does not save signatures of remote files. This caused a problem, as described in #1411, that the workflow would not re-execute when a remote file is changed.
The solution was to check the existence and signature of remote files, which was implemented as something like
ssh remotehost sos get_signature_of file

This is done for each substep, so separately for each remote file. Multiple sos processes will appear on the cluster, which will be slow especially on a crowded headnode.
Let me see if I can at least check the signatures of all files in one run.

This might be caused by #1411
Basically, before this release sos trusts that the remote file exists and it does not save signatures of remote files. This caused a problem, as described in #1411, that the workflow would not re-execute when a remote file is changed.
The solution was to check the existence and signature of remote files, which was implemented as something like
ssh remotehost sos get_signature_of file

This is done for each substep, so separately for each remote file. Multiple sos processes will appear on the cluster, which will be slow especially on a crowded headnode.
Let me see if I can at least check the signatures of all files in one run.

That makes sense. I've also been avoiding the #scratch notation because it
also takes tens of minutes to transfer a few files over a gigabit lan
connection. Instead I've been using rclone to sync the input and output
files and using remote() in my scripts.
On Fri, Jan 15, 2021 at 5:20 AM Bo<EMAIL_ADDRESS>wrote:

This might be caused by #1411 https://github.com/vatlab/sos/issues/1411
Basically, before this release sos trusts that the remote file exists and
it does not save signatures of remote files. This caused a problem, as
described in #1411 https://github.com/vatlab/sos/issues/1411, that the
workflow would not re-execute when a remote file is changed.
The solution was to check the existence and signature of remote files,
which was implemented as something like
ssh remotehost sos get_signature_of file
This is done for each substep, so separately for each remote file.
Multiple sos processes will appear on the cluster, which will be slow
especially on a crowded headnode.
Let me see if I can at least check the signatures of all files in one run.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vatlab/sos/issues/1424#issuecomment-760619184, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABMGASBOS6PLJMY4P27MXBLSZ6X7PANCNFSM4VLYFLIQ
.

That makes sense. I've also been avoiding the #scratch notation because it
also takes tens of minutes to transfer a few files over a gigabit lan
connection. Instead I've been using rclone to sync the input and output
files and using remote() in my scripts.
On Fri, Jan 15, 2021 at 5:20 AM Bo<EMAIL_ADDRESS>wrote:

This might be caused by #1411 https://github.com/vatlab/sos/issues/1411
Basically, before this release sos trusts that the remote file exists and
it does not save signatures of remote files. This caused a problem, as
described in #1411 https://github.com/vatlab/sos/issues/1411, that the
workflow would not re-execute when a remote file is changed.
The solution was to check the existence and signature of remote files,
which was implemented as something like
ssh remotehost sos get_signature_of file
This is done for each substep, so separately for each remote file.
Multiple sos processes will appear on the cluster, which will be slow
especially on a crowded headnode.
Let me see if I can at least check the signatures of all files in one run.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vatlab/sos/issues/1424#issuecomment-760619184, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABMGASBOS6PLJMY4P27MXBLSZ6X7PANCNFSM4VLYFLIQ
.

A feasible solution is to create a persistent ssh channel between local and remote host to process various requests, which is technically very challenging but perhaps I can find a third-party library for such things. I will keep you posted.

A feasible solution is to create a persistent ssh channel between local and remote host to process various requests, which is technically very challenging but perhaps I can find a third-party library for such things. I will keep you posted.

Just a note in saying it is possible to start a short-lived server on the remote server with something like the following
p = subprocess.Popen('ssh ictr-bpeng.ad.bcm.edu nohup sleep 15',
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
        shell=True    )

This should clear technical difficulty in fixing this bug.

Just a note in saying it is possible to start a short-lived server on the remote server with something like the following
p = subprocess.Popen('ssh ictr-bpeng.ad.bcm.edu nohup sleep 15',
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
        shell=True    )

This should clear technical difficulty in fixing this bug.

Another note: The server should not be working as a REQ/REP pair because whereas the server side should have only one process, there could be multiple clients. The existing proof-of-concept implementation works all right in simple cases but quickly develops deadlock etc with multiple clients.

Another note: The server should not be working as a REQ/REP pair because whereas the server side should have only one process, there could be multiple clients. The existing proof-of-concept implementation works all right in simple cases but quickly develops deadlock etc with multiple clients.


Add a pull request template

We should clarify what our expectations are of external developers contributing to our codebase.
A good way to do this is through a pull request template, which lets developers check off what they’ve done. A lot of the GOV.UK frontend repos do this, eg
https://github.com/alphagov/govuk_elements/blob/3c5c43a6fd4209cf2b6d180e7a5eadf6fa94c132/.github/PULL_REQUEST_TEMPLATE.md
This commit also documents how we do version numbering, so that an external developer would have a reasonable chance of being able to increment the version number in the right way.

This is indeed a very good idea! It would have helped me a lot previously, and will definitely help next time.
I would consider adding a few more things, to make it more complete:
1.) Regarding bumping the version: the update_version.sh script takes care of the README.md, and probably the pom.xml , but maybe not src/test/java/uk/gov/service/notify/NotificationClientTest.java (although running all unit tests would catch this immediately - see next point).
2.) To the checklist:

[ ] I've run all tests and they all pass.
[ ] When adding new dependencies I've considered keeping them to the minimum (within reason).
[ ] I've updated CHANGELOG.md

3.) Regarding the semantic versioning - is the -RELEASE suffix constant?


[Designer Accessibility] Render the entire card (even if a portion of it is hidden) on the initial load

Issue
The entire card needs to be rendered on initial load, including elements hidden by ShowCard.
Without the entire card rendered, the hidden elements are not present in the DOM tree, so we lose focus when they are inserted in the tree.

@carlos-zamora On our test site, I am no longer seeing this issue. I still see that the inner card isn't shown in the Card Structure pane, but focus does not appear to be lost. Are you able to repro the issue?

I'm still seeing this in the test site:

The hidden card isn't added to the tree until "Set Due Date" is invoked


Opentracing bridge

I believe this is ready for review again (and probably could do with me squashing work)

I'd remove the "shim" suffix everywhere.

Also, the suggested terminology here is "bridges"

@fbogsany so something like opentracingbridge/lib/opentelemetry/opentracingbridge/span.rb ?

Either of the following:
opentelemetry-ruby/opencensus/lib/opentelemetry/opencensus/span.rb
opentelemetry-ruby/opencensus/opentelemetry-opencensus.gemspec
opentelemetry-ruby/opentracing/lib/opentelemetry/opentracing/span.rb
opentelemetry-ruby/opentracing/lib/opentelemetry-opentracing.gemspec

or
opentelemetry-ruby/bridges/opentracing/lib/opentelemetry/bridge/opentracing/span.rb
opentelemetry-ruby/bridges/opentracing/opentelemetry-bridge-opentracing.gemspec
...

I kinda like the second. I'm a little 🤷♂ about pluralizing the top-level directory, but leaning toward the plural. So:
api
sdk
bridges
exporters
adapters

at the top-level.

The plural makes sense to me, on it

Can we remove the WIP and ask @indrekj for a 👀? He and @mwear have the most context on this.

Also note that @elskwid is working on a change to the in_span API that'll simplify the mapping considerably.

@rdooley I believe the in_span change to which @fbogsany was referring is in #118.

@mwear I believe this is ready for review now. I'm unsure why circleci jruby is failing on the jaeger exporter but will be looking into that today

I can just hold off until things are settled and update this PR then. Thank you for the update.

It might make more sense to address everything except for baggage in this PR. We can revisit baggage once context propagation has settled. What do think about that @rdooley?

Works for me!

:wave: @rdooley I definitely owe you a review, which I'll try to get in tomorrow. Thanks for your patience 🙏!

This is out of date and the spec has tightened up around OT compatibility for Otel. Closing this in favour of a new implementation.


Implement Sharing endpoints

Hello,
when you are planning to implement sharing links?

Is there any plan to release this before the API v1 end date (June 28)?

any news on sharing ?  only want to use api for sharing. ;(

Hey all, I'm finding it a bit hard to spare time to work on the sharing endpoints due to other commitments.
Currently the SDK doesn't have dedicated methods for interacting with the Sharing endpoints, similar to how you have dedicated methods to get metadata of files/folders or upload a file.
However, the SDK can still make requests to the sharing endpoints (or any Dropbox API endpoint) and fetch/decode data.
Here's how you can work with the Sharing endpoints (or any other Dropbox endpoint) through the SDK.
Also, I'm making this issue (project) a part of Hacktoberfest.
Any and all contributions, big or small, are highly encouraged and gladly welcome.
Happy Hacktoberfest! 🎉


Class links in "Orleans stream providers"#"Queue adapters" broken

Within the section "Queue adapters", both source code links are broken.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d8473367-03f8-92ec-9e52-58853ff99cf9
Version Independent ID: d8202876-68ea-e136-94e2-e1ad5f731fe4
Content: Orleans stream providers
Content Source: docs/orleans/streaming/stream-providers.md
Product: dotnet-orleans
GitHub Login: @IEvangelist
Microsoft Alias: dapine

Thanks @JesperTreetop. This has been fixed now.


Issue-add d3-time format, redux logger, lock version of moment

Automated Review Comments
Issues in examples/streamGraphExample/package.json:

(10, 'Hardcoded string found')
(11, 'Hardcoded string found')
(12, 'Hardcoded string found')
(16, 'Hardcoded string found')
(17, 'Hardcoded string found')
(20, 'Hardcoded string found')

I got it.


Willow: cannot enter password screen

Willow gives index error 0X40 upon attempting to enter password screen...tried several times.

I couldn't reproduce the exact error but the spreadsheet was missing a note about disabling "NMI mode, at vblank", the game ran fine for me after disabling that.

I couldn't reproduce the exact error but the spreadsheet was missing a note about disabling "NMI mode, at vblank", the game ran fine for me after disabling that.

that did it for the password screen, IDK if i'll get through the whole game, but i'll report back what i find, thanks a ton!!!