Strange debug message from analytics

Description
Seeing a strange message from console when running the app (built for release)
Reproducing the issue
No response
Firebase SDK Version
10.17
Xcode Version
15.0.1
Installation Method
Swift Package Manager
Firebase Product(s)
Analytics
Targeted Platforms
iOS
Relevant Log Output
The request of a upload task should not contain a body or a body stream, use `upload(for:fromFile:)`, `upload(for:from:)`, or supply the body stream through the `urlSession(_:needNewBodyStreamForTask:)` delegate method.

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

{
  "pins" : [
    {
      "identity" : "abseil-cpp-binary",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/abseil-cpp-binary.git",
      "state" : {
        "revision" : "bfc0b6f81adc06ce5121eb23f628473638d67c5c",
        "version" : "1.2022062300.0"
      }
    },
    {
      "identity" : "firebase-ios-sdk",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/firebase/firebase-ios-sdk",
      "state" : {
        "revision" : "8872dbd7d947acf757abab933da10e83c1842280",
        "version" : "10.17.0"
      }
    },
    {
      "identity" : "googleappmeasurement",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GoogleAppMeasurement.git",
      "state" : {
        "revision" : "6b332152355c372ace9966d8ee76ed191f97025e",
        "version" : "10.17.0"
      }
    },
    {
      "identity" : "googledatatransport",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GoogleDataTransport.git",
      "state" : {
        "revision" : "7874c1b48cbffd086ce8a052c4be873a78613775",
        "version" : "9.2.3"
      }
    },
    {
      "identity" : "googleutilities",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GoogleUtilities.git",
      "state" : {
        "revision" : "871d43135925cde39ef7421d8723ce47edfdcc39",
        "version" : "7.11.1"
      }
    },
    {
      "identity" : "grpc-binary",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/grpc-binary.git",
      "state" : {
        "revision" : "a673bc2937fbe886dd1f99c401b01b6d977a9c98",
        "version" : "1.49.1"
      }
    },
    {
      "identity" : "gtm-session-fetcher",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/gtm-session-fetcher.git",
      "state" : {
        "revision" : "d415594121c9e8a4f9d79cecee0965cf35e74dbd",
        "version" : "3.1.1"
      }
    },
    {
      "identity" : "interop-ios-for-google-sdks",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/interop-ios-for-google-sdks.git",
      "state" : {
        "revision" : "2d12673670417654f08f5f90fdd62926dc3a2648",
        "version" : "100.0.0"
      }
    },
    {
      "identity" : "leveldb",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/firebase/leveldb.git",
      "state" : {
        "revision" : "0706abcc6b0bd9cedfbb015ba840e4a780b5159b",
        "version" : "1.22.2"
      }
    },
    {
      "identity" : "nanopb",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/firebase/nanopb.git",
      "state" : {
        "revision" : "819d0a2173aff699fb8c364b6fb906f7cdb1a692",
        "version" : "2.30909.0"
      }
    },
    {
      "identity" : "promises",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/promises.git",
      "state" : {
        "revision" : "ec957ccddbcc710ccc64c9dcbd4c7006fcf8b73a",
        "version" : "2.2.0"
      }
    },
    {
      "identity" : "swift-protobuf",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/apple/swift-protobuf.git",
      "state" : {
        "revision" : "0af9125c4eae12a4973fb66574c53a54962a9e1e",
        "version" : "1.21.0"
      }
    }
  ],
  "version" : 2
}

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

Replace this line with the contents of your Podfile.lock!

Thanks, @visskiss. This seems similar to #11536, and may be caused by the version of GoogleUtilities specified in the attached Package.resolved:
    {
      "identity" : "googleutilities",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GoogleUtilities.git",
      "state" : {
        "revision" : "871d43135925cde39ef7421d8723ce47edfdcc39",
        "version" : "7.11.1" 💥 The fix to #11536 was released in  7.11.4
      }
    },

I believe to update this transitive dependency of Firebase,  File -> Packages -> Update to Latest Package Versions should work.
Could you please try updating that dependency and see if the issue persists?

That fixed it. Thank you.


maybe useful, maybe not: convert beginners-luck.cabal to hpack

This takes advantage of stack's builtin support for hpack, a new package description language embedded in YAML. You may like this, you may not; I won't mind if you didn't merge it in. Compared to the Cabal format, hpack has two cool features:

No more writing down all your modules into exposed-modules. hpack infers the modules by looking at your source code.

No more copying and pasting the list of dependencies into each target. hpack allows a top-level dependencies list that each target then can inherit from.

@2016rshah @matthandlersux

This seems cool, is there any additional setup instructions people getting started with the project would need? Or is it just something you run once or twice every so often and won't influence most contributors to the project?
If it changes the getting started process we should add instructions on doing so to the README. This could be as simple as a section that just lists the commands to install hpack, run the tool, etc.

This seems cool, is there any additional setup instructions people getting started with the project would need? Or is it just something you run once or twice every so often and won't influence most contributors to the project?

Nope! Works automatically with stack.

Awesome, thanks!


Is there a year zero?

When debugging why the year number was off by one in my code, I realized that chrono has a year zero.
In other words, this code prints 0000-01-03T00:00:00 and not 0001-01-03T00:00:00:
let base = NaiveDateTime::new(NaiveDate::from_ymd(-1, 1, 1), NaiveTime::from_hms(0, 0, 0));
let result = base + Duration::days(367);
println!("{:?}", result);

Since I'm not an expert in calendars, what is the policy about year zero? Is it a bug that there is one, or is it normal? And if it is normal, is there a way to automatically adjust?

It is intentional to have the year zero, which equals to 1 BCE. This is a behavior due to ISO 8601 and makes everything vastly simple and consistent. For the purpose of formatting you can use Datelike::year_ce which has no year zero, though.


Using class_weights in fit_generator causes continuous increase in CPU memory usage until depletion. OOM

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04.2 LTS
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Not Relevant.
TensorFlow installed from (source or binary):
TensorFlow version (use command below): tf-nightly-gpu           1.15.0.dev20190728
Python version: Python 3.7.4
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: release 10.0, V10.0.130
NVIDIA-SMI 418.43       Driver Version: 418.43       CUDA Version: 10.1
GPU model and memory:
GeForce RTX 2080Ti 11GB

Describe the current behavior
When using class_weights in fit_generator causes the training process to continuously consume more and more CPU RAM until depletion. There is a stepped increased in memory usage after each epoch. See below for the reproducible example.  To keep the reproducible example small, I decreased the size of the dataset and batch size, which shows the trend of increasing memory. While training with my actual data, it depletes the full 128GB RAM by 70 EPOCS.
In the code below, if you comment out the class weights, the program trains without depleting memory.
Describe the expected behavior
Not leak memory.
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
import tensorflow as tf
tf.enable_eager_execution()
import numpy as np

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import CuDNNLSTM, Dense
from tensorflow.keras.optimizers import Adadelta

feature_count = 25
batch_size = 16
look_back = 5
target_groups = 10

def random_data_generator( ):
    x_data_size =(batch_size, look_back, feature_count) # batches, lookback, features
    x_data = np.random.uniform(low=-1.0, high=5, size=x_data_size)

    y_data_size = (batch_size, target_groups)
    Y_data = np.random.randint(low=1, high=21, size=y_data_size)

    return x_data, Y_data

def get_simple_Dataset_generator():
    while True:
        yield random_data_generator()

def build_model():
    model = Sequential()
    model.add(CuDNNLSTM(feature_count,
                    batch_input_shape=(batch_size,look_back, feature_count),
                    stateful=False))
    model.add(Dense(target_groups, activation='softmax'))
    optimizer = Adadelta(learning_rate=1.0, epsilon=None)
    model.compile(loss='categorical_crossentropy', optimizer=optimizer)
    return model

def run_training():

    model = build_model()
    train_generator = get_simple_Dataset_generator()
    validation_generator = get_simple_Dataset_generator()
    class_weights = {0:2, 1:8, 2:1, 3:4, 4:8, 5:35, 6:30, 7:4, 8:5, 9:3}

    model.fit_generator(generator = train_generator,
            steps_per_epoch=1,
            epochs=1000,
            verbose=2,
            validation_data=validation_generator,
            validation_steps=20,
            max_queue_size = 10,
            workers = 0,
            use_multiprocessing = False,
            class_weight = class_weights
            )

if __name__ == '__main__':
    run_training()

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
Memory Usage When class_weights are used

Memory Without class_weights

Was able to reproduce the issue on Colab with Tf-nightly-gpu version. Please have a look at Colab link. Thanks!

Any workarounds or updates this? Thanks

I'm not able to reproduce this - can you update to the latest tf-nightly-gpu package and try again? Thanks!

@talipini This was resolved in the !pip install tf-nightly-gpu
Please check the gist here.
I am closing the issue here. Please feel free to open the issue if it persists again. Thanks!

Looks like new nightly build has fixed this issue. But training time is significantly longer (about 30%) in my case between 1.15.0.dev20190728 and 1.15.0.dev20190813. Same codebase takes 30% longer to train in the newer nightly build but it definitely seems to solve the memory issue though. I will try to create a reproducible example and open an new issue. Thanks for checking into this.

@talipini Sure. Thanks for your support.

I'm still having the same memory issue when using class_weights with Keras model.fit_generator. Running the stable version of TF 2.0.

@TKassis Please open a new issue with details about your issue, platform details and a standalone code to reproduce the issue. Thanks!

@TKassis Please open a new issue with details about your issue, platform details and a standalone code to reproduce the issue. Thanks!

Thanks, it seems multiple people have reported the issue already.

@TKassis do you have a link to the other issues? I am not fining them. Thank you!

@talipini Thanks for the issue!
Can you please try this again with the latest nightly (pip install -U tf-nightly) an report back? I believe this issue should be fixed in the latest nightly


Enable musttag linter

Description
The musttag linter has been disabled, because the lastest version of golangci-lint includes the 0.4.5 version, which has a major bug.
musttag already fixed that bug with version 0.5.0. But we need to wait until golangci-lint upgrades to the version 0.5.0 or higher. If that is done, the image of golangci-lint has to be upgraded in our pipelines. If that is done, we can re-enable it by removing the follwoing line: https://github.com/kyma-project/runtime-watcher/blob/main/skr/.golangci.yaml#L5
ACs

[ ] enable musttag in the runtime-watcher/skr

musttag is now referenced with v0.6.0, unblocking this.


Support mixing optgroups and options

Need a way to construct select boxes that mix optgroups and option elements.
<select>
    <option>1</option>
    <option>2</option>
    <optgroup label="example">
      <option>3</option>
    </optgroup>
</select>

Perhaps just support ng:options on option elements and optgroup elements.
<select>
  <option ng:options="..."></option>
  <optgroup ng:options="..." label="example"></optgroup>
  <option ng:options="..."></option>
</select>

the whole reason for ng:options being attached to the select element instead of options has to do with a circular dependency that otherwise exists between the select element and option elements. I've been there once and I don't think we want to go back :-)
We must figure out a better way to handle this.


Prim's minimum spanning tree algorithm

💥 Proposal
I have searched for this algo in cpp section. It's neither there nor any issue is created regarding this problem.
Have you read the Contributing Guidelines on Pull Requests?
Yes

/assign

Why it is closed?

@ankitaggarwal23

May I know the reason??


Added toHaveTagName

Quick shorthand for programatically checking tagname. Case-insensitive

why not just use .toEqual('div') ?


Issue with building from source

I'm not sure if this is considered a bug report or not so I did not label it as one but cloning the repository, and running install and run build as per the README.md, it states the following on console:
1 rules skipped due to selector errors:
  legend+* -> Cannot read properties of undefined (reading 'type')
- Generating index html...
1 rules skipped due to selector errors:
  legend+* -> Cannot read properties of undefined (reading 'type')
✔ Index html generation complete.

Alongside some warnings but I don't feel they're important:
Warning: A:\pigallery2\src\common\config\private\PrivateConfig.ts depends on 'typeconfig/src/decorators/class/SubConfigClass'. CommonJS or AMD dependencies can cause optimization bailouts.
For more info see: https://angular.io/guide/build#configuring-commonjs-dependencies

Is it something I should be worried of or can it be ignored?

I'm closing this due to inactivity.
let me know if you run into any issue with this.


Feat encryption key exchange on g meter

https://smartsocietyservices.atlassian.net/browse/SLIM-256

All is well.

All is well.

All is well.

All is well.

All is well.

All is well.

SonarQube analysis reported 1 issue:

 1 critical

Watch the comments in this conversation to review them.
Note: the following issues could not be reported as comments because they are located on lines that are not displayed in this pull request:

 Make "dataObject" transient or serializable.

All is well.

All is well.

All is well.


Rule 4.6.5 needs some fixes in order to be CIS compliant

Describe the Issue
The rule gets a "FAIL" result from CIS.
Expected Behavior
The task from this rule needs to make sure that the following files: /etc/login.defs, /etc/profile and /etc/bashrc contain UMASK 027 (for /etc/login.defs ) or umask 027 (for /etc/profile and /etc/bashrc).
Actual Behavior
Firstly, the regexp is not quite good and it does not edit the /etc/login.defs accordingly, adding a new line with UMASK 027 instead of editing the existing one.
Secondly, the way the task is written, it does not edit the /etc/bashrc file in a proper manner, only modifying one of the umask lines, not all of them.
Control(s) Affected
4.6.5 Ensure default user umask is 027 or more restrictive
Environment (please complete the following information):

branch being used: [e.g. devel]

Additional Notes
Anything additional goes here
Possible Solution
First part of the solution would be to modify the regexp so that it will match the UMASK line in the /etc/login.defs file.
The second part would be to remove the /etc/bashrc file from being edited with the ansible.builtin.lineinfile module, and create another task that edits it with ansible.builtin.replace module (this would be the only manner in which we can ensure that all of the matching lines from this file will get edited properly).
By applying these modifications the files are edited according to CIS regulations and the rule passes!
This solution will be provided in a PR.

hi @DianaMariaDDM
I noticed that you have closed the associated PR for this issue, i have however included a fix in to the Feb24 updates that have been applied.
I'd like to get a new release out for this repo asap, Can you confirm that this has been addressed?
Many thanks
uk-bolly


olm  fails with wasm validation error: at offset 4: failed to match magic number

Olm JS Bug Report
Getting
wasm validation error: at offset 4: failed to match magic number

when running olm js package .
Describe the bug
Web page displays wasm validation error: at offset 4: failed to match magic number, instead of running the actual web assembly.
To Reproduce

npm install olm js package from https://packages.matrix.org/npm/olm/
Initialize the olm and create an instance of Account.
Call the generate_one_time_keys() method on the account insance.
npm run dev
Observe error

Expecated Behaviour
I expected to see the result of Olm to generate a one time key.
Screenshots

System Information

MacOS Big Sur
Firefox 86/Safari 14/Chrome 88
React version 17
Olm package version 3.2.1

Additional context
I followed the example given in the README.MD for the olm package.

It is probably loading the wrong file.  Can you check that it is trying to load the .wasm file from the right location?


Update README.md

Do it. please... i promise it wont scare the onlookers
-Smooth brain shrek. <3

this is a tough decision :'(
should it be readable so ppl can actually install it
vs
uwu

mate, dw, if they want uwu then they must understand uwu

ehhh i still stand by my original decision to keep it unuwu'd for now
thanks for raising this issue though


User should be directly able to select image for avatar from the gallery

Describe the feature you'd like
The user should be directly able to select an image from the gallery for his avatar rather than uploading an URL for it.
Would you like to work on the issue?
YES

Duplicate of #1182

@iamareebjamal can i work on #1182  instead

Yes


Error in specifing solver in capacity expansion and dispatch tutorials

For both avaiable tutorials, the specification of CPLEX as a usable solver is incorrect. Furthermore, no option of MacOS is available. Update on the tutorial page is required to solve these issues.

Closed by #64


Allow Distributions ability to incorporate vetoes

Allow the Distribution class the ability to take a list of veto segments and disregard background events that lie in these segments. I'm not sure what the best way to implement this is. For example, the Cluster distribution class assumes contiguous times, so removing times would, at the moment, be problematic. One basic approach would be to set the network outputs for all vetoed times to -inf in the Distributions .fit method.

Would it be sufficient to handle this in the parent class's .fit method by breaking a Segment up if necessary into multiple sub-segments, then running .update on each one of these new, shorter segments?


Execute lambda on mutation?

I'm successfully connecting and using lambdas connected to the specific fields in the graphql type schema. When I send a query containing that field, my function gets executed.
Is there a way to specify a lambda which will be executed on mutation? ( I'm thinking of using it for capturing and logging some specific mutations).

On second thought... I guess I could simply request a field which is connected to the lambda as part of the mutation, and probably the context will have sufficient information to figure out that was executed as part of a mutation?

Just a suggestion, but you could use DynamoDB streams and create a Lambda trigger once a field mutates as mentioned here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.html
Another option to look into would be Pipeline Resolvers: https://docs.aws.amazon.com/appsync/latest/devguide/pipeline-resolvers.html

I wanted to look into the possibility to attach lambda to dynamodb streams anyway for different goal, however there are two issues of a) custom setup / cloudformation templates which would somehow need to be connected into amplify environment deployment , so that it creates/connects those streams for every new environment and specifically to this goal - b) the lambda will only have access to the data which is stored in the DB, and not the runtime data available to any lambda which might be executed through the appsync resolvers.
I saw on a different ticket now, that one can define custom mutation, and connect it to lambda - I'm guessing that solution might be using pipeline resolvers - I will check it out.

You can attach Lambda functions to @model created types in Amplify now which should execute on your mutation: https://aws-amplify.github.io/docs/cli-toolchain/quickstart#as-a-part-of-the-graphql-api-types-with-model-annotation

@undefobj , thanks for pointing that out! I missed it!
Any chance I also missed that you can somehow specify a lambda which could be used for "transformation/augmenting" the graphql mutation data before it gets received by the default generated resolver (and sent to DynamoDB)?

You could potentially fill out items in the attributes map by reverse engineering and editing the generated resolvers in VTL. If you look at one of the examples here: https://docs.aws.amazon.com/appsync/latest/devguide/resolver-util-reference.html#dynamodb-helpers-in-util-dynamodb
    #set( $myFoo = $util.dynamodb.toMapValues($ctx.args) )
    #set( $myFoo.version = $util.dynamodb.toNumber(1) )
    #set( $myFoo.timestamp = $util.dynamodb.toString($util.time.nowISO8601()))

    "attributeValues" : $util.toJson($myFoo)

The $myFoo.timestamp is being generated on the server in the same way that you want updatedAt.

@undefobj , thanks for the pointer!


Debugging for #4479

Cannot reproduce #4479 locally, so hoping that we see more output if this PR hits the problem in GHA.

Our observations so far: the kafka-configs process is exiting with exit code 137, which means that it is receiving SIGKILL. @bsideup tried upgrading the image to one that has a cgroup-aware JDK version, in case a GHA change is causing containers to be OOMKilled. It doesn't look like that's helped.
I'm still considering the hypothesis that startup is just taking too long, since it involves pulling the image and then starting N instances of Kafka in parallel (25s+ just to start the container looks really long). If deepStart is timing out, perhaps it's killing the containers (and by coincidence, killing kafka-configs).


Link to blog post is broken

The blog link contained in the README file is broken.

:hamster: yep


[Android] Alarm sound is dependent on the "Media volume" but there is a seperate volume called "Alarm volume"

Alarm sound is dependent on the "Media volume" but there is a seperate volume called "Alarm volume", sometimes there is a need to play alarm with sound even when the media volume is down, like native alarm clock.
Checkout: https://github.com/ryanheise/audio_session.
Please make it an option and it will almost like native alarm clock!
It's not hard. Just add this:
final session = await AudioSession.instance;
    await session.configure(const AudioSessionConfiguration(
      androidAudioAttributes: AndroidAudioAttributes(
        usage: AndroidAudioUsage.alarm,
      ),
    ));

Good point. I could add a maxVolume optional parameter. Thanks !

Hey @dtkdt100
I was working on this feature and I realized that due to privacy reasons, there is no official way to change device alarm volume in iOS. I really want the alarm plugin to have the exact same behavior for Android and iOS so I will probably not add this feature.
I am adding this to the README FAQ:

Why isn't my alarm ringing when the device volume is off?
iOS prevents third-party apps from modifying volumes. If media volume is off, then the alarm is muted. To ensure the alarm rings, users should turn on media volume before setting the alarm. For consistency, we don't add volume checks for Android, maintaining a uniform user experience on both platforms. Feel free to add one in your app is desired.

Let me know if you have a comment or any other question.

@gdelataillade I think in order to achive this behavoir, alarm apps in iOS are turning up the volume and then turring it down again. (https://pub.dev/packages/perfect_volume_control)
Furthermore, you could use these parmas to stop and replay any other music that the iphone user:
avAudioSessionCategory: AVAudioSessionCategory.playback,
avAudioSessionCategoryOptions: AVAudioSessionCategoryOptions.none,

in the audio_session package^.
Maybe you could to the same behavoir with that?

I think the perfect_volume_control changes the volume relatively to the device volume set by the user. I mean if the device volume is 70% and I set it to 1.0, then my audio will player with volume 70%. If I set it to 0.5, then it will play with 35%.
Good point for the audio_session feature thanks.

hmm, intersting. Is there any other volume package that will change the app volume not relativly? I mean there has to be something right? how does the native alarm is working in iOS?
https://pub.dev/packages/volume_control
https://pub.dev/packages/lecle_volume_flutter

I just tested the app Alarmy and what they do is that they set the media volume to the maximum at the moment the alarm starts ringing, and then they put it back like it was before.
So yes there's a way to do it !

@dtkdt100  This feature will require some research and refactoring. Honestly I don't have a lot of time at the moment, so feel free to contribute if you want. Otherwise, I'll do it but I can't tell you right now when it will be done.

This feature was released in the latest version !

@gdelataillade Thank you very much!


[GATTACA] Cannot use the local databases if run in the docker container.

The docker container has to mount folders in order to access files in the host system. The local database cannot therefore be accessed if in an arbitrary (unmounted) folder.
Right now, the script mounts the output and input folders. Possible solutions:

Enforce that the local database to be placed in the input folder, changing the yaml options to just take the filename (instead of the path) of the file.
Read the path from the yaml options at runtime and mount the folder dynamically. This would require that GATTACA.R be informed about this to load the file accordingly (the mounted folder in the container is different than the path outside the container, so we cannot just reuse the same path from the yaml).
Do not allow using a local database.

I suggest the compromise of option 1.

Local databases have been removed since several versions ago.


Add Custom AutoComplete For Plugins / Libraries Just Like VsCode

I want to be able to edit Android Studio Files , But AutoComplete For android studio Library not available

Like Linting AutoComplete, IntelliSense Plugins for Frameworks like Vue, Ionic, Android Java / Kotlin and mor


Allowed memory size exhausted

If you add to every step 'skip' => true, you will run into infinite loop and get "Allowed memory size of X bytes  exhausted" error.

Interesting. What do you suggest should happen in that case? I mean, it could either be a misconfigured flow or happen in a real use case after evaluating (rather complex and maybe weird) skip conditions.


README/CompanyGitHub: Network UPS tools

Is your feature request related to a problem? Please describe.
Additional resource
Describe the solution you'd like
Network UPS Tools project|
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context

README

References

Company GitHub

| Network UPS Tools project |

#1840 contains #1839


Remove symm block constraint added by the symmnet constraint

Comment the symm block constraint generation in the symm net constraint routine.

@soneryaldiz @kkunal1408 This PR is ready for review. I have removed the symmetric block constraint generation within the placer when a symmetric net constraint is specified.


App interface crash issue in Chinese language

Test device：Windows 11 / Android 12 (OneUI)
The file transfer confirmation screen crashes when using the Simplified Chinese interface, but there is no problem with English
   (English interface is normal)
   (Android app crash screen, window is the same as this)

I hit this issue too.  There was an i18n structure change in 1.6.1, LocalSend probably isn't catching exceptions when fetching values from language files, hehe 😁.

I the progress page I used the i18n plural feature but forgot to register it for Chinese. I guess another hotfix release is needed

+1

Fixed in v1.6.2.
On Google Play and App Store it is already available. On Windows Store and MacOS, it will take around 12 hours.


feat: invalidateQueryAndRefetchIfListeners()

First of all great work!
Regarding my issue, I searched ages and dived deep into the framework code on how to be able to invalidate a query and only refetch it, if it actually is used somewhere. Turns out its not possible out of the box so I wrote myself an extension:
extension ExtensionOnQueryBase<T, State extends QueryState<dynamic>> on QueryBase<T, State> {

  void invalidateQueryAndRefetchIfListeners()
  {
    invalidateQuery();

    if (hasListener) refetch();
  }
}

I know the name is not the best, but explaining people that this is what you want regarding not asking the server to much and still having a nice user experience and it is not shipped within the docs or the framework might be an issue for many people beeing guided in the wrong driection IMO

Not sure about adding in another feature for this as this can be done either as you have done for a single query or with a query search eg:
// reftech all querys with listeners
CachedQuery.instance.refetchCurrentQueries();

// invalidate all queries
CachedQuery.instance.invalidateCache();

// or using a where search for more control
final queries = CachedQuery.instance.whereQuery((query) => !query.hasListener);

However, the docs should be improved if you were unable to find this information. Or even adding your suggestion to the docs.
Are there any part of the docs that you feel are miss leading?
Have I missed anything or miss understood your request?

No, you are completely right with what you just said.
Maybe pointing it out in the docs would be great for new developers. I ran into this case often, like have a list of Items then going to the detail page of 1 Item updating the Item and invalidate the list of Items and refetch it if a screen is open displaying the list of Items (list query having listeners).
Of course you could have updated the list query manually with the newly recieved item from the update call. But this adds another layer of complexity to the developer and often times its much easier to just invalidate those queries and refetch them.
And because I only saw invalidate() and refetch() I thought it is possible to do it with one of those functions.
To be honest I can only imagine a few special cases where you wanna refetch a query that is not beeing used. I was thinking about something like refetch({bool onlyIfUsed = false}) that would not break anyones code but still has the direct option. The problem is that you would have to call invalidate() before that regardless of the refetch because it could stop the refetch().
So I Guess you are right putting it somewhere in the docs might be a good place to start.

Just to explain my example more, often times I had api calls I knew would change my infinite lists but had no return object to optimistic update that list. Either I am coding wrong or this happens a lot of times :D


singularity container run_bulkModule.sh script fails

This looks good:
Singularity> ./validate.py ../config/bulk-config.json --ignore-input-output --software
Singularity> ./validate.py ../config/sc-config.json --ignore-input-output --software

But the 'run_bulkModule.sh' pipeline fails on Getting broad peak summits step:
Singularity> /opt/CUT-RUNTools-2.0/run_bulkModule.sh /opt/CUT-RUNTools-2.0/config/bulk-config.json GATA1_D7_30min_chr11
==================================== Bulk data analysis pipeline will run ==============================================================
## Input FASTQ folder:            /opt/CUT-RUNTools-2.0/exampleData
## Sample name:                   GATA1_D7_30min_chr11
## Workdir folder:                /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11
## Experiment name:               GATA1_D7_30min_chr11
## Experiment type:               CUT&RUN
## Reference genome:              hg19
## Spike-in genome:               FALSE
## Spike-in normalization:        FALSE
## Fragment 120 filtration:       TRUE
=================================================================================================================================
[info] Input file is GATA1_D7_30min_chr11_R1_001.fastq.gz and GATA1_D7_30min_chr11_R2_001.fastq.gz
Wed Jul 13 12:25:45 EDT 2022
[info] Trimming file GATA1_D7_30min_chr11 ...
Wed Jul 13 12:25:45 EDT 2022
[info] Use Truseq adaptor
[info] Second stage trimming GATA1_D7_30min_chr11 ...
Wed Jul 13 12:25:53 EDT 2022
[info] Aligning file GATA1_D7_30min_chr11 to reference genome...
Wed Jul 13 12:26:03 EDT 2022
[info] Bowtie2 command: --dovetail --phred33
[info] The dovetail mode is enabled [as parameter frag_120 is on]
[info] FASTQ files won't be aligned to the spike-in genome
[info] Filtering unmapped fragments... GATA1_D7_30min_chr11.bam
Wed Jul 13 12:26:28 EDT 2022
[info] Sorting BAM... GATA1_D7_30min_chr11.bam
Wed Jul 13 12:26:32 EDT 2022
INFO	2022-07-13 16:26:33	SortSam

********** NOTE: Picard's command line syntax is changing.
**********
********** For more information, please see:
********** https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)
**********
********** The command line looks like this in the new syntax:
**********
**********    SortSam -INPUT sorted/GATA1_D7_30min_chr11.step1.bam -OUTPUT sorted/GATA1_D7_30min_chr11.bam -SORT_ORDER coordinate -VALIDATION_STRINGENCY SILENT
**********

16:26:33.573 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/opt/CUT-RUNTools-2.0/install/picard-2.8.0.jar!/com/intel/gkl/native/libgkl_compression.so
[Wed Jul 13 16:26:33 UTC 2022] SortSam INPUT=sorted/GATA1_D7_30min_chr11.step1.bam OUTPUT=sorted/GATA1_D7_30min_chr11.bam SORT_ORDER=coordinate VALIDATION_STRINGENCY=SILENT    VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
[Wed Jul 13 16:26:33 UTC 2022] Executing as ponomarevsy@ai-hpcn106 on Linux 3.10.0-1160.11.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 11.0.15-internal+0-adhoc..src; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.21.7-SNAPSHOT
INFO	2022-07-13 16:26:33	SortSam	Seen many non-increasing record positions. Printing Read-names as well.
INFO	2022-07-13 16:26:38	SortSam	Finished reading inputs, merging and writing to output now.
[Wed Jul 13 16:26:44 UTC 2022] picard.sam.SortSam done. Elapsed time: 0.18 minutes.
Runtime.totalMemory()=1166016512
[info] Marking duplicates... GATA1_D7_30min_chr11.bam
Wed Jul 13 12:26:44 EDT 2022
/opt/CUT-RUNTools-2.0/src/bulk/bulk-pipeline.sh: line 124: 12845 Killed                  $javabin/java -jar $picardbin/$picardjarfile MarkDuplicates INPUT=sorted/"$base".bam OUTPUT=dup.marked/"$base".bam VALIDATION_STRINGENCY=SILENT METRICS_FILE=metrics."$base".txt 2> $logdir/"$base".mark.dup
[info] Removing duplicates... GATA1_D7_30min_chr11.bam
Wed Jul 13 12:26:51 EDT 2022
[info] Filtering to <120bp... dup.marked and dedup BAMs
Wed Jul 13 12:26:51 EDT 2022
[info] Creating bam index files... GATA1_D7_30min_chr11.bam
Wed Jul 13 12:26:51 EDT 2022
[info] Reads shifting
Wed Jul 13 12:26:51 EDT 2022
[info] Your data won't be shifted as the experiment_type is specified as CUT&RUN...
[info] Peak calling using MACS2... GATA1_D7_30min_chr11.bam
[info] Logs are stored in /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs
Wed Jul 13 12:26:51 EDT 2022
[info] Peak calling with BAM file with NO duplications
[info] macs2 narrow peak calling
[info] macs2 broad peak calling
[info] Getting broad peak summits
Traceback (most recent call last):
  File "/opt/CUT-RUNTools-2.0/install/get_summits_broadPeak.py", line 14, in <module>
    f = open(sys.argv[1])
FileNotFoundError: [Errno 2] No such file or directory: '/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/macs2.broad/GATA1_D7_30min_chr11_peaks.broadPeak'
[info] SEACR stringent peak calling
Traceback (most recent call last):
  File "/opt/CUT-RUNTools-2.0/install/change.bdg.py", line 12, in <module>
    f = open(sys.argv[1])
FileNotFoundError: [Errno 2] No such file or directory: '/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/seacr/GATA1_D7_30min_chr11_treat_pileup.bdg'
chmod: changing permissions of '/opt/CUT-RUNTools-2.0/install/SEACR_1.1.sh': Read-only file system
Calling enriched regions without control file
Proceeding without normalization of control to experimental bedgraph
Using stringent threshold
Creating experimental AUC file: Wed Jul 13 12:26:52 EDT 2022
Calculating optimal AUC threshold: Wed Jul 13 12:26:52 EDT 2022
Using user-provided threshold: Wed Jul 13 12:26:52 EDT 2022
Error in read.table(argsL$exp) : no lines available in input
Execution halted
Unable to access /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/seacr/GATA1_D7_30min_chr11_treat.stringent.bed
Traceback (most recent call last):
  File "/opt/CUT-RUNTools-2.0/install/get_summits_seacr.py", line 14, in <module>
    f = open(sys.argv[1])
FileNotFoundError: [Errno 2] No such file or directory: '/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/seacr/GATA1_D7_30min_chr11_treat.stringent.bed'
[info] Generating the normalized signal file with BigWig format...
Wed Jul 13 12:26:52 EDT 2022
cp: cannot stat '/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/macs2.narrow/GATA1_D7_30min_chr11.cpm.norm.bw': No such file or directory
cp: cannot stat '/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/macs2.narrow/GATA1_D7_30min_chr11.cpm.norm.bw': No such file or directory
[info] Your bigwig file won't be normalized with spike-in reads
[info] Input file is /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/macs2.narrow/GATA1_D7_30min_chr11_peaks.narrowPeak
cat: GATA1_D7_30min_chr11_peaks.narrowPeak: No such file or directory
cat: GATA1_D7_30min_chr11_summits.bed: No such file or directory
[info] Get randomized [1000] peaks from the top [2000] peaks...
[info] Filtering the blacklist regions for the selected peak files
[info] Getting Fasta sequences
Warning: the index file is older than the FASTA file.
[info] Start MEME analysis for de novo motif finding ...
[info] Up to 10 will be output ...
Log::Log4perl configuration looks suspicious: No loggers defined at /opt/conda/envs/meme/lib/site_perl/5.26.2/Log/Log4perl/Config.pm line 325.
Starting getsize: getsize random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa 1> $metrics
getsize ran successfully in 0.032232 seconds
Starting fasta-most: fasta-most -min 50 < random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa 1> $metrics
fasta-most ran successfully in 0.069094 seconds
Starting fasta-center: fasta-center -dna -len 100 < random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa 1> random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered
fasta-center ran successfully in 0.076943 seconds
Starting fasta-shuffle-letters: fasta-shuffle-letters random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-shuffled -kmer 2 -tag -dinuc -dna -seed 1
fasta-shuffle-letters ran successfully in 0.013534 seconds
Starting fasta-get-markov: fasta-get-markov -nostatus -nosummary -dna -m 1 random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa random1000/MEME_GATA1_D7_30min_chr11_shuf/background
fasta-get-markov ran successfully in 0.013644 seconds
Starting meme: meme random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered -oc random1000/MEME_GATA1_D7_30min_chr11_shuf/meme_out -mod zoops -nmotifs 10 -minw 6 -maxw 30 -bfile random1000/MEME_GATA1_D7_30min_chr11_shuf/background -dna -revcomp -nostatus
No sequences found in file `random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered'.  Check file format.
meme exited with error code 1
Starting dreme-py3: dreme-py3 -verbosity 1 -oc random1000/MEME_GATA1_D7_30min_chr11_shuf/dreme_out -png -dna -p random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered -n random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-shuffled -m 10
Traceback (most recent call last):
  File "/opt/conda/envs/meme/bin/dreme-py3", line 1912, in <module>
    if __name__ == '__main__': main()
  File "/opt/conda/envs/meme/bin/dreme-py3", line 1801, in main
    pos_seqs = sequence.convert_ambigs(sequence.readFASTA(pos_seq_file_name, None, True), alph)
  File "/opt/conda/envs/meme/lib/meme-5.0.2/python/sequence_py3.py", line 154, in readFASTA
    raise RuntimeError("No sequences on FASTA format found in this file")
RuntimeError: No sequences on FASTA format found in this file
dreme-py3 exited with error code 1
Starting meme-chip_html_to_tsv: meme-chip_html_to_tsv random1000/MEME_GATA1_D7_30min_chr11_shuf/meme-chip.html random1000/MEME_GATA1_D7_30min_chr11_shuf/summary.tsv "meme-chip -oc random1000/MEME_GATA1_D7_30min_chr11_shuf -dreme-m 10 -meme-nmotifs 10 random1000/padded.fa/GATA1_D7_30min_chr11_summits_padded.fa" 5.0.2 "Thu Aug 30 14\:26\:08 2018 -0700"
meme-chip_html_to_tsv ran successfully in 0.141266 seconds
[info] De Novo motifs can be found: random1000/MEME_GATA1_D7_30min_chr11_shuf ...
[info] Loading the De Novo motifs ...
Traceback (most recent call last):
  File "/opt/CUT-RUNTools-2.0/install/read.meme.py", line 94, in <module>
    dreme_matrices = read_dreme(this_dir + "/dreme_out/dreme.txt")
  File "/opt/CUT-RUNTools-2.0/install/read.meme.py", line 47, in read_dreme
    f = open(n)
FileNotFoundError: [Errno 2] No such file or directory: 'random1000/MEME_GATA1_D7_30min_chr11_shuf/dreme_out/dreme.txt'
[info] The signficance cutoff of Fimo scaning is 0.0005...
[info] Motif files can be found: random1000/MEME_GATA1_D7_30min_chr11_shuf/motifs
[info] Filtering the blacklist regions for the selected peak files
[info] Getting Fasta sequences
Warning: the index file is older than the FASTA file.
[info] Scaning the De Novo motifs for each peak
ls: cannot access 'random1000/MEME_GATA1_D7_30min_chr11_shuf/motifs': No such file or directory
[info] Output can be found: fimo.result/GATA1_D7_30min_chr11
#
#        Congrats! The bulk data analysis is complete!
#
--------------------------------------------------------------------------

Looking inside log files I see:
Singularity> more /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/*
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11.bowtie2
::::::::::::::
586516 reads; of these:
  586516 (100.00%) were paired; of these:
    2416 (0.41%) aligned concordantly 0 times
    374417 (63.84%) aligned concordantly exactly 1 time
    209683 (35.75%) aligned concordantly >1 times
    ----
    2416 pairs aligned concordantly 0 times; of these:
      122 (5.05%) aligned discordantly 1 time
    ----
    2294 pairs aligned 0 times concordantly or discordantly; of these:
      4588 mates make up the pairs; of these:
        1256 (27.38%) aligned 0 times
        1491 (32.50%) aligned exactly 1 time
        1841 (40.13%) aligned >1 times
99.89% overall alignment rate
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11.gene.bw
::::::::::::::
WARNING! '/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam' does not have any
 mapped reads. Please check that the file is properly indexed and that it contains mapped reads.
WARNING! '/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam' does not have any
 mapped reads. Please check that the file is properly indexed and that it contains mapped reads.
normalization: CPM
Traceback (most recent call last):
  File "/opt/conda/envs/app/bin/bamCoverage", line 12, in <module>
    main(args)
  File "/opt/conda/envs/app/lib/python3.6/site-packages/deeptools/bamCoverage.py", line 162, in main
    scale_factor = get_scale_factor(args, stats)
  File "/opt/conda/envs/app/lib/python3.6/site-packages/deeptools/getScaleFactor.py", line 279, in get_scale_factor
    scale_factor *= 1.0 / (million_reads_mapped)
ZeroDivisionError: float division by zero
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11.macs2.broad
::::::::::::::
INFO  @ Wed, 13 Jul 2022 12:26:51:
# Command line: callpeak -t /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam
-g hs -f BAMPE -n GATA1_D7_30min_chr11 --outdir /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/macs2.broad -
-broad --broad-cutoff 0.1 -B --SPMR --keep-dup all
# ARGUMENTS LIST:
# name = GATA1_D7_30min_chr11
# format = BAMPE
# ChIP-seq file = ['/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam']
# control file = None
# effective genome size = 2.70e+09
# band width = 300
# model fold = [5, 50]
# qvalue cutoff for narrow/strong regions = 5.00e-02
# qvalue cutoff for broad/weak regions = 1.00e-01
# The maximum gap between significant sites is assigned as the read length/tag size.
# The minimum length of peaks is assigned as the predicted fragment length "d".
# Larger dataset will be scaled towards smaller dataset.
# Range for calculating regional lambda is: 10000 bps
# Broad region calling is on
# Paired-End mode is on
# MACS will save fragment pileup signal per million reads

INFO  @ Wed, 13 Jul 2022 12:26:51: #1 read fragment files...
INFO  @ Wed, 13 Jul 2022 12:26:51: #1 read treatment fragments...
INFO  @ Wed, 13 Jul 2022 12:26:51: 0 fragments have been read.
Traceback (most recent call last):
  File "/opt/conda/envs/app/bin/macs2", line 653, in <module>
    main()
  File "/opt/conda/envs/app/bin/macs2", line 51, in main
    run( args )
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 64, in run
    if options.PE_MODE: (treat, control) = load_frag_files_options (options)
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 350, in load_frag_files_options
    treat = tp.build_petrack()
  File "MACS2/IO/Parser.pyx", line 1267, in MACS2.IO.Parser.BAMPEParser.build_petrack
  File "MACS2/IO/Parser.pyx", line 1311, in MACS2.IO.Parser.BAMPEParser.build_petrack
ZeroDivisionError: float division
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11.macs2.narrow
::::::::::::::
INFO  @ Wed, 13 Jul 2022 12:26:51:
# Command line: callpeak -t /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam
-g hs -f BAMPE -n GATA1_D7_30min_chr11 --outdir /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/macs2.narrow
-q 0.01 -B --SPMR --keep-dup all
# ARGUMENTS LIST:
# name = GATA1_D7_30min_chr11
# format = BAMPE
# ChIP-seq file = ['/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam']
# control file = None
# effective genome size = 2.70e+09
# band width = 300
# model fold = [5, 50]
# qvalue cutoff = 1.00e-02
# The maximum gap between significant sites is assigned as the read length/tag size.
# The minimum length of peaks is assigned as the predicted fragment length "d".
# Larger dataset will be scaled towards smaller dataset.
# Range for calculating regional lambda is: 10000 bps
# Broad region calling is off
# Paired-End mode is on
# MACS will save fragment pileup signal per million reads

INFO  @ Wed, 13 Jul 2022 12:26:51: #1 read fragment files...
INFO  @ Wed, 13 Jul 2022 12:26:51: #1 read treatment fragments...
INFO  @ Wed, 13 Jul 2022 12:26:51: 0 fragments have been read.
Traceback (most recent call last):
  File "/opt/conda/envs/app/bin/macs2", line 653, in <module>
    main()
  File "/opt/conda/envs/app/bin/macs2", line 51, in main
    run( args )
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 64, in run
    if options.PE_MODE: (treat, control) = load_frag_files_options (options)
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 350, in load_frag_files_options
    treat = tp.build_petrack()
  File "MACS2/IO/Parser.pyx", line 1267, in MACS2.IO.Parser.BAMPEParser.build_petrack
  File "MACS2/IO/Parser.pyx", line 1311, in MACS2.IO.Parser.BAMPEParser.build_petrack
ZeroDivisionError: float division
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11.mark.dup
::::::::::::::
INFO	2022-07-13 16:26:44	MarkDuplicates

********** NOTE: Picard's command line syntax is changing.
**********
********** For more information, please see:
********** https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)
**********
********** The command line looks like this in the new syntax:
**********
**********    MarkDuplicates -INPUT sorted/GATA1_D7_30min_chr11.bam -OUTPUT dup.marked/GATA1_D7_30min_chr11.bam -VALIDATION_STRI
NGENCY SILENT -METRICS_FILE metrics.GATA1_D7_30min_chr11.txt
**********

16:26:45.377 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/opt/CUT-RUNTools-2.0/install/picard-2.8.0.
jar!/com/intel/gkl/native/libgkl_compression.so
[Wed Jul 13 16:26:45 UTC 2022] MarkDuplicates INPUT=[sorted/GATA1_D7_30min_chr11.bam] OUTPUT=dup.marked/GATA1_D7_30min_chr11.bam
 METRICS_FILE=metrics.GATA1_D7_30min_chr11.txt VALIDATION_STRINGENCY=SILENT    MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 MAX_FI
LE_HANDLES_FOR_READ_ENDS_MAP=8000 SORTING_COLLECTION_SIZE_RATIO=0.25 TAG_DUPLICATE_SET_MEMBERS=false REMOVE_SEQUENCING_DUPLICATE
S=false TAGGING_POLICY=DontTag CLEAR_DT=true DUPLEX_UMI=false ADD_PG_TAG_TO_READS=true REMOVE_DUPLICATES=false ASSUME_SORTED=fal
se DUPLICATE_SCORING_STRATEGY=SUM_OF_BASE_QUALITIES PROGRAM_RECORD_ID=MarkDuplicates PROGRAM_GROUP_NAME=MarkDuplicates READ_NAME
_REGEX=<optimized capture of last three ':' separated fields as numeric values> OPTICAL_DUPLICATE_PIXEL_DISTANCE=100 MAX_OPTICAL
_DUPLICATE_SET_SIZE=300000 VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD
5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
[Wed Jul 13 16:26:45 UTC 2022] Executing as ponomarevsy@ai-hpcn106 on Linux 3.10.0-1160.11.1.el7.x86_64 amd64; OpenJDK 64-Bit Se
rver VM 11.0.15-internal+0-adhoc..src; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.21.7-S
NAPSHOT
INFO	2022-07-13 16:26:45	MarkDuplicates	Start of doWork freeMemory: 36218032; totalMemory: 41943040; maxMemory: 32178700
288
INFO	2022-07-13 16:26:45	MarkDuplicates	Reading input file and constructing read end information.
INFO	2022-07-13 16:26:45	MarkDuplicates	Will retain up to 116589493 data points before spilling to disk.
INFO	2022-07-13 16:26:48	MarkDuplicates	Read     1,000,000 records.  Elapsed time: 00:00:02s.  Time for last 1,000,000:
   2s.  Last read position: chr11:108,223,704
INFO	2022-07-13 16:26:48	MarkDuplicates	Tracking 16 as yet unmatched pairs. 16 records in RAM.
INFO	2022-07-13 16:26:48	MarkDuplicates	Read 1168200 records. 0 pairs never matched.
INFO	2022-07-13 16:26:49	MarkDuplicates	After buildSortedReadEndLists freeMemory:<PHONE_NUMBER>; totalMemory:<PHONE_NUMBER>; m
axMemory:<PHONE_NUMBER>8
INFO	2022-07-13 16:26:49	MarkDuplicates	Will retain up to<PHONE_NUMBER> duplicate indices before spilling to disk.
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11.seacr
::::::::::::::
INFO  @ Wed, 13 Jul 2022 12:26:52:
# Command line: callpeak -t /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam
-g hs -f BAMPE -n GATA1_D7_30min_chr11 --outdir /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/seacr -q 0.01
 -B --SPMR --keep-dup all
# ARGUMENTS LIST:
# name = GATA1_D7_30min_chr11
# format = BAMPE
# ChIP-seq file = ['/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam']
# control file = None
# effective genome size = 2.70e+09
# band width = 300
# model fold = [5, 50]
# qvalue cutoff = 1.00e-02
# The maximum gap between significant sites is assigned as the read length/tag size.
# The minimum length of peaks is assigned as the predicted fragment length "d".
# Larger dataset will be scaled towards smaller dataset.
# Range for calculating regional lambda is: 10000 bps
# Broad region calling is off
# Paired-End mode is on
# MACS will save fragment pileup signal per million reads

INFO  @ Wed, 13 Jul 2022 12:26:52: #1 read fragment files...
INFO  @ Wed, 13 Jul 2022 12:26:52: #1 read treatment fragments...
INFO  @ Wed, 13 Jul 2022 12:26:52: 0 fragments have been read.
Traceback (most recent call last):
  File "/opt/conda/envs/app/bin/macs2", line 653, in <module>
    main()
  File "/opt/conda/envs/app/bin/macs2", line 51, in main
    run( args )
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 64, in run
    if options.PE_MODE: (treat, control) = load_frag_files_options (options)
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 350, in load_frag_files_options
    treat = tp.build_petrack()
  File "MACS2/IO/Parser.pyx", line 1267, in MACS2.IO.Parser.BAMPEParser.build_petrack
  File "MACS2/IO/Parser.pyx", line 1311, in MACS2.IO.Parser.BAMPEParser.build_petrack
ZeroDivisionError: float division
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11_trim1.log
::::::::::::::
TrimmomaticPE: Started with arguments:
 -threads 8 -phred33 /opt/CUT-RUNTools-2.0/exampleData/GATA1_D7_30min_chr11_R1_001.fastq.gz /opt/CUT-RUNTools-2.0/exampleData/GA
TA1_D7_30min_chr11_R2_001.fastq.gz /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/trimmed/GATA1_D7_30min_chr11_1.paired.
fastq.gz /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/trimmed/GATA1_D7_30min_chr11_1.unpaired.fastq.gz /hpcdata/scratc
h/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/trimmed/GATA1_D7_30min_chr11_2.paired.fastq.gz /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D
7_30min_chr11/trimmed/GATA1_D7_30min_chr11_2.unpaired.fastq.gz ILLUMINACLIP:/opt/CUT-RUNTools-2.0/adapters/Truseq3.PE.fa:2:15:4:
4:true LEADING:20 TRAILING:20 SLIDINGWINDOW:4:15 MINLEN:25
Using PrefixPair: 'TACACTCTTTCCCTACACGACGCTCTTCCGATCT' and 'GTGACTGGAGTTCAGACGTGTGCTCTTCCGATCT'
Using Long Clipping Sequence: 'AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA'
Using Long Clipping Sequence: 'AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC'
Using Long Clipping Sequence: 'GTGACTGGAGTTCAGACGTGTGCTCTTCCGATCT'
Using Long Clipping Sequence: 'TACACTCTTTCCCTACACGACGCTCTTCCGATCT'
ILLUMINACLIP: Using 1 prefix pairs, 4 forward/reverse sequences, 0 forward only sequences, 0 reverse only sequences
Input Read Pairs: 586516 Both Surviving: 586516 (100.00%) Forward Only Surviving: 0 (0.00%) Reverse Only Surviving: 0 (0.00%) Dr
opped: 0 (0.00%)
TrimmomaticPE: Completed successfully
Singularity>

Please let me know

I figure the issue is related to macs2 where the python you specified is not consistent with that macs2 used

I do not worry too much about it. I updated last few days for adding the executable permission of scripts in the install folder, but you can always do that by chmod +x ./install/SEACR_1.1.sh.
Yes, this error constantly occurs as the peak was not called with macs2 so I figure the issue is about macs2. Another evidence is that SEACR will actually use a bdg file generated by macs2.
I wonder if it is helpful to set up PYTHONPATH manually before you run the pipeline? https://www.biostars.org/p/155579/

I notice this
INFO  @ Wed, 13 Jul 2022 12:26:52: 0 fragments have been read.
Traceback (most recent call last):
  File "/opt/conda/envs/app/bin/macs2", line 653, in <module>
    main()
  File "/opt/conda/envs/app/bin/macs2", line 51, in main
    run( args )
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 64, in run
    if options.PE_MODE: (treat, control) = load_frag_files_options (options)
  File "/opt/conda/envs/app/lib/python3.6/site-packages/MACS2/callpeak_cmd.py", line 350, in load_frag_files_options
    treat = tp.build_petrack()
  File "MACS2/IO/Parser.pyx", line 1267, in MACS2.IO.Parser.BAMPEParser.build_petrack
  File "MACS2/IO/Parser.pyx", line 1311, in MACS2.IO.Parser.BAMPEParser.build_petrack
ZeroDivisionError: float division
::::::::::::::
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs/GATA1_D7_30min_chr11_trim1.log
::::::::::::::

can you also check if the input bam file for macs2 stage is good?

(1) I did change permissions inside the Singularity image but it still complains. I think it is trying to write files inside the container but has no permission to do so (when Singularity is run by a regular user):
Singularity> ls -l /opt/CUT-RUNTools-2.0/install/
total 21556
-rwxr-xr-x 1 root 1079     5558 Jul 11 16:16 SEACR_1.1.R
-rwxr-xr-x 1 root 1079     8098 Jul 12 21:15 SEACR_1.1.sh
-rwxr-xr-x 1 root 1079     8057 Jul 12 21:02 SEACR_1.1.sh.bak
-rwxr-xr-x 1 root 1079     1476 Jul 11 16:16 assemblies.install.sh
-rwxr-xr-x 1 root 1079      666 Jul 11 16:16 atactk.install.sh
-rw-r--r-- 1 root 1079  5169808 Jul 11 16:16 bedGraphToBigWig
-rwxr-xr-x 1 root 1079      811 Jul 11 16:16 change.bdg.py
-rwxr-xr-x 1 root 1079     1161 Jul 11 16:16 check_coordinate.py
-rw-r--r-- 1 root 1079      262 Jul 11 16:16 environment1.yml
-rw-r--r-- 1 root 1079       64 Jul 11 16:16 environment2.yml
-rw-r--r-- 1 root 1079      181 Jul 11 16:16 filter_below.awk
-rwxr-xr-x 1 root 1079      445 Jul 11 16:16 fix_sequence.py
-rwxr-xr-x 1 root 1079      838 Jul 11 16:16 get_summits_broadPeak.py
-rwxr-xr-x 1 root 1079      958 Jul 11 16:16 get_summits_seacr.py
drwxr-sr-x 3 root 1079      512 Jul 11 16:16 git
-rw-r--r-- 1 root 1079     8030 Jul 11 16:16 kseq.h
-rwxr-xr-x 1 root 1079    13080 Jul 11 16:16 kseq_test
-rw-r--r-- 1 root 1079     1906 Jul 11 16:16 kseq_test.c
-rw-r--r-- 1 root 1079     2419 Jul 11 16:16 make_cut_matrix.patch
-rwxr-xr-x 1 root 1079       37 Jul 11 16:16 make_kseq_test.sh
-rw-r--r-- 1 root 1079     1890 Jul 11 16:16 metrics.py.patch
-rwxr-xr-x 1 root 1079 16582549 Jul 11 16:16 picard-2.8.0.jar
-rw-r--r-- 1 root 1079     1558 Jul 11 16:16 r-pkgs-install.r
-rwxr-xr-x 1 root 1079     2913 Jul 11 16:16 read.meme.py
-rwxr-xr-x 1 root 1079     1253 Jul 11 16:16 run_centipede_parker.R
-rw-r--r-- 1 root 1079       24 Jul 11 16:16 sample.bed
-rwxr-xr-x 1 root 1079   126230 Jul 11 16:16 trimmomatic-0.36.jar
-rwxr-xr-x 1 root 1079    10374 Jul 11 16:16 validate.py
Singularity> ls
(2) Looks like the input bam file for macs2 stage is actually empty (is this the correct BAM?)!!!:
-rw-r--r-- 1 root 1079   0 Jul 13 12:33 /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/GATA1_D7_30min_chr11.bam

Singularity> find /hpcdata/scratch/CUT-RUNTools-2.0/* -name GATA1_D7_30min_chr11.bam -print
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/GATA1_D7_30min_chr11.bam
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/sorted/GATA1_D7_30min_chr11.bam
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dup.marked/GATA1_D7_30min_chr11.bam
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup/GATA1_D7_30min_chr11.bam
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dup.marked.120bp/GATA1_D7_30min_chr11.bam
/hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam

Singularity> ls -l /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/GATA1_D7_30min_chr11.bam /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/sorted/GATA1_D7_30min_chr11.bam /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dup.marked/GATA1_D7_30min_chr11.bam /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup/GATA1_D7_30min_chr11.bam /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dup.marked.120bp/GATA1_D7_30min_chr11.bam /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam

-rw-r--r-- 1 root 1079   0 Jul 13 12:33 /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/GATA1_D7_30min_chr11.bam
-rw-r--r-- 1 root 1079 231 Jul 13 12:33 /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup.120bp/GATA1_D7_30min_chr11.bam
-rw-r--r-- 1 root 1079 201 Jul 13 12:33 /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dedup/GATA1_D7_30min_chr11.bam
-rw-r--r-- 1 root 1079 209 Jul 13 12:33 /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dup.marked.120bp/GATA1_D7_30min_chr11.bam
-rw-r--r-- 1 root 1079  92 Jul 12 21:42 /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/dup.marked/GATA1_D7_30min_chr11.bam
-rw-r--r-- 1 root 1079  92 Jul 13 12:33 /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/aligned/sorted/GATA1_D7_30min_chr11.bam
Singularity>

Looks like there are 2 conda envs in the Singularity image:
Singularity> ls /opt/conda/envs/
app  meme

Note that 'meme' is installed separately and has its own python library:
Singularity> ls /opt/conda/envs/meme/lib/python3.6/site-packages/
README.txt			  easy_install.py		       setuptools
__pycache__			  ghostscript			       setuptools-49.6.0.post20210108-py3.6.egg-info
_distutils_hack			  pip				       wheel
certifi				  pip-21.2.4-py3.9.egg-info	       wheel-0.37.1-py3.8.egg-info
certifi-2021.5.30-py3.6.egg-info  pkg_resources
distutils-precedence.pth	  python3_ghostscript-0.5.0.dist-info
Singularity>

and its own Python:
Singularity> ls /opt/conda/envs/meme/bin/python*
/opt/conda/envs/meme/bin/python		 /opt/conda/envs/meme/bin/python3.6	    /opt/conda/envs/meme/bin/python3.6m-config
/opt/conda/envs/meme/bin/python3	 /opt/conda/envs/meme/bin/python3.6-config
/opt/conda/envs/meme/bin/python3-config  /opt/conda/envs/meme/bin/python3.6m

The default python is:
Singularity> which python
/opt/conda/envs/app/bin/python

No meme is the last step
I figure the issue may occur earlier, can you check the size of trimed files

Singularity> ls -l /hpcdata/scratch/CUT-RUNTools-2.0/
total 69
drwxrwxrwx+ 7 username admins   512 Jul 12 21:47 GATA1_D7_30min_chr11
drwxrwxrwx+ 7 username admins 32768 Jul 13 16:19 aligned
-rw-rw-rw-+ 1 username admins     0 Jul 12 21:09 fugzhmpVZ8Ltu.auc
-rw-rw-rw-+ 1 username admins     0 Jul 12 21:09 fugzhmpVZ8Ltu.auc.bed
drwxrwxrwx+ 2 username admins   512 Jul 12 16:52 logs
drwxrwxrwx+ 5 username admins   512 Jul 12 16:51 peakcalling
drwxrwxrwx+ 2 username admins   512 Jul 12 16:51 trimmed
drwxrwxrwx+ 2 username admins   512 Jul 12 16:51 trimmed2
Singularity> ls -l /hpcdata/scratch/CUT-RUNTools-2.0/trimmed
total 0
Singularity> ls -l /hpcdata/scratch/CUT-RUNTools-2.0/trimmed2
total 1
-rw-rw-rw-+ 1 username admins 20 Jul 13 16:19 _1.paired.fastq.gz
-rw-rw-rw-+ 1 username admins 20 Jul 13 16:19 _2.paired.fastq.gz
Singularity>
Does that look okey to you, @fl-yu?

No, it should be something like this

Did the files and sizes match? trimmed folder is the first you will get, trimmed2 is the second etc.

Okay can you check all the files and sizes in aligned folder?

These don't look very good... Most of them are empty files:
# ls -lh aligned/
total 43M
-rw-rw-rw-+ 1 username admins    0 Jul 12 21:48 33Us4SeIMM6VG.auc
-rw-rw-rw-+ 1 username admins    0 Jul 12 21:48 33Us4SeIMM6VG.auc.bed
-rw-rw-rw-+ 1 username admins    0 Jul 13 12:26 C1Vw9EhU0BMqW.auc
-rw-rw-rw-+ 1 username admins    0 Jul 13 12:26 C1Vw9EhU0BMqW.auc.bed
-rw-rw-rw-+ 1 username admins    0 Jul 13 12:12 cQUBjfV7FsBIP.auc
-rw-rw-rw-+ 1 username admins    0 Jul 13 12:12 cQUBjfV7FsBIP.auc.bed
drwxrwxrwx+ 2 username admins  512 Jul 12 21:47 dedup
drwxrwxrwx+ 2 username admins  512 Jul 12 21:47 dedup.120bp
drwxrwxrwx+ 2 username admins  512 Jul 12 21:47 dup.marked
drwxrwxrwx+ 2 username admins  512 Jul 12 21:47 dup.marked.120bp
-rw-rw-rw-+ 1 username admins    0 Jul 12 21:49 fPGYxdKar9heS.auc
-rw-rw-rw-+ 1 username admins    0 Jul 12 21:49 fPGYxdKar9heS.auc.bed
-rw-rw-rw-+ 1 username admins  43M Jul 13 12:26 GATA1_D7_30min_chr11.bam
-rw-rw-rw-+ 1 username2     lsb_fl    0 Jul 13 11:00 goc0wxEESZJFp.auc
-rw-rw-rw-+ 1 username2     lsb_fl    0 Jul 13 11:00 goc0wxEESZJFp.auc.bed
-rw-rw-rw-+ 1 username admins    0 Jul 13 11:49 HRZqVoYCjSZfd.auc
-rw-rw-rw-+ 1 username admins    0 Jul 13 11:49 HRZqVoYCjSZfd.auc.bed
-rw-rw-rw-+ 1 username admins    0 Jul 12 21:48 iNJJIleRqbuBV.auc
-rw-rw-rw-+ 1 username admins    0 Jul 12 21:48 iNJJIleRqbuBV.auc.bed
-rw-rw-rw-+ 1 username admins 1.1K Jul 12 21:49 metrics.GATA1_D7_30min_chr11.txt
drwxrwxrwx+ 2 username admins  512 Jul 13 12:26 sorted

As the initial bam file was generated properly and the subsequential bam files in folders of dedup, deup.120bp etc were not. I will assume bowtie2 works well.

# ls -lh aligned/sorted/
total 34M
-rw-rw-rw-+ 1 username admins 34M Jul 13 12:26 GATA1_D7_30min_chr11.bam

# ls -lh aligned/dedup
total 32K
-rw-rw-rw-+ 1 username admins 3.1K Jul 13 12:26 GATA1_D7_30min_chr11.bam

# ls -lh aligned/dedup.120bp/
total 64K
-rw-rw-rw-+ 1 username admins 3.2K Jul 13 12:26 GATA1_D7_30min_chr11.bam
-rw-rw-rw-+ 1 username admins 1.6K Jul 13 12:26 GATA1_D7_30min_chr11.bam.bai

# ls -lh aligned/dup.marked
total 32K
-rw-rw-rw-+ 1 username admins 3.2K Jul 12 21:49 GATA1_D7_30min_chr11.bam

# ls -lh aligned/dup.marked.120bp/
total 64K
-rw-rw-rw-+ 1 username admins 3.1K Jul 13 12:26 GATA1_D7_30min_chr11.bam
-rw-rw-rw-+ 1 username admins 1.6K Jul 13 12:26 GATA1_D7_30min_chr11.bam.bai

the issue begins here:
Wed Jul 13 12:26:44 EDT 2022
/opt/CUT-RUNTools-2.0/src/bulk/bulk-pipeline.sh: line 124: 12845 Killed                  $javabin/java -jar $picardbin/$picardjarfile MarkDuplicates INPUT=sorted/"$base".bam OUTPUT=dup.marked/"$base".bam VALIDATION_STRINGENCY=SILENT METRICS_FILE=metrics."$base".txt 2> $logdir/"$base".mark.dup

I still can not figure out how it comes by looking at the log files. Did you have enough memory to run the pipe?

Brilliant! I was asking for 8G of RAM (1 CPU). It is now running on 16 CPUs with 128G of RAM and, so far, looks good:
Singularity> /opt/CUT-RUNTools-2.0/run_bulkModule.sh /opt/CUT-RUNTools-2.0/config/bulk-config.json GATA1_D7_30min_chr11
==================================== Bulk data analysis pipeline will run ==============================================================
## Input FASTQ folder:            /opt/CUT-RUNTools-2.0/exampleData
## Sample name:                   GATA1_D7_30min_chr11
## Workdir folder:                /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11
## Experiment name:               GATA1_D7_30min_chr11
## Experiment type:               CUT&RUN
## Reference genome:              hg19
## Spike-in genome:               FALSE
## Spike-in normalization:        FALSE
## Fragment 120 filtration:       TRUE
=================================================================================================================================
[info] Input file is GATA1_D7_30min_chr11_R1_001.fastq.gz and GATA1_D7_30min_chr11_R2_001.fastq.gz
Thu Jul 14 14:05:20 EDT 2022
[info] Trimming file GATA1_D7_30min_chr11 ...
Thu Jul 14 14:05:20 EDT 2022
[info] Use Truseq adaptor
[info] Second stage trimming GATA1_D7_30min_chr11 ...
Thu Jul 14 14:05:28 EDT 2022
[info] Aligning file GATA1_D7_30min_chr11 to reference genome...
Thu Jul 14 14:05:37 EDT 2022
[info] Bowtie2 command: --dovetail --phred33
[info] The dovetail mode is enabled [as parameter frag_120 is on]
[info] FASTQ files won't be aligned to the spike-in genome
[info] Filtering unmapped fragments... GATA1_D7_30min_chr11.bam
Thu Jul 14 14:06:00 EDT 2022
[info] Sorting BAM... GATA1_D7_30min_chr11.bam
Thu Jul 14 14:06:03 EDT 2022
INFO	2022-07-14 18:06:04	SortSam

********** NOTE: Picard's command line syntax is changing.
**********
********** For more information, please see:
********** https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)
**********
********** The command line looks like this in the new syntax:
**********
**********    SortSam -INPUT sorted/GATA1_D7_30min_chr11.step1.bam -OUTPUT sorted/GATA1_D7_30min_chr11.bam -SORT_ORDER coordinate -VALIDATION_STRINGENCY SILENT
**********

18:06:04.746 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/opt/CUT-RUNTools-2.0/install/picard-2.8.0.jar!/com/intel/gkl/native/libgkl_compression.so
[Thu Jul 14 18:06:04 UTC 2022] SortSam INPUT=sorted/GATA1_D7_30min_chr11.step1.bam OUTPUT=sorted/GATA1_D7_30min_chr11.bam SORT_ORDER=coordinate VALIDATION_STRINGENCY=SILENT    VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
[Thu Jul 14 18:06:04 UTC 2022] Executing as ponomarevsy@ai-hpcn045 on Linux 3.10.0-1160.11.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 11.0.15-internal+0-adhoc..src; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.21.7-SNAPSHOT
INFO	2022-07-14 18:06:04	SortSam	Seen many non-increasing record positions. Printing Read-names as well.
INFO	2022-07-14 18:06:08	SortSam	Finished reading inputs, merging and writing to output now.
[Thu Jul 14 18:06:14 UTC 2022] picard.sam.SortSam done. Elapsed time: 0.16 minutes.
Runtime.totalMemory()=1166016512
[info] Marking duplicates... GATA1_D7_30min_chr11.bam
Thu Jul 14 14:06:14 EDT 2022
[info] Removing duplicates... GATA1_D7_30min_chr11.bam
Thu Jul 14 14:06:31 EDT 2022
[info] Filtering to <120bp... dup.marked and dedup BAMs
Thu Jul 14 14:06:33 EDT 2022
[info] Creating bam index files... GATA1_D7_30min_chr11.bam
Thu Jul 14 14:06:40 EDT 2022
[info] Reads shifting
Thu Jul 14 14:06:40 EDT 2022
[info] Your data won't be shifted as the experiment_type is specified as CUT&RUN...
[info] Peak calling using MACS2... GATA1_D7_30min_chr11.bam
[info] Logs are stored in /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/logs
Thu Jul 14 14:06:40 EDT 2022
[info] Peak calling with BAM file with NO duplications
[info] macs2 narrow peak calling
[info] macs2 broad peak calling
[info] Getting broad peak summits
[info] SEACR stringent peak calling
chmod: changing permissions of '/opt/CUT-RUNTools-2.0/install/SEACR_1.1.sh': Read-only file system
Calling enriched regions without control file
Proceeding without normalization of control to experimental bedgraph
Using stringent threshold
Creating experimental AUC file: Thu Jul 14 14:06:58 EDT 2022
Calculating optimal AUC threshold: Thu Jul 14 14:06:58 EDT 2022
Using user-provided threshold: Thu Jul 14 14:06:58 EDT 2022
Creating thresholded feature file: Thu Jul 14 14:06:59 EDT 2022
Empirical false discovery rate = 0.01
Merging nearby features and eliminating control-enriched features: Thu Jul 14 14:06:59 EDT 2022
Removing temporary files: Thu Jul 14 14:06:59 EDT 2022
Done: Thu Jul 14 14:06:59 EDT 2022
[info] Generating the normalized signal file with BigWig format...
Thu Jul 14 14:06:59 EDT 2022
[info] Your bigwig file won't be normalized with spike-in reads
[info] Input file is /hpcdata/scratch/CUT-RUNTools-2.0/GATA1_D7_30min_chr11/peakcalling/macs2.narrow/GATA1_D7_30min_chr11_peaks.narrowPeak
[info] Get randomized [1000] peaks from the top [2000] peaks...
[info] Filtering the blacklist regions for the selected peak files
[info] Getting Fasta sequences
Warning: the index file is older than the FASTA file.
[info] Start MEME analysis for de novo motif finding ...
[info] Up to 10 will be output ...
Log::Log4perl configuration looks suspicious: No loggers defined at /opt/conda/envs/meme/lib/site_perl/5.26.2/Log/Log4perl/Config.pm line 325.
Starting getsize: getsize random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa 1> $metrics
getsize ran successfully in 0.097219 seconds
Starting fasta-most: fasta-most -min 50 < random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa 1> $metrics
fasta-most ran successfully in 0.081656 seconds
Starting fasta-center: fasta-center -dna -len 100 < random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa 1> random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered
fasta-center ran successfully in 0.107601 seconds
Starting fasta-shuffle-letters: fasta-shuffle-letters random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-shuffled -kmer 2 -tag -dinuc -dna -seed 1
fasta-shuffle-letters ran successfully in 0.021673 seconds
Starting fasta-get-markov: fasta-get-markov -nostatus -nosummary -dna -m 1 random1000/MEME_GATA1_D7_30min_chr11_shuf/GATA1_D7_30min_chr11_summits_padded.fa random1000/MEME_GATA1_D7_30min_chr11_shuf/background
fasta-get-markov ran successfully in 0.019077 seconds
Starting meme: meme random1000/MEME_GATA1_D7_30min_chr11_shuf/seqs-centered -oc random1000/MEME_GATA1_D7_30min_chr11_shuf/meme_out -mod zoops -nmotifs 10 -minw 6 -maxw 30 -bfile random1000/MEME_GATA1_D7_30min_chr11_shuf/background -dna -revcomp -nostatus
...

What does this script do (I mean, how important it is) and it there a way to adapt it to Singularity?
chmod: changing permissions of '/opt/CUT-RUNTools-2.0/install/SEACR_1.1.sh': Read-only file system

Let me get this package installed...
Execution halted```

There are two different R versions installed in the image, one in "/usr/bin" (v4.2.0), the other is in "/opt/conda/envs/app/bin" (v3.2.2). Package 'CENTIPEDE' fails to install in older R, so, setting the PATH to newer R (inside bulk-config.json  and sc-config.json) with that package in-place saved my day. Not sure why R v3.2.2 gets installed anyway, since all required packages get installed in R v4.2.0 (looking at the Singularity definition file).

Thank you for your update and glad to hear it works!!


[Amusement Park Improvements]: Add test to match story

As discussed.

Is this what you mean?

Thank you @safwansamsudeen !


file upload

Is there any middleware or example to allow gulp-connect to support put request to upload a binary file

Here is the connect documentation:
https://www.npmjs.com/package/connect
gulp.task('connect', function() {
  connect.server({
    middleware: function(connect, opt) {
      function handlePut() {
        // todo...
      }
      return [handlePut]
    }
  });
});

i see thanks @justinmchase , so I can basicly use gulp-connect the same way as I use connect?

Pretty much, it just wraps connect so it's gulp friendly.

thank @justinmchase
I got it working using route-connect :+1:

Thank @AveVlad, I just accept the PR's from time to time :smile:


Web - Input in overlay components (alertdialog, modal)

Description
Can't lose focus by clicking outside input field
CodeSandbox/Snack link
https://snack.expo.dev/@exneval/nativebase-x-input-x-overlay
Steps to reproduce

Try to lose focus of input inside an overlay component

NativeBase Version
3.3.6
Platform

[ ] Android
[X] CRA
[ ] Expo
[ ] iOS
[X] Next

Other Platform
Web
Additional Information
No response

Hello @exneval, Thanks for reporting the issue, we'll look into this.

Any updates here? @mayank-96

We're still working on it. Hope to fix this in the coming release. Thanks for your patience.

Hi @exneval, This issue is fixed in the latest version of native-base.


Least squares polynomial fit with Dask

Per discussion with @matt-long, it would be useful to add a least squares polynomial fit to esmlab:
import xarray as xr
import dask.array as da
def _order_and_stack(darr, dim):
    dims_stacked = [d for d in darr.dims if d != dim]
    new_dims = [dim, ] + dims_stacked
    if darr.ndim > 2:
        darr_stacked = (darr.transpose(*new_dims).stack(other_dims=dims_stacked).dropna('other_dims'))

    elif darr.ndim == 2:
        darr_stacked = darr_stacked.transpose(*new_dims)

    else:
        darr_stacked = darr

    return darr_stacked

def _unstack(darr):
    if 'other_dims' in darr.dims:
        darr_unstacked = darr.unstack('other_dims')

    else:
        darr_unstacked = darr

    return darr_unstacked

def _polyfit(darr, x, deg, dim):
    dim_chunk = darr.chunks[darr.get_axis_num(dim)][0]
    x = x.chunk(chunks={dim: dim_chunk, 'degree': deg + 1})
    darr_stacked = _order_and_stack(darr, dim)
    p, err, _, _ = da.linalg.lstsq(x.data, darr_stacked.data)
    new_name = f'{darr.name}_poly_coeffs'
    new_dims = ('degree',) + darr_stacked.dims[1:]
    new_coords = {coord: darr_stacked.coords[coord] for coord in darr_stacked.coords
                  if coord != dim}
    lstsq_coeffs = xr.DataArray(p, name=new_name, coords=new_coords, dims=new_dims)
    return lstsq_coeffs

def polyfit(darr, deg=1, dim=None, coord=None):
    if not dim:
        dim = 'time'

    if not coord:
        coord = darr[dim]

    t = coord

    x = xr.concat([t ** d for d in range(deg + 1)], dim='degree')
    x = x.transpose(dim, 'degree')

    coeffs = _polyfit(darr, x, deg, dim)
    coeffs = coeffs.assign_coords(degree=range(deg + 1))
    return _unstack(coeffs)

One thing we've come across at climpred is that polyfit and other functions return zeros or NaNs when they encounter missing data in a time series. This isn't a problem to worry about if you only anticipate esmlab to be used in climate model output, but observational data frequently has missing data.
We deal with this by temporarily doing a linear interpolation to fill in the NaNs, then force those back to NaNs on the returned output (in the case of detrending; wouldn't need to replace the NaNs if just getting the slope): https://github.com/bradyrx/climpred/blob/8c1700e94d124424f5f2adf5250a22f2ff8bb3dc/climpred/stats.py#L118
We also have some snippets in there to handle Datasets in addition to DataArrays.

@kmpaul , I'm curious why this issue was closed with no explanatory comment.
It doesn't look like the enhancement has been implemented.

@klindsay28, I just re-opened it

I am planning on working on this as part of https://github.com/pydata/xarray/issues/3349


Issue 525

This pull request deals with issue #525 .
By testing the feature through scorm everything seems to be fine, I did not implement any further testing though, please inform me for any additional changes that need to be done.
If you want to test the changes via scorm player you need to ignore all the changes in the bin/exam.py file and to change the line 143 to not include shuffleQuestionGroups as well as the default value of the property shuffleQuestionGroups to true (line 230) in runtime/scripts/exam.js .

@artemdou @EVratsanou thank you very much for this! I've changed it so that Exam.question_groups ends up in the shuffled order, rather than having to refer back to questionGroupOrder all the time. Other than that, it works well!
Can you add yourselves to CONTRIBUTORS.md please?


Switched to my IB client nuget package and fixed the broken tests

Everything is OK with the IB datasource and the tests, except for #15.

top! should i merge issue-21 branch (#21) to master after PR?

Sure


Tutorial

MLIR source: https://github.com/j2kun/mlir-tutorial

Can you remove the .DS_Store file?

sure


Fix InstanceType tab loading state

📝 Description
This Pr fixes:

loading state of InstanceType tab under the catalog - instead of sections load one by one and modal to pop, I add a loading spinner to make the page loading look better
By getting inside dev tools it was possible to remove the enable IT modal and create a VM, so adding another defense to disable the create button if the feature is disabled.

🎥 Demo
Before:
https://github.com/kubevirt-ui/kubevirt-plugin/assets/67270715/397e684a-bc24-422c-aba5-5c5fea09d07d
After:
https://github.com/kubevirt-ui/kubevirt-plugin/assets/67270715/6df48245-d00e-4163-96be-276f1d5d738c

/lgtm


HELP    Performing configure step for 'foonathan_memory'

loading initial cache file /home/fenghui/Micro-XRCE-DDS-Agent/build/foonathan_memory/tmp/foonathan_memory-cache-.cmake
-- Checking 13 features
CMake Error at cmake/comp/comp_base.cmake:288 (message):
error downloading feature file cpp11_lang/alignof.cmake: "Couldn't connect
to server".  Check spelling of feature.
Trying <IP_ADDRESS>...

TCP_NODELAY set
connect to <IP_ADDRESS> port 443 failed: Connection refused
Trying ::...

TCP_NODELAY set
connect to :: port 443 failed: Connection refused
Failed to connect to raw.githubusercontent.com port 443: Connection refused
Closing connection 0
Call Stack (most recent call first):
cmake/comp/comp_base.cmake:347 (_comp_fetch_feature)
cmake/comp/comp_base.cmake:369 (_comp_handle_feature)
cmake/compatibility.cmake:13 (comp_target_features)
CMakeLists.txt:50 (include)
-- Configuring incomplete, errors occurred!
See also "/home/fenghui/Micro-XRCE-DDS-Agent/build/foonathan_memory/src/foonathan_memory-build/CMakeFiles/CMakeOutput.log".
CMakeFiles/foonathan_memory.dir/build.make:105: recipe for target 'foonathan_memory/src/foonathan_memory-stamp/foonathan_memory-configure' failed
make[2]: *** [foonathan_memory/src/foonathan_memory-stamp/foonathan_memory-configure] Error 1
CMakeFiles/Makefile2:295: recipe for target 'CMakeFiles/foonathan_memory.dir/all' failed
make[1]: *** [CMakeFiles/foonathan_memory.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

I think that Failed to connect to raw.githubusercontent.com port 443: Connection refused is related to your internet connection. We have had no problem building the Micro-XRCE Agent, is possible for you to retry with another computer and/or in another network?