PoolByteArray method for copying data between arrays

It should be possible to copy data between PoolByteArrays with similar behavior to C memcpy, except hopefully with more memory safety (error on failure?).
One thing this would allow is a GDScript implementation of a PoolByteArray-builder, by creating an Array of PoolByteArrays and copying them all into a final PoolByteArray.

This is what Read and Write structures are for (take a look at PoolVector and the way it is used), I remember asking for a way to expose them to GDNative to avoid locking at every single index access^^" maybe that can benefit to GDScript too

by creating an Array of PoolByteArrays and copying them all into a final PoolByteArray.

You can already do that easily with PoolByteArray.append_array
append_array ( PoolByteArray array )

@Faless This is not efficient though, because resizing of PoolByteArray is linear in time, and must be done for every append, resulting in a total running time quadratic in the number of strings to append.
Anyways, my use case is covered by StreamPeerBuffer as mentioned in #9663, in case anyone cares.

Actually StreamPeerBuffer already allows byte array copying, through the StreamPeer method put_partial_data (after seeking to the spot to copy to).

This is not efficient though

You are right.

Actually StreamPeerBuffer already

I was just thinking of that.
I think this can be closed then. The only thing left seems to be backporting it to 2.1 in case.

I think this can be closed then. The only thing left seems to be backporting it to 2.1 in case.

Which is requested in #10230, so closing.

Yes, thanks, I just forgot to hit close.


Update packages & A new webpack setup & Remove useless files & Update info in package.json

React: v0.14.5  -->  v17.0.2
Webpack: v1.9.11  -->  v5.64.4
...

So this PR must be a huge update.

/build-test

The build test website can not load on my browser and there are two js files are in the root path rather than PR path.
Is webpack configured right?

It was due to a wrong output.publicPath in webpack.config.js, I just fixed it.
Here is why:

publicPath: "/" --> relative to server
publicPath: "" --> relative to HTML file

/build-test

It works fine on my side, all elements are rendered as before.


Handle other state highway shields

Or should we go with California's?

This was handled in a couple of versions, but was fixed in the new update taking from Mapzen's walkabout yaml file


the img of puzzle 9 is blank

Hi，
I have discovered a problem where the image of puzzle 9 is blank.
I don't know if it's a problem with my download or if there's a problem with the dataset itself.
Can you take a look?

This is by design. All puzzles not having an associated puzzle image (i.e., puzzles that are text only) are provided a dummy white image.


Release is breaking on jenkins.cd.test.fabric8.io

Hey @nicolaferraro @rhuss
Need your help for few things related to PR build and FMP Release

The Jenkins file executed at https://ci.fabric8.io/job/fabric8-maven-plugin-pullreq-merge doesn't look like the one checked in with the code. Is the job configured to run a different Jenkins file?

What is the difference b/w https://ci.fabric8.io/job/fabric8-maven-plugin-pullreq-merge and  https://ci.fabric8.io/job/fabric8-maven-plugin-pullreq

Are we using different Jenkins files for PR builds and release?

Thanks
cc @hrishin

I'm closing it. Feel free to reopen if anyone faces this again.


Add zeighamietal module

This PR adds utility functions/variables for the Zeighami et al. 2015 and 2019 notebooks.
The helper functions are:

get_t1_cohort: returns 3D T1 information for either the main cohort (used in 2015 and 2019 papers) or the validation cohort (used in 2019 paper)
load_ppmi_csv: loads and preprocesses a PPMI csv file
convert_date_cols: converts a dataframe's date columns from strings to pandas.Timestamp
filter_date: filters a dataframe based on dates
mean_impute: imputes missing values in a dataframe with the mean

constants.py contains variables for filenames/column names/codes/etc. that are used instead of hardcoding strings everywhere. Also contains conversion maps and parameters for extracting the main/validation T1 cohorts.

Wonderful work @michellewang, thank you!


Publish the extension to Open VSX

Hello there!
Please publish this extension to the Open VSX marketplace.
Context
Unfortunately, as Microsoft prohibits usages of the Microsoft marketplace by any other products or redistribution of .vsix files from it, in order to use VS Code extensions in non-Microsoft products, we kindly ask that you take ownership of the VS Code extension namespace in OpenVSX and publish this extension on Open VSX.
What is OpenVSX? Why does it exist?
OpenVSX is a vendor neutral alternative to the MS marketplace used by most other derivatives of VS Code like VSCodium, Gitpod, OpenVSCode, Theia based IDEs, and so on.
How can you publish to Open VSX?
The docs to publish an extension can be found here. This process is straightforward and shouldn't take too long. Essentially, you need an authentication token and to execute the command ovsx publish to publish your plugin. See this example from RedHat publishes their Java extension: redhat-developer/vscode-java@ff51dbf/Jenkinsfile#L73-L82
You can also find an example PR that contributes a release GitHub Action which publishes to GitHub, MS and OpenVSX at the same time. I would like to contribute an update to https://github.com/dafny-lang/ide-vscode/blob/master/.github/workflows/ci.yaml if it's wanted in order to enable publishing.

I would like to contribute an update to https://github.com/dafny-lang/ide-vscode/blob/master/.github/workflows/ci.yaml if it's wanted in order to enable publishing.

Yes feel free to suggest a PR for that, I can review it !

I would like to contribute an update to https://github.com/dafny-lang/ide-vscode/blob/master/.github/workflows/ci.yaml if it's wanted in order to enable publishing

That would be great. We'd love to be in the OpenVSX marketplace.

any news on that?

Publishing the Dafny Extension in parallel to OpenVSX would benefit the Dafny community. It should be easy and can be done with these two steps:

Claiming the namespace “dafny-lang” (can be different) on open-vsx.org and get a token (how to)

Modifying the Github pipeline to publish it to OpenVSX (very similar as already been done for VS Marketplace)

@RustanLeino @keyboardDrummer I would be happy to help with that and sende a PR, however, especially the first step has to be done from a member of https://github.com/dafny-lang to prove legitimacy (or you add me temporarily, I work with the initial developers of the extension at OST).

@RustanLeino @keyboardDrummer is there any news on this? Let me know if I can help

Sorry for the lack of action on our side. I've created the open-vsx namespace and a request to authorize it.

Published on OpenVSX now: https://open-vsx.org/extension/dafny-lang/ide-vscode

Very nice, thanks!


Building OSGP using maven after code change

Hello,
The OSGP platform is configured  with eclipse running in tomcat. I have made little cosmetic code changes and would like to build the project osgp-adapter-domain-smart-metering. How do I rebuild the project?
Can you help me with steps for rebuilding the platform by to reflect my changes in the server?
Thanks in advance for the hep.
br,
Saravanan G

Hello Saravanan,
After making changes, you can rebuild the platform (and other projects as well) using these steps:

go to the folder of the project, /home/kevin/sources/Platform for example
run mvn clean install
in Eclipse select the platform modules and hit F5 (or right click and choose Refresh)
then use Project->Clean... to clean and build all projects

Now you should be able to see any problems/mistakes in the Markers view of Eclipse.
Then start tomcat again and your updated component(s) should be deployed and started.
Kind regards,
Kevin

Thanks Kevin for the quick response.
I tried to execute the steps provided. The Maven build is failing  with missing child modules.
dev@dev-box:~/OSGP/sources/Platform-development$ mvn clean install
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project com.alliander.osgp:platform:3.4.0-SNAPSHOT (/home/dev/OSGP/sources/Platform-development/pom.xml) has 2 errors
[ERROR]     Child module /home/dev/OSGP/sources/Platform-development/Shared/shared of /home/dev/OSGP/sources/Platform-development/pom.xml does not exist
[ERROR]     Child module /home/dev/OSGP/sources/Platform-development/Shared/osgp-dto of /home/dev/OSGP/sources/Platform-development/pom.xml does not exist
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException

Does the contents of ~/Shared project to be included inside ~/Platform-Development/Shared?
Thanks again.
br,
Saravanan G

Either you have to initialize or update the git submodule 'Shared' of the platform project. See the git documentation:
https://git-scm.com/docs/git-submodule
You can get started with these commands:

git submodule status
git submodule update --remote

Thanks a lot Kevin for throwing light on the problem.1
I was able to build the platform successfully.
br,
Saravanan G


Query all "SCHEDULED" tasks

Hi,
I have a use case where I would like to query for all the scheduled tasks (of any type) instead of polling for scheduled tasks of a specific tasktype.  Currently I have tried the task search API and with status="SCHEDULED" and it seems to be working, but I know this goes through elasticsearch to find the tasks in this state.  Would there be a way to do this without having to go through elasticsearch as I would prefer to eliminate the dependency on elasticsearch if possible.
Thanks.

@cquon without ES this does not exist today.


Ampersand character in file name causes conversion to fail [Conversion Problem]

Describe the problem
Attempting to convert an asset with a filename which includes an ampersand (possibly other characters also, I haven't tested) i.e "PLOTS12&3REVF3DViewExport.fbx" will cause an internal server errors during the conversion process.
"Conversion status update:" "converting" -> "conversion_failed" ". Message: " "InternalServerError: ConversionProcessError - An internal conversion process error occured."
Desired behavior
Either the conversion process should handle the ampersand character in the file name or the error message should be more informative to allow the user to remedy the issue.

Yep. I'm seeing this too. Thanks for bringing it to our attention! We'll try to get it resolved in a release soon.

Hey there. Just to say that we're working on this and we should have a fix soon. We'll let you know when it's resolved.

This should be resolved now. Thanks for filing the bug!


load model failed

When I load model, the software remind me that File formate is not tensorflow.saveModel(invalid wire type 7 at offset 17). I guess  this is a bug for this software?

How did you get the file and can you share it? What format do you expect the file to be in?

saved_model.zip
This is my model file to test, I have test this model using tensorflow. So the model must can be used.

The file is called saved_model.pb but is not in tensorflow.SavedModel format. If you rename the file to any name other than saved_model it will load as tensorflow.GraphDef.
See tensorflow/tensorflow#15061

Thank you very much!


Lambda@Edge tried to add, delete, or change a read-only header

Next.js version: 12.3.4
Hi, I am running a server function on Lambda@Edge with OpenNEXT.
# open-next.config.ts
import type { OpenNextConfig } from 'open-next/types/open-next';

const config = {
  default: {
    override: {
      converter: 'aws-cloudfront',
    },
  },
  imageOptimization: {
    arch: 'x64',
  },
  dangerous: {
    disableTagCache: true,
    disableIncrementalCache: true,
  },
} satisfies OpenNextConfig;

export default config;
export type Config = typeof config;

This application worked with<EMAIL_ADDRESS>but not with<EMAIL_ADDRESS>Specifically, the following error is displayed when accessing API Routes.
The Lambda function result failed validation: The function tried to add, delete, or change a read-only header. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.

I think the cause is that a read-only Content-Length header is added.
# Debug log for<EMAIL_ADDRESS>INFO	{
  status: '200',
  statusDescription: 'OK',
  headers: {
    'cache-control': [ [Object] ],
    'content-type': [ [Object] ],
    etag: [ [Object] ],
    'X-OpenNext': [ [Object] ],
    'X-OpenNext-Version': [ [Object] ],
    'X-OpenNext-RequestId': [ [Object] ]
  },
  bodyEncoding: 'text',
  body: ...
}

# Debug log for<EMAIL_ADDRESS>INFO	{
  status: '200',
  statusDescription: 'OK',
  headers: {
    'cache-control': [ [Object] ],
    'content-type': [ [Object] ],
    etag: [ [Object] ],
    'content-length': [ [Object] ],
    'X-OpenNext': [ [Object] ],
    'X-OpenNext-Version': [ [Object] ],
    'X-OpenNext-RequestId': [ [Object] ]
  },
  bodyEncoding: 'text',
  body: ...
}

Can this be fixed?

Hmm i see, it's caused by this commit https://github.com/sst/open-next/commit/3b004dda67af90e077f1237aea38966e7d2ffd92. We need to find a proper way of dealing with that.
In the meantime, you can update OpenNext to 3.0.2 and just use this https://open-next.js.org/config/custom_overrides#custom-converter.
Instead of adding stuff you just remove the headers you don't want, and use aws-cloudfront as the converter instead of aws-apigw-v2

Thanks for the reply.
I'll try this until it's fixed!

In the meantime, you can update OpenNext to 3.0.2 and just use this https://open-next.js.org/config/custom_overrides#custom-converter.
Instead of adding stuff you just remove the headers you don't want, and use aws-cloudfront as the converter instead of aws-apigw-v2


Bump Gatsby to 2.23.18 to fix  error on build.

Hey guys, I set up a new site today using the starter, but kept getting the "Error: Reducers may not dispatch actions." action whenever I tried to build (both locally and on Netlify). This looks to be a version of the issue mentioned here:
https://github.com/gatsbyjs/gatsby/issues/25478
Bumping Gatsby to 2.23.18 fixed this for me, so I'm submitting this in hopes that we can update it—or at least anyone else who runs into the issue might see this!
Thanks, let me know if you have any questions!

Thanks @jdselig !


Custom type (Un)Marshaler

Hi all,
I want to ask whether it's possible to make a field in the model fetched and stored in the database by a certain way ?
for example: Let's say that I have the following struct :
type Session struct {
  gorm.Model
  State State `gorm:"type:varchar(200)"`
}

What I need is that this state whenever it gets written in the database it calls a certain function to serialize it and whenever it gets fetched it calls another function to deserialize it. Something like json.Marshaler and json.Unmarshaler. Is this possible ? If not can it be considered as a feature request ? Thank you.

Seems like what I'm talking about is driver.Value and sql.Scanner, never knew about them before. Sorry for the inconvenience.


CRF layer v3.0

With a subclassing approch, we have a nicer API and it's very flexible.
Works only with TF 2.2+
@howl-anderson  for the review and the CLA
The plan is to show users how to do the subclassing for the CRF. We shouldn't provide and API to save them some code there because it's going to become very complex to design a good API and to maintain it later on.
So the CRF layer is a public API and for the CRF loss, we give a good tutorial about subclassing.
Quick tutorial right now:
import tensorflow as tf
from tensorflow_addons.layers.crf import CRF
from tensorflow_addons.text.crf import crf_log_likelihood

def unpack_data(data):
    if len(data) == 2:
        return data[0], data[1], None
    elif len(data) == 3:
        return data
    else:
        raise TypeError("Expected data to be a tuple of size 2 or 3.")

class ModelWithCRFLoss(tf.keras.Model):
    """Wrapper around the base model for custom training logic."""

    def __init__(self, base_model):
        super().__init__()
        self.base_model = base_model

    def call(self, inputs):
        return self.base_model(inputs)

    def compute_loss(self, x, y, sample_weights, training=False):
        y_pred = self(x, training=training)
        _, potentials, sequence_length, chain_kernel = y_pred

        crf_loss = -crf_log_likelihood(potentials, y, sequence_length, chain_kernel)[0]

        if sample_weights is not None:
            crf_loss = crf_loss * sample_weights

        return tf.reduce_mean(crf_loss), sum(self.losses)

    def train_step(self, data):
        x, y, sample_weight = unpack_data(data)

        with tf.GradientTape() as tape:
            crf_loss, internal_losses = self.compute_loss(
                x, y, sample_weight, training=True
            )
            total_loss = crf_loss + internal_losses

        gradients = tape.gradient(total_loss, self.trainable_variables)
        self.optimizer.apply_gradients(zip(gradients, self.trainable_variables))

        return {"crf_loss": crf_loss, "internal_losses": internal_losses}

    def test_step(self, data):
        x, y, sample_weight = unpack_data(data)
        crf_loss, internal_losses = self.compute_loss(x, y, sample_weight)
        return {"crf_loss_val": crf_loss, "internal_losses_val": internal_losses}

x_np, y_np = get_test_data()

x_input = tf.keras.layers.Input(shape=x_np.shape[1:])
crf_outputs = CRF(5)(x_input)
base_model = tf.keras.Model(x_input, crf_outputs)
model = ModelWithCRFLoss(base_model)

model .compile("adam")
model .fit(x=x_np, y=y_np)
model .evaluate(x_np, y_np)
model .predict(x_np)
model.save("my_model.tf")

If some users want to try this feature before it's merged, we have some wheels available

Hi,
I am relatively new to Tensorflow. Just wondering I can't seem to find tensorflow_addons.layers.crf?
And when I 'from tensorflow_addons.layers.crf im port CRF', the error shows no module named 'tensorflow_addons.layers.crf.
Thanks

Hi,
Is it possible to add some feature to allow sample weights?
Thanks!

Hi,
Is it possible to include an example with sample weights?
Also an example where the input of model.fit is a generator which yield (x, y, sample_weight) for eatch batch?
Thanks!

Any updates?

Any updates?

@gabrieldemarmiesse, are you still going to work on this PR?

Sorry, I'm very busy nowadays, somebody else is more than welcome to take this branch and open a new pull request with it :)

@gabrieldemarmiesse Hey! I'm working on a model with CRF layer recently and your solution here is very helpful. I'd gladly help finish this PR if you are not available currently ;)

I'm glad it helped you! Feel free to pull this branch into your fork and open a new PR :)

Closing in favor of #1999

base_model

Hi ~ How to use this code to build a bi-lstm-crf model ? like this ?
x_input = tf.keras.layers.Input(shape=x_np.shape[1:])
bilstm_output = bi_lstm_model(x_input)
crf_outputs = CRF(5)(bilstm_output )
base_model = tf.keras.Model(x_input, crf_outputs)
model = ModelWithCRFLoss(base_model)

With a subclassing approch, we have a nicer API and it's very flexible.
Works only with TF 2.2+
@howl-anderson for the review and the CLA
The plan is to show users how to do the subclassing for the CRF. We shouldn't provide and API to save them some code there because it's going to become very complex to design a good API and to maintain it later on.
So the CRF layer is a public API and for the CRF loss, we give a good tutorial about subclassing.
Quick tutorial right now:
import tensorflow as tf
from tensorflow_addons.layers.crf import CRF
from tensorflow_addons.text.crf import crf_log_likelihood

def unpack_data(data):
    if len(data) == 2:
        return data[0], data[1], None
    elif len(data) == 3:
        return data
    else:
        raise TypeError("Expected data to be a tuple of size 2 or 3.")

class ModelWithCRFLoss(tf.keras.Model):
    """Wrapper around the base model for custom training logic."""

    def __init__(self, base_model):
        super().__init__()
        self.base_model = base_model

    def call(self, inputs):
        return self.base_model(inputs)

    def compute_loss(self, x, y, sample_weights, training=False):
        y_pred = self(x, training=training)
        _, potentials, sequence_length, chain_kernel = y_pred

        crf_loss = -crf_log_likelihood(potentials, y, sequence_length, chain_kernel)[0]

        if sample_weights is not None:
            crf_loss = crf_loss * sample_weights

        return tf.reduce_mean(crf_loss), sum(self.losses)

    def train_step(self, data):
        x, y, sample_weight = unpack_data(data)

        with tf.GradientTape() as tape:
            crf_loss, internal_losses = self.compute_loss(
                x, y, sample_weight, training=True
            )
            total_loss = crf_loss + internal_losses

        gradients = tape.gradient(total_loss, self.trainable_variables)
        self.optimizer.apply_gradients(zip(gradients, self.trainable_variables))

        return {"crf_loss": crf_loss, "internal_losses": internal_losses}

    def test_step(self, data):
        x, y, sample_weight = unpack_data(data)
        crf_loss, internal_losses = self.compute_loss(x, y, sample_weight)
        return {"crf_loss_val": crf_loss, "internal_losses_val": internal_losses}

x_np, y_np = get_test_data()

x_input = tf.keras.layers.Input(shape=x_np.shape[1:])
crf_outputs = CRF(5)(x_input)
base_model = tf.keras.Model(x_input, crf_outputs)
model = ModelWithCRFLoss(base_model)

model .compile("adam")
model .fit(x=x_np, y=y_np)
model .evaluate(x_np, y_np)
model .predict(x_np)
model.save("my_model.tf")

If some users want to try this feature before it's merged, we have some wheels available

Hi ~ How to use this code to build a bi-lstm-crf model ? like this ?
x_input = tf.keras.layers.Input(shape=x_np.shape[1:])
bilstm_output = bi_lstm_model(x_input)
crf_outputs = CRF(5)(bilstm_output)
base_model = tf.keras.Model(x_input, crf_outputs)
model = ModelWithCRFLoss(base_model)

I build a crf tool for TF2, you can refer to tf2crf. And pip install tf2crf.

I build a crf tool for TF2, you can refer to tf2crf. And pip install tf2crf.

What's the difference between your CRF layer and tensorflow_addons.layers.crf ？

Because during a longtime the tfa.layers.crf has problems with mixed
precision and other bugs. So I implemented a CRF layer using the CRF
functions in tfa.text.crf.
On Mon, Mar 15, 2021 at 9:35 AM DachuanZhao @.***>
wrote:

I build a crf tool for TF2, you can refer to tf2crf
https://github.com/xuxingya/tf2crf. And pip install tf2crf.
What's the difference between your CRF layer and
tensorflow_addons.layers.crf ？
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/addons/pull/1733#issuecomment-799031821,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADFZVRFOGNEYKJYP4XGW3ELTDVP73ANCNFSM4MRF3B4Q
.


Feature Request: add CORS API indicating 'gitea'

TL;DR
I'd like there to be some way to indicate that the service is gitea, as opposed to gitlab.
Example
I'd like to be able to do something like this:
await fetch('https://try.gitea.io/api/v1/version', { mode: 'cors' }).toJSON()
// { "name": "gitea", "version": "1.15.0" }

Or similar to this:
await fetch('https://try.gitea.io/.well-known/openid-configuration', { mode: 'cors' }).toJSON()
// { "service_documentation": "https://docs.gitea.io" }

Just something that reveals "yes, this is gitea".
Rationalé
The reason is that I'm building a tool Bliss, which works with GitHub, GitLab, and Gitea and I'd like to be able to detect which system a user-supplied URL represents, since they have slightly different APIs.

I wouldn't necessarily recommend the oidc wellknown endpoint for discovery of software type based on a docs URL (there must be a wellknown endpoint for what type of forge is in place), although regardless of purpose if a PR were to be made for the docs URL I'd recommend using https://docs.gitea.io/en-us/oauth2-provider/
I'd also hesitate to use the version endpoint, because users are able to mandate that endpoint require credentials.

A PR for the wellknown nodeinfo endpoint can be found here: https://github.com/go-gitea/gitea/pull/16953


feat(elasticsearch): make service as an index

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

24 of 29 (82.76%)  changed or added relevant lines in 3 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.2%) to 81.227%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/store/elasticsearch/discovery_repository.go
13
18
72.22%

Files with Coverage Reduction
New Missed Lines
%

internal/store/elasticsearch/discovery_repository.go
1
74.75%

Totals

Change from base Build<PHONE_NUMBER>:
0.2%

Covered Lines:
4500

Relevant Lines:
5540

💛  - Coveralls


no output for biotype result?

Hi, I am trying to use this software to analyze my MS data. But I'm not sure if the calculation is finished, especially since none of the biotype related results are output.
Here are my input,
BAM_directories.tsv:
0_treat_1	/xx1/2stpass_sample-1_allAligned.sortedByCoord.out.bam
0_treat_2	/xx2/2stpass_sample-2_allAligned.sortedByCoord.out.bam
0_treat_3	/xx3/2stpass_sample-3_allAligned.sortedByCoord.out.bam
48_treat_1	/xx4/2stpass_sample-4_allAligned.sortedByCoord.out.bam
48_treat_2	/xx5/2stpass_sample-5_allAligned.sortedByCoord.out.bam
48_treat_3	/xx6/2stpass_sample-6_allAligned.sortedByCoord.out.bam

peptides.tsv:（600+ peptide）
LVSPPTKFV	Mouse_1st_9_peptide
WLGMKDEYT	Mouse_1st_9_peptide
TKPKLGLPE	Mouse_1st_9_peptide
RTKTKKCVP	Mouse_1st_9_peptide
VVPTPLKPH	Mouse_1st_9_peptide
WVAKGRGPV	Mouse_1st_9_peptide
LLQETAPVV	Mouse_1st_9_peptide
...

And log file is
2023-10-30 10:52:59,089 INFO Path to input folder : /xx/Input/
2023-10-30 10:52:59,089 INFO Path to output folder : /xx/output/
2023-10-30 10:52:59,089 INFO =============== # ===================
2023-10-30 10:52:59,089 INFO =============== Start Parameters ===============
2023-10-30 10:52:59,089 INFO  - BamQuery id : SB_9peptide
2023-10-30 10:52:59,089 INFO  - Mode : normal, Strandedness :  False, Light:  False
2023-10-30 10:52:59,089 INFO  - Single-Cell experiment (sc) :  False, Count UMIs : False
2023-10-30 10:52:59,089 INFO  - dbSNP :  mouse_GRCm38, COMMON SNPs : False, Genome Version : M24
2023-10-30 10:52:59,089 INFO  - Plots : True
2023-10-30 10:52:59,089 INFO  - Keep Variant Alignments : False, Keep High Amount Alignments : False
2023-10-30 10:52:59,089 INFO  - Counting overlapping reads : False
2023-10-30 10:52:59,089 INFO  - Mouse Genome : True
2023-10-30 10:52:59,090 INFO  - Threads : 12
2023-10-30 10:52:59,090 INFO =============== End Parameters ===============
2023-10-30 10:54:42,002 INFO
2023-10-30 10:54:42,002 INFO
2023-10-30 10:54:42,002 INFO BamQuery analysis will continue where it left....
2023-10-30 10:54:42,002 INFO Path to input folder : /xx/Input/
2023-10-30 10:54:42,002 INFO Path to output folder : /xx/output/
2023-10-30 10:54:42,002 INFO =============== # ===================
2023-10-30 10:54:42,002 INFO =============== Start Parameters ===============
2023-10-30 10:54:42,002 INFO  - BamQuery id : SB_9peptide
2023-10-30 10:54:42,002 INFO  - Mode : normal, Strandedness :  False, Light:  False
2023-10-30 10:54:42,002 INFO  - Single-Cell experiment (sc) :  False, Count UMIs : False
2023-10-30 10:54:42,002 INFO  - dbSNP :  mouse_GRCm38, COMMON SNPs : False, Genome Version : M24
2023-10-30 10:54:42,002 INFO  - Plots : True
2023-10-30 10:54:42,002 INFO  - Keep Variant Alignments : False, Keep High Amount Alignments : False
2023-10-30 10:54:42,003 INFO  - Counting overlapping reads : False
2023-10-30 10:54:42,003 INFO  - Mouse Genome : True
2023-10-30 10:54:42,003 INFO  - Threads : 12
2023-10-30 10:54:42,003 INFO =============== End Parameters ===============
2023-10-30 10:54:42,048 INFO Total Bam Files to Query : 6.
2023-10-30 10:54:42,053 INFO Peptides to evaluate in Peptide Mode : 613
2023-10-30 10:54:42,053 INFO Peptides to evaluate in Coding Sequence (CS) Mode : 0
2023-10-30 10:54:42,053 INFO Peptides to evaluate in Manual Mode: 0
2023-10-30 10:54:42,053 INFO Total Peptides to evaluate : 613
2023-10-30 10:54:42,053 INFO ========== Treatment File : Done! ============
2023-10-30 10:55:21,410 INFO Total Coding Sequences : 63476032
Total time run function reverse_translation to end : 0.6558857679367065min
2023-10-30 10:55:21,413 INFO ========== Reverse Translation : Done! ============
2023-10-30 10:55:21,630 INFO Command to Align using STAR : ulimit -s 8192; STAR --runThreadN 12 --genomeDir /xx/bamquery/lib/genome_versions/genome_mouse_m24/Index_STAR_2.7.9a/ --seedSearchStartLmax 20 --alignEndsType EndToEnd --sjdbOverhang 32 --alignSJDBoverhangMin 1 --alignSJoverhangMin 10000 --outFilterMismatchNmax 4 --outFilterIntronMotifs RemoveNoncanonicalUnannotated --scoreGapNoncan -50 --outFilterType BySJout --winAnchorMultimapNmax 1000 --outFilterMultimapNmax 1000000 --readFilesIn /xx/output/genome_alignments/SB_9peptide.fastq --outSAMattributes NH HI MD --limitOutSJcollapsed 5000000 --limitOutSAMoneReadBytes 266000000 --outFilterMultimapScoreRange 2 --alignTranscriptsPerWindowNmax 1000 --alignWindowsPerReadNmax 15000 --seedNoneLociPerWindow 1000 --seedPerWindowNmax 1000 --alignTranscriptsPerReadNmax 20000 --outFileNamePrefix /xx/output/genome_alignments/
2023-10-30 11:31:22,279 INFO Total time run function alignment_CS_to_genome to end : 36.010810 min
2023-10-30 11:31:22,491 INFO Using genome version /xx/bamquery/lib/genome_versions/genome_mouse_m24/GRCm38.primary_assembly.genome.fa.
2023-10-30 11:31:22,494 INFO Using dbSNP database mouse_GRCm38 with COMMON SNPs = False. Database Path : /xx/bamquery/lib//snps/snps_dics_mouse_GRCm38/
2023-10-30 11:31:58,345 INFO Time reading SAM file : 0.597516 min Chromosomes 65
2023-10-30 11:38:50,853 INFO Total time run function get_alignments to end : 7.476200 min
2023-10-30 11:38:50,854 INFO Total alignments : 423
2023-10-30 11:38:53,474 INFO Alignments Information save to : /xx/output/alignments/Alignments_information.dic
2023-10-30 11:38:53,481 INFO ========== Alignment : Done! ============
2023-10-30 11:38:53,483 INFO Total missed_peptides : 413. Find the list in : /xx/output/alignments/missed_peptides.info.
2023-10-30 11:38:53,483 INFO ========== Common_to_modes : Done! ============
2023-10-30 11:38:53,486 INFO Total MCS mapped : 423
2023-10-30 11:38:53,487 INFO Total unique regions : 353
2023-10-30 11:38:53,666 INFO It was not possible to associate these chromosomes in the genome annotations reference: GL456210.1,GL456211.1,GL456212.1,GL456213.1,GL456216.1,GL456219.1,GL456221.1,GL456233.1,GL456239.1,GL456350.1,GL456354.1,GL456359.1,GL456360.1,GL456366.1,GL456367.1,GL456368.1,GL456370.1,GL456372.1,GL456378.1,GL456379.1,GL456381.1,GL456382.1,GL456383.1,GL456385.1,GL456387.1,GL456389.1,GL456390.1,GL456392.1,GL456393.1,GL456394.1,GL456396.1,JH584292.1,JH584293.1,JH584294.1,JH584295.1,JH584296.1,JH584297.1,JH584298.1,JH584299.1,JH584300.1,JH584301.1,JH584302.1,JH584303.1,JH584304.1 to any chr in the bam file investigated: /xx/2stpass_sample-1_allAligned.sortedByCoord.out.bam.
Regions in these chromosomes will be associated with a count of zero.
2023-10-30 11:38:56,509 INFO Processed Bam File : 0 0_treat_bam_0_treat_1. Time : 0.048571 min
2023-10-30 11:38:56,573 INFO It was not possible to associate these chromosomes in the genome annotations reference: GL456210.1,GL456211.1,GL456212.1,GL456213.1,GL456216.1,GL456219.1,GL456221.1,GL456233.1,GL456239.1,GL456350.1,GL456354.1,GL456359.1,GL456360.1,GL456366.1,GL456367.1,GL456368.1,GL456370.1,GL456372.1,GL456378.1,GL456379.1,GL456381.1,GL456382.1,GL456383.1,GL456385.1,GL456387.1,GL456389.1,GL456390.1,GL456392.1,GL456393.1,GL456394.1,GL456396.1,JH584292.1,JH584293.1,JH584294.1,JH584295.1,JH584296.1,JH584297.1,JH584298.1,JH584299.1,JH584300.1,JH584301.1,JH584302.1,JH584303.1,JH584304.1 to any chr in the bam file investigated: /xx/2stpass_sample-4_allAligned.sortedByCoord.out.bam.
Regions in these chromosomes will be associated with a count of zero.
2023-10-30 11:38:59,397 INFO Processed Bam File : 1 48_treat_bam_48_treat_1. Time : 0.048127 min
2023-10-30 11:38:59,471 INFO It was not possible to associate these chromosomes in the genome annotations reference: GL456210.1,GL456211.1,GL456212.1,GL456213.1,GL456216.1,GL456219.1,GL456221.1,GL456233.1,GL456239.1,GL456350.1,GL456354.1,GL456359.1,GL456360.1,GL456366.1,GL456367.1,GL456368.1,GL456370.1,GL456372.1,GL456378.1,GL456379.1,GL456381.1,GL456382.1,GL456383.1,GL456385.1,GL456387.1,GL456389.1,GL456390.1,GL456392.1,GL456393.1,GL456394.1,GL456396.1,JH584292.1,JH584293.1,JH584294.1,JH584295.1,JH584296.1,JH584297.1,JH584298.1,JH584299.1,JH584300.1,JH584301.1,JH584302.1,JH584303.1,JH584304.1 to any chr in the bam file investigated: /xx/2stpass_sample-2_allAligned.sortedByCoord.out.bam.
Regions in these chromosomes will be associated with a count of zero.
2023-10-30 11:39:02,172 INFO Processed Bam File : 2 0_treat_bam_0_treat_2. Time : 0.046251 min
2023-10-30 11:39:02,247 INFO It was not possible to associate these chromosomes in the genome annotations reference: GL456210.1,GL456211.1,GL456212.1,GL456213.1,GL456216.1,GL456219.1,GL456221.1,GL456233.1,GL456239.1,GL456350.1,GL456354.1,GL456359.1,GL456360.1,GL456366.1,GL456367.1,GL456368.1,GL456370.1,GL456372.1,GL456378.1,GL456379.1,GL456381.1,GL456382.1,GL456383.1,GL456385.1,GL456387.1,GL456389.1,GL456390.1,GL456392.1,GL456393.1,GL456394.1,GL456396.1,JH584292.1,JH584293.1,JH584294.1,JH584295.1,JH584296.1,JH584297.1,JH584298.1,JH584299.1,JH584300.1,JH584301.1,JH584302.1,JH584303.1,JH584304.1 to any chr in the bam file investigated: /xx/2stpass_sample-5_allAligned.sortedByCoord.out.bam.
Regions in these chromosomes will be associated with a count of zero.
2023-10-30 11:39:05,199 INFO Processed Bam File : 3 48_treat_bam_48_treat_2. Time : 0.050428 min
2023-10-30 11:39:05,264 INFO It was not possible to associate these chromosomes in the genome annotations reference: GL456210.1,GL456211.1,GL456212.1,GL456213.1,GL456216.1,GL456219.1,GL456221.1,GL456233.1,GL456239.1,GL456350.1,GL456354.1,GL456359.1,GL456360.1,GL456366.1,GL456367.1,GL456368.1,GL456370.1,GL456372.1,GL456378.1,GL456379.1,GL456381.1,GL456382.1,GL456383.1,GL456385.1,GL456387.1,GL456389.1,GL456390.1,GL456392.1,GL456393.1,GL456394.1,GL456396.1,JH584292.1,JH584293.1,JH584294.1,JH584295.1,JH584296.1,JH584297.1,JH584298.1,JH584299.1,JH584300.1,JH584301.1,JH584302.1,JH584303.1,JH584304.1 to any chr in the bam file investigated: /xx/2stpass_sample-3_allAligned.sortedByCoord.out.bam.
Regions in these chromosomes will be associated with a count of zero.
2023-10-30 11:39:07,788 INFO Processed Bam File : 4 0_treat_bam_0_treat_3. Time : 0.043154 min
2023-10-30 11:39:07,842 INFO It was not possible to associate these chromosomes in the genome annotations reference: GL456210.1,GL456211.1,GL456212.1,GL456213.1,GL456216.1,GL456219.1,GL456221.1,GL456233.1,GL456239.1,GL456350.1,GL456354.1,GL456359.1,GL456360.1,GL456366.1,GL456367.1,GL456368.1,GL456370.1,GL456372.1,GL456378.1,GL456379.1,GL456381.1,GL456382.1,GL456383.1,GL456385.1,GL456387.1,GL456389.1,GL456390.1,GL456392.1,GL456393.1,GL456394.1,GL456396.1,JH584292.1,JH584293.1,JH584294.1,JH584295.1,JH584296.1,JH584297.1,JH584298.1,JH584299.1,JH584300.1,JH584301.1,JH584302.1,JH584303.1,JH584304.1 to any chr in the bam file investigated: /xx/2stpass_sample-6_allAligned.sortedByCoord.out.bam.
Regions in these chromosomes will be associated with a count of zero.
2023-10-30 11:39:10,805 INFO Processed Bam File : 5 48_treat_bam_48_treat_3. Time : 0.050287 min
2023-10-30 11:39:10,848 INFO Average time to process a BamFile : 0.047803 min
2023-10-30 11:39:10,887 INFO Counts Information saved to : /xx/output/res/temps_files/SB_9peptide_rna_count.csv
2023-10-30 11:39:10,887 INFO Total time run function get_counts to end : 0.290035 min

Hi Sundy,
I have a suggestion and also some questions for you.
First of all, I noticed that you are querying 613 peptides and only for 200 of them you have found alignments in the genome.
So I suggest you include in the search the parameters --var and --maxmm, this will hopefully help to find locations for the missing peptides.
I searched for your peptides and when I am not using the two parameters 6/7 peptides are missed but when I add those two parameters only 1 peptide is missed. So the parameters should look like this:
2023-11-02 21:23:29,897 INFO =============== Start Parameters ===============
2023-11-02 21:23:29,897 INFO  - BamQuery id : mouse_example
2023-11-02 21:23:29,897 INFO  - Mode : normal, Strandedness :  False, Light:  False
2023-11-02 21:23:29,897 INFO  - Single-Cell experiment (sc) :  False, Count UMIs : False
2023-11-02 21:23:29,897 INFO  - dbSNP :  mouse_GRCm38, COMMON SNPs : False, Genome Version : M24
2023-11-02 21:23:29,897 INFO  - Plots : True
2023-11-02 21:23:29,897 INFO  - Keep Variant Alignments : True, Keep High Amount Alignments : True
2023-11-02 21:23:29,897 INFO  - Counting overlapping reads : False
2023-11-02 21:23:29,897 INFO  - Mouse Genome : True
2023-11-02 21:23:29,897 INFO  - Threads : 4
2023-11-02 21:23:29,897 INFO =============== End Parameters ===============

Also, I wanted to make sure that the bam files you are querying the peptides in were aligned in the murine GRCm38 genome.
Lastly, did you get any errors?
It looks like BamQuery didn't finish correctly, that's why you don't have the biotype classification, but the console should indicate if there were any errors that prevented BamQuery from finishing.

Hi, thank you for reply.
I tried two sets of bam files, the first one I added multiple parameters when comparing with STAR, and the second one I didn't add any parameters, and neither of them output the biotype inference. It is assumed that the bam file format is the cause of the error. Seems there is no mention of bam file requirements in the tutorial. Is there some information I'm missing?
Here is the error I got
Traceback (most recent call last):   File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 646, in <module>     main(sys.argv[1:])   File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 604, in main     BamQuery(path_to_input_folder, path_to_output_folder, name_exp, mode, strandedness, th_out, light, dev, plots, dbSNP, c, super_logger, bam_files_logger, sc, umi, var, maxmm, genome_version, overlap, mouse, t)   File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 53, in __init__     self.run_bam_query_normal_mode(bam_files_logger)   File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 103, in run_bam_query_normal_mode     plots.get_heat_map(df_counts_rna, self.path_to_output_folder+'plots/heat_maps/transcription_evidence_heatmap/', self.mode, path_temps_file, self.name_exp, '_rna_counts', False, self.th_out)   File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/plotting/plots.py", line 85, in get_heat_map     tissue = bam_files_info_query[sample][6] KeyError: 'Position'
another try with different bam (STAR run with no additional parameters), the error is
`Traceback (most recent call last):
File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/genomics/normalization.py", line 61, in get_normalization
info_bam_file = dictionary_total_reads_bam_files[name_bam_file]
KeyError: 'Position'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 646, in
main(sys.argv[1:])
File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 604, in main
BamQuery(path_to_input_folder, path_to_output_folder, name_exp, mode, strandedness, th_out, light, dev, plots, dbSNP, c, super_logger, bam_files_logger, sc, umi, var, maxmm, genome_version, overlap, mouse, t)
File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 53, in init
self.run_bam_query_normal_mode(bam_files_logger)
File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/BamQuery.py", line 108, in run_bam_query_normal_mode
def_norm_rna = normalization.get_normalization(df_counts_rna, '_rna_norm.csv')
File "/slurm/home/yrd/lulab/lulinrong/bamquery/BamQuery/genomics/normalization.py", line 84, in get_normalization
raise Exception("\nBefore to continue you need to verify that the primary read count for the bam file "+name_bam_file+" is already included in the dictionary. To do so: verify in the log Get_Read_Count_BAM_directories.log that the primary read count processes have finished. Please re-launch BamQuery, once all the primary read counts have been included." )
Exception:
Before to continue you need to verify that the primary read count for the bam file Position is already included in the dictionary. To do so: verify in the log Get_Read_Count_BAM_directories.log that the primary read count processes have finished. Please re-launch BamQuery, once all the primary read counts have been included.
`
Then I check the code, my bam_files_info_query.dic file is
>>> bam_files_info_query {'bam_file_sample-1_all': ['/xx/bam_file/sample-1_all/2stpass_sample-1_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '0_treat_1', 83803720, 'R1', '0h'], 'bam_file_sample-2_all': ['/xx/bam_file/sample-2_all/2stpass_sample-2_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '0_treat_2', 73483150, 'R2', '0h'], 'bam_file_sample-3_all': ['/xx/bam_file/sample-3_all/2stpass_sample-3_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '0_treat_3', 73895430, 'R3', '0h'], 'bam_file_sample-4_all': ['/xx/bam_file/sample-4_all/2stpass_sample-4_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '48_treat_1', 70786450, 'R1', '48h'], 'bam_file_sample-5_all': ['/xx/bam_file/sample-5_all/2stpass_sample-5_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '48_treat_2', 75071165, 'R2', '48h'], 'bam_file_sample-6_all': ['/xx/bam_file/sample-6_all/2stpass_sample-6_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '48_treat_3', 77277771, 'R3', '48h']}

Hi, thank you for reply.
I tried two sets of bam files, the first one I added multiple parameters when comparing with STAR, and the second one I didn't add any parameters, and neither of them output the biotype inference. It is assumed that the bam file format is the cause of the error. Seems there is no mention of bam file requirements in the tutorial. Is there some information I'm missing?
Here is the error I got
Traceback (most recent call last): File "/xx/bamquery/BamQuery/BamQuery.py", line 646, in <module> main(sys.argv[1:]) File "/xx/bamquery/BamQuery/BamQuery.py", line 604, in main BamQuery(path_to_input_folder, path_to_output_folder, name_exp, mode, strandedness, th_out, light, dev, plots, dbSNP, c, super_logger, bam_files_logger, sc, umi, var, maxmm, genome_version, overlap, mouse, t) File "/xx/bamquery/BamQuery/BamQuery.py", line 53, in __init__ self.run_bam_query_normal_mode(bam_files_logger) File "/xx/bamquery/BamQuery/BamQuery.py", line 103, in run_bam_query_normal_mode plots.get_heat_map(df_counts_rna, self.path_to_output_folder+'plots/heat_maps/transcription_evidence_heatmap/', self.mode, path_temps_file, self.name_exp, '_rna_counts', False, self.th_out) File "/xx/bamquery/BamQuery/plotting/plots.py", line 85, in get_heat_map tissue = bam_files_info_query[sample][6] KeyError: 'Position'
another try with different bam (STAR run with no additional parameters), the error is
Traceback (most recent call last): File "/xx/bamquery/BamQuery/genomics/normalization.py", line 61, in get_normalization info_bam_file = dictionary_total_reads_bam_files[name_bam_file] KeyError: 'Position' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/xx/bamquery/BamQuery/BamQuery.py", line 646, in  main(sys.argv[1:]) File "/xx/bamquery/BamQuery/BamQuery.py", line 604, in main BamQuery(path_to_input_folder, path_to_output_folder, name_exp, mode, strandedness, th_out, light, dev, plots, dbSNP, c, super_logger, bam_files_logger, sc, umi, var, maxmm, genome_version, overlap, mouse, t) File "/xx/bamquery/BamQuery/BamQuery.py", line 53, in init self.run_bam_query_normal_mode(bam_files_logger) File "/xx/bamquery/BamQuery/BamQuery.py", line 108, in run_bam_query_normal_mode def_norm_rna = normalization.get_normalization(df_counts_rna, '_rna_norm.csv') File "/xx/bamquery/BamQuery/genomics/normalization.py", line 84, in get_normalization raise Exception("\nBefore to continue you need to verify that the primary read count for the bam file "+name_bam_file+" is already included in the dictionary. To do so: verify in the log Get_Read_Count_BAM_directories.log that the primary read count processes have finished. Please re-launch BamQuery, once all the primary read counts have been included." ) Exception: Before to continue you need to verify that the primary read count for the bam file Position is already included in the dictionary. To do so: verify in the log Get_Read_Count_BAM_directories.log that the primary read count processes have finished. Please re-launch BamQuery, once all the primary read counts have been included.
Then I check the code, my bam_files_info_query.dic file is
>>> bam_files_info_query {'bam_file_sample-1_all': ['/xx/bam_file/sample-1_all/2stpass_sample-1_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '0_treat_1', 83803720, 'R1', '0h'], 'bam_file_sample-2_all': ['/xx/bam_file/sample-2_all/2stpass_sample-2_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '0_treat_2', 73483150, 'R2', '0h'], 'bam_file_sample-3_all': ['/xx/bam_file/sample-3_all/2stpass_sample-3_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '0_treat_3', 73895430, 'R3', '0h'], 'bam_file_sample-4_all': ['/xx/bam_file/sample-4_all/2stpass_sample-4_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '48_treat_1', 70786450, 'R1', '48h'], 'bam_file_sample-5_all': ['/xx/bam_file/sample-5_all/2stpass_sample-5_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '48_treat_2', 75071165, 'R2', '48h'], 'bam_file_sample-6_all': ['/xx/bam_file/sample-6_all/2stpass_sample-6_allAligned.sortedByCoord.out.bam', 'unstranded', 'unstranded', '48_treat_3', 77277771, 'R3', '48h']}


Error response from daemon: unauthorized: access to the requested resource is not authorized

It seems the nfs-provisioner in quay is not publically accessible?
$ docker pull quay.io/hpestorage/nfs-provisioner:v1.0.0
Error response from daemon: unauthorized: access to the requested resource is not authorized

Just to verify my docker is setup ok:
$ docker pull quay.io/coreos/etcd
Using default tag: latest
latest: Pulling from coreos/etcd
ff3a5c916c92: Downloading [===>                                               ]  164.7kB/2.066MB
...

Hi Chris, we have not migrated that image to Quay just yet, it's still in docker.io.
$ docker pull hpestorage/nfs-provisioner:v1.0.0
v1.0.0: Pulling from hpestorage/nfs-provisioner
ed5dc850ecaf: Downloading [=======>                                           ]  4.941MB/32.25MB


Android

Hi, I see that you have issue for android plugin but Is there any way to use reflection-no-reflection for android?

This project has been designed mainly for Android. It's not production
ready yet, but we are working on it.
next week, we even plan a lot of coding on it...
But for now, sorry, I don't think this project can be useful to you, except
maybe if you are writing an annotation processor lib and wanna play with
some experimental stuff. But the runtime part of RnR is not ready yet.
2015-07-15 14:30 GMT-07:00 bobroalexandr<EMAIL_ADDRESS>
Hi, I see that you have issue for android plugin but Is there any way to
use reflection-no-reflection for android?
—
Reply to this email directly or view it on GitHub
https://github.com/stephanenicolas/reflection-no-reflection/issues/3.

Thank you for responce. Looking forward for some updates. Idea is really great.

Thx for your encouraging words. Stay tuned ;)
2015-07-15 14:34 GMT-07:00 bobroalexandr<EMAIL_ADDRESS>
Thank you for responce. Looking forward for some updates. Idea is really
great.
—
Reply to this email directly or view it on GitHub
https://github.com/stephanenicolas/reflection-no-reflection/issues/3#issuecomment-121757182
.


[BEAM-4176] Filtering Metrics and Splittable ParDo tests for Portable Runner

Metrics and Splittable ParDo are not yet supported on portable runners.
Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.


Fixed images sourced from tars and IDs to have final filesystem

fixes #11
@aaron-prindle @nkubala @abbytiz

lgtm

LGTM


[Typo/Bug] review and revise 'gonadotropin releasing neuron'

CL term
CL:0011111 'gonadotropin releasing neuron'
Description of typo, bug or error
The concepts of "gonadotropin-releasing hormone" and "gonadotropin" seem to have been conflated when creating CL:0011111 'gonadotropin releasing neuron'. The label, dbxref, exact synonyms and EA do not describe the same cell type.

I propose the following actions to fix this issue:

[ ] Update name to gonadotropin-releasing hormone neuron
[ ] - [ ] Update text definition:

A neuroendocrine cell that is born in the nasal placode during embryonic development and migrates through the nose and forebrain to the hypothalamus. This cell regulates reproduction by secreting gonadotropin-releasing hormone into the pituitary portal vessels to induce the release of gonadotropins into the general circulation.

[ ] Remove equivalent axiom and add the two following subclasses: 'neuroendocrine cell' and 'capable of' some 'regulation of gonadotropin secretion'.

Thanks for looking into this, @aleixpuigb.
Agree with the label change.
For text def, I recommend the first sentence to explicitly define the cell: A neuroendocrine cell that secretes gonadotropin-releasing hormone.
Regarding "'capable of' some 'regulation of gonadotropin secretion'"...
the gonadotropin-releasing hormone (GnRH) is the entity which regulates gonadotropin secretion.
'capable of' is not a transitive property.
Therefore, 'capable of regulating' some 'gonadotropin secretion' may be more accurate.
'capable of regulating' def:
Holds between c and p if and only if c is capable of some activity a, and a regulates p.
In the present example:
c = gonadotropin-releasing hormone neuron
a = GnRH secretion
p = gonadotropin secretion

thank you for your comments, @bvarner-ebi.

Therefore, 'gonadotropin-releasing hormone neuron' 'capable of regulating' some 'gonadotropin secretion' may be more accurate.

Perfect! I wasn't very convinced either by it, so this option is great!


Remove jQuery dependency

It should mostly be backwards-compatible, but there are some breaking changes:

No IE 8 support: Although I haven't tested it yet, IE 9 should still work.
No "programmatic value changes": There's just no way to detect el.value changes on an element. If you want to change the handle position, you'll have to call the setPosition method for now.

I haven't touched any of the actual calculations or styling parts for this pull request, but I think parts of them could be improved:

Use CSS transformations instead of left and width to update the handle and fill elements
Use requestAnimationFrame to throttle resize and *move events
Cache some stuff like getBoundingClientRect etc.
Not really sure about the difference between getPositionFromNode (internal) and getBoundingClientRect, but as far as I have seen it's more or less the same. Using only the latter, we could save some bytes and also tweak performance. EDIT: What is this part good for, anyway? I temporarily removed it and everything worked just fine. If we get rid of that part, we could also remove the getPositionFromNode function.
Use % values instead of px values to position the handle. That way, you could, among other things, call the window resize event much less often.

Also, if this doesn't get merged, it's no big deal. I needed a small input type="range" "polyfill", anyway.
Resolves #175

@mrksbnch first of all thanks for the effort and the pr.
Give me some time to review and think about it ✌

@mrksbnch : Awesome! You should probably rebase the commits to make the PR clean. There are lots of good docs floating around the web on how to do it.
TL;DR; Basically you just rebase, amend commit messages as necessary and force push, which will update the PR automatically.

@avindra Good idea... done

@mrksbnch, Hi!
Thank you for this PR! Unfortunately, I've came across a bug on iPad. I can't move "rangeslider__handle". To be more precise, it jumps to the beginning of the rangeslider and doesn't change its position anymore...

@sasha-bichkov Thanks, I don't have an iPad here to test this, but it works on all of my other devices. Could you do me a favor and check if there's any error message in the console? If you are on a Mac, you can connect your iPad to your Mac and use Safari to check the console.
Also, please note that I haven't committed any files in the dist folder.  So I did that and it turns out that you are right. Thanks again for reporting this!
Although I fixed and tested it, I'd highly appreciate it, if you could have another look on your iPad. It could be that the iOS Simulator is not 100% correct.

I've been trying this pr out, and it seems like uglify is doing strange things to the constructor, i.e. adding in a return to the first statement in the constructor regardless of what that first statement is, which means that basically nothing in the constructor gets called. Is anyone else getting this?

@mrksbnch I've just checked it out and it works perfect! I've tested on both an iPhone and an iPad. Thanks!

@mrmagooey I just checked the example page with the minified JavaScript file and it works for me. So I'm not exactly sure why you are having this issue.

Has this been merged? Can we get a version of script without the jquery plugin code?

Hey everyone - great to check this thread has been going for a while! Is this being considered for merge any time soon?

+1

Is someone looking into the merge request? It would be awesome to have this done soon. Thanks!

@andreruffert can you please nominate a contributor as a new maintainer?
Seems like you dont have time for this project, which is cool, but there are folks still using it 😃

I'm closing this pull request. I don't think it's worth resolving the conflicts considering that it'll probably never get merged.


JumpCloud related changes

What does this PR do?
Changes to integrate JumpCloud requirement. Fixed https://github.com/boxyhq/jackson/issues/117
Type of change

[X] New feature (non-breaking change which adds functionality)
[X] This change requires a documentation update

How should this be tested?

[X] Existing unit tests

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code and corrected any misspellings
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes

@ukrocks007 It looks like this PR also includes the file changes from the Setup Link PR.

@ukrocks007 It looks like this PR also includes the file changes from the Setup Link PR.

I had to include that as the flow required Setup Link

@ukrocks007 It looks like this PR also includes the file changes from the Setup Link PR.

We needed those changes for the flow

@ukrocks007 Unit tests are failing


Simple annotation produces no output (acessos)

I provided a simple annotation as shown in the screen below, and the output is empty, and when switching to output mode the colors disappear. zip of the project also attached.

acessos.zip

Status on this one?

solved in 2.10.0-rc1, although some issues remain on getting the property type correct.


feat(kiali): add OWNERS file to kiali* plugin

Add owners to kiali* plugin
All of them are in the  kiali GOVERNANCE

/approve

/lgtm

/verify-owners


Twap integration

Motivation:
Price feeds are critical for letting others build derivatives, integrating the TWAP into the pool creates incentives for builders to integrate into the pool, creating higher usage and demand for the pool.
Implementation:
Create a rolling price history of the pool balance between swaps, at the start of the swap, append a price snapshot to the price history. I chose to do 5 rolling snapshots as it seemed like a good balance and based on historical usage there are around 5 txs every 5-15 minutes in the juno-axlusdc pool.
Using a query msg to expose the history allows builders to adjust how they would like to calculate the final price feed. Some may choose a simple avg, a time-weighted avg, or another scheme to best get an oracle.
Testing:
I created unit tests, showing its usefulness in a Flash Loan Attack, and how it would require more capital and time ( arb risks for attacker ), to successfully manipulate the oracle.
Additionally, I created a ts script that can deploy the contracts and demo the price feed. You may need to add a umockusdc to your LocalJuno genesis file to run it. I saw the deploy scripts are written in bash, but standard practice and ease of use says, a ts file makes more sense.

Great work. This is a very important feature. A couple of comments

This deviates a bit from how TWAPs are typically done. The pair contract typically doesn’t store actual price feeds. It just reveals cumulative prices (weighted by time). Price feeds are implemented in a separate oracle contract that samples the cumulative prices at whatever interval.

There’s only 3 things you really need in state:

price0_cumulative: sum of price0 * time_since_last_update
price1_cumulative: sum of price1 * time_since_last_update
timestamp: time of last update, needed to calculate 1 and 2

And all an oracle client has to do is sample the price0_cumulative or price1_cumulative at whatever time interval.

In the current implementation, there’s no time weighted price accumulator at all really. So TWAP oracle clients can’t reliably sample the prices at custom time intervals. Imagine if there’s 5 swaps in 5 consecutive blocks, the max time granularity would be 1-5 blocks which is not ideal. This is why a generic accumulator should exist and the price feed logic should be implemented in a separate oracle contract.
You should probably call update_twap in add / remove liquidity handlers as well

Checkout: Uniswap V2 and Astroport for examples

Thanks!
I do agree that oracles are typically stored and maintained in a separate contract, but that adds an unnecessary and depending on the number of oracles + sample frequency,  creates an expensive upfront cost to anyone trying to build derivatives etc. In many cases, it is prohibitively expensive. Having the pool retain a memory adds some gas costs to swappers, but in return, it enables the infrastructure of derivatives and greatly expedites the speed of development.
Without recreating Chainlink or Band, I do believe exposing the price feed with timestamps adds sufficient robustness and manipulation resistance. In the case of manipulation, one would have to do a bad trade, and maintain the trade there for at least 5 blocks (which is a parameter and 5 is just a starting place for discussion). In the world of arbs, 5 blocks is an eternity, and the attacker will likely be arb'd to death through an osmosis pool or other DEX. Crypto markets are generally efficient within themselves and there are people looking for arb opportunities between DEXs at all times often picking up only pennies, so I am not so worried about someone trying to take on that risk.
In terms of why this approach, I was trying to let a protocol or any user decide how they create their oracle from the feed. In the integration test, I show one approach of forming a traditional TWAP @verminator23 , maybe the PR is misnambed and should be called price memory to enable TWAPs and other oracles ?
Lastly, I didn't add the update twap in the add and remove liquidity as they don't have a price impact as you are give / given a proportional amount of both

I just don't think there's any way to actually use these price feeds securely. In UNI V2 oracles, you typically have to manipulate the price for an extended period of time e.g., 30 min / 1 hr, which is prohibitively expensive. In this case, you only have to manipulate the price for up to 5 blocks (~30 seconds).  And this lower bound on duration is effectively hardcoded and not configurable, since the pair contract has an explicit cap on how many price updates it stores. UNI V2 TWAP oracles allow you to have custom oracle durations pretty easily, since an accumulator is stored directly in the pair contract.
I guess if you just added the price accumulators this would support same features as UNI V2 TWAP oracles. But wouldn't recommend anyone use the price feeds stored here.
Do you have an examples of a system like this being used in any popular DEXes in production? And Chainlink price feeds don't count because they aggregate prices from various centralized exchanges based on volume (fundamentally different security assumptions / properties). The UNI V2 TWAP design is elegant and has been battle tested. We don't need to reinvent anything here.

5 swaps is meant to be a starting place for discussion, so I wouldn't get hung up on 5 being insufficient, I agree, and 50 is probably overkill, so there likely exists a happy middle ground. the main ethos is to have the oracle price / price feed on the pool is to not increase the scope of projects that require oracles. Other 'popular' DEXs live in ecosystems that are much more mature and can rely on Link or Band to save the day, this is not the case for Juno as of writing, so there is less of a need for this type of development.
And yes you can do it in a separate contract, but let's say you need a price accurate to the minute, you would need to update it 1x60x24x365x (gas fee) x (price of juno) x (number of oracles), I think we can see it's going to be prohibitively expensive to do so for small projects of which Juno is almost exclusively composed of.
I can implement the uniswap design, but I mainly wanted to let protocols decide how they aggregate their prices instead of forcing a particular design. @verminator23 thoughts on this?

Yep @professionallyatortoise we really appreciate the PR but I 100% concur with @verminator23. The weighted average part of a TWAP is key to its resilience.
We have already ported the Astroport oracle to JunoSwap so your energies may be better spent reviewing/contributing there than reimplementing Uniswap.

Your concept of invoking the update before the price changes is really interesting though. It completely removes the possibility for a sandwich attack and ensures the price used is always the the last price from the previous block.
If we exclusively allowed the wasmswap contract to update the oracle, this could be a really solid design improvement.

The TWAP can be built from the price history, you have the prices, you have times which means you have the twap, you can check the integration test for a sample implementation of building a twap based on these values. Though there's certainly an argument to be made for inefficiency, if there's a clear winner in twap implementations then just do it once instead of making each protocol come up with its own scheme.

Yes, if you do the current price you are going to be attacked much easier, so it is best to update the twap before any swapping occurs, or as you said 1 swap behind the actual price, requiring at least 1 arb opportunity

And yes in terms of calling a separate oracle contract to update the price, yes, that's certainly possible, but why not just have it all in one place where you are updating values on the contract itself instead of calling a separate contract? It seems like this is a missed opportunity, to bring more relevance to the pool as more teams will be incentived to hook into, instead of other DEXs like what loop is building for example.

What are thoughts about porting your twap code into the pool contract itself so it's a uniswap / astroport style twap but built into the contract?

@verminator23 @mccallofthewild

I am by no means an expert in price oracles, but it feels to me that we could solve this whole issue by just storing every historical price and time of that price in state. Then, anyone wanting a TWAP of whatever accuracy can just query for the time window they're interested in.
For example, in state we can store:
/// Maps (block_height, transaction_index) to the price _before_ that TX occured.
/// See: `env.transaction.index`.
pub const PRICE_HISTORY: Map<(u64, u32), PriceSnapshot> = Map::new("price_history")

Then, on every price change we can update this map. Now, to calculate the TWAP using the last ten blocks of price information. I can get the needed data with:
let iterator_over_snapshots = cw_storage_plus::range_with_prefix(deps.storage, PRICE_HISTORY.namespace(), Some(Bound::inclusive(height - 10).to_raw_bound(), None, Order::Ascending);

CosmWasm storage is prefixed, so you can get all the values belonging to a prefix in a gas efficient manner. We set our keys as (height, index) so that we can have more than one key at a given height, and still efficiently iterate over all the keys at a height. The alternative would be to store Map(u64, Vec<PriceSnapshot>) but this kills our time complexity because we have to load in a large vec for every height. By doing it this prefixed lookup we can outsource that heavy lifting to builtin CosmWasm database functions which should give us more favorable gas usage.
This should provide O(number of prices you want) performance, and we can pretty easily paginate this so that larger queries can be supported if needed.

@ezekiiel every block is fine. We don't have a usecase for the price at every swap. As is, the range query would run out of gas if there were hundreds of swaps in the past 10 blocks. And we can index both height and timestamp with a MultiIndex.

@professionallyatortoise twap can be built from price history. But the idea is that the price updates slowly over time to accurately indicate general market sentiment. If we look at prices over the course of a day stored at every block, it's around 14k records.

why not just have it all in one place where you are updating values on the contract itself instead of calling a separate contract?

I err on the side of conservative bytecodes. I'll leave it up to the junoswap team.

@professionallyatortoise twap can be built from price history. But the idea is that the price updates slowly over time to accurately indicate general market sentiment. If we look at prices over the course of a day stored at every block, it's around 14k records.

I'm not so sure about a snapshot of every swap, it seems a bit like overkill with values becoming irrelevant very quickly, I also believe there's a contract size limit not sure if that includes storage (but something to consider), which is why I was a fan of a rolling price history.

why not just have it all in one place where you are updating values on the contract itself instead of calling a separate contract?

I err on the side of conservative bytecodes. I'll leave it up to the junoswap team.

In your design, you have to actively maintain that price feed, a slip execution or internet loss or something could cause you to have a stale price having many downstream affects (such as bad liquidations, bad debt, etc), in this design you don't run into such a problem, the price feed is gathered and stored in-contract requiring no 3rd party. I am leaning towards the price accumulator or price history, but on-contract
Anyone on the wasmswap team want to chime in?

The size limit is the bytecode size limit I referred to above. That's why it's generally unwise to cram two contracts into one.

In your design...

The TWAP would be updated on-chain before the swap. Directly or in a prehook. Once per block. There is no third party.

@ben2x4 The goal here is to make a junoswap oracle. Would you prefer the oracle in the AMM or in a separate contract?

The size limit is the bytecode size limit I referred to above. That's why it's generally unwise to cram two contracts into one.

In your design...

The TWAP would be updated on-chain before the swap. Directly or in a response message. Once per block. There is no third party.

Not sure how my proposal is 'cramming' two contracts in one...when it's a simple operation... saving a few data points doesn't need to be its own contract...Also, your suggestion would introducing an outside dependency to the wasmswap contract by broadcasting it directly, which in my opinion is far from ideal, maintaining 2 contracts, initializing it from the start, adding dependencies to your contract interface etc. The AMM should be all-inclusive and not be making unnecessary outbound execute msgs

Wow, ton of discussion here! Will dig through this tomorrow!

@mccallofthewild The size limit is the bytecode size limit I referred to above

FWIW - I asked about this in the CosmWasm Discord. I think we are unlikely to hit any issues here. From the discord:

The size of the map doesn't matter. There's a cost per-read and a cost per-byte. There's also a cost per-iteration but it's not clear to me if that's in addition or in place of the read cost.

My read is that storing a price value at every block there is a transaction would be fine. IMO this is a pretty simple contract change and could significantly reduce the cost of using Junoswap as an oracle as there is no need to pay TX fees to update an external contract. Will defer to @ben2x4's wisdom here though. :)
As an aside, if we're worried about increased fees for users swapping we've been doing a lot of thinking about gas costs at DAO DAO. The general conclusion has tended to be that gas costs from CosmWasm execution are generally dwarfed by other ones related to submitting a TX. We're working on a way to test gas usage and we could use that here to compare before and after swap prices if we're particularly concerned: https://github.com/DA0-DA0/dao-contracts/pull/410

@ezekiiel We're working on a way to test gas usage

This is awesome 🔥
I'm not opposed to the price history idea. But before moving forward we should have a consensus on how it can be converted into a manipulation-resistant oracle.


Added phantomjs on-fly proxy support

Hi Binux,
I have added a few lines to 'phantomjs_fetcher.js' to allow using proxy for phantomjs in the script.
Best,
magicknight

It have one issue, when multiple requests with different proxy sent to phantomjs, the proxy will mess up.

It is possible to set proxy for page https://github.com/ariya/phantomjs/issues/14157


Fix docker PWD and network

I successfully used the llmserver and the chatbot, using llama and Mistral.
I Did not try the other parts

Thanks @the023 !!! 🙏  Great fixes!


Naive recipe and thin global evaluators - draf for now, for future consideration

For evaluation, the streams are not so long, and operators must not be thin. They can hold a whole stream, lying in main memory. As done by huggingface.


change indices argument during training?

Hi,
Thanks for the amazing work. My question is kind of basic. I want to know if I can change the indices the loader accesses during training. I'll try to explain through the use of an example. Suppose my dataset is [1..100] and I want to train on the set [0..49] on one epoch and on the set [50..100] for the subsequent epoch and I want to alternate between these two sets. In Pytorch, I can achieve this by changing the set of indices to be sampled with the SubsetRandomSampler class. Can I do something similar with ffcv or do I have to recompile my dataloader every epoch?

@rraju1 there is a parameter indices in the constructor of the Loader. It's not technically part of the API but you could just update loader.indices = SET_1 or SET_2 before you start each epoch. You shouldn't incur any performance penalty.
Hope it helps!
(feel free to reopen if it doesn't work for you)

@GuillaumeLeclerc I tried your suggestion but it didn't update number of batches the network was processing (going from full set to a subset). But when I changed train_loader.indices = SET1 and train_loader.traversal_order.indices = SET1 together, it seems to work. Thanks!

Happy that it worked for you. Enjoy FFCV!

I found it didn't work when I change the indices of a dataloader using
       mask = np.arange(1000)         dataloader.indices = mask
It still gives the original dataloader with 50,000 training data points in make_dataloaders()
https://github.com/libffcv/ffcv/blob/bf07b5c6ac7f4cce788348d1718c15e78da8ae9d/examples/cifar/train_cifar.py#L69

when I changed train_loader.indices = SET1 and train_loader.traversal_order.indices = SET1 together, it seems to work (the number of batches in an epoch change)

Thank you, @rraju1! It seems to work here too.


Add validation in event handler

TL;DR
Currently in pmap, the validation totally relies on github workflow https://github.com/abcxyz/pmap/blob/main/.github/workflows/check.yml#L51. It means if users fail to configure the workflow, our pmap backend won't be able to catch the bad data.  For example, if user send policy data to pmap mapping server, pmap mapping server will still store the policy data into mapping table. CAIS asset inventory  processor validation won't work here because policy data won't go through the CAIS processor. We should add validation logic in the pmap backend.
Detailed design
No response
Alternatives considered
No response
Additional information
No response

Closed it as I can't reproduce it.


Android crash due to SYS_SECCOMP (illegal syscalls)

We've just started noticing crashes on Android x86_64 devices. We've never seen these before likely because there are hardly any such devices on the market. The device in question was Pixle Slate, but the following can be reproduced on the x86_64 simulator with Android 9 (and probably Android 8).
REPRO

Create AVD with Android 9 and x86_64
Run any code using libuv
Observe crash with the following stack trace

Build fingerprint: 'google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:user/release-keys'
Revision: '0'
ABI: 'x86_64'
pid: 6144, tid: 6190, name: Thread-8  >>> XXX.XXX.XXX <<<
signal 31 (SIGSYS), code 1 (SYS_SECCOMP), fault addr --------
Cause: seccomp prevented call to disallowed x86_64 system call 232

N.B. I can't provide our code but any libuv usage should crash.
The cause of this is due to new secuiryt features in android/linux:  https://android-developers.googleblog.com/2017/07/seccomp-filter-in-android-o.html
The offending code is uv__epoll_wait which makes the sys call syscall(__NR_epoll_wait, epfd, events, nevents, timeout);. The Android bionic source code shows the allowed list of syscalls here:
https://android.googlesource.com/platform/bionic.git/+/master/libc/SYSCALLS.TXT
Note that epoll_wait is not whitelisted, only epoll_pwait is. Android libc actually defines epoll_wait in terms of epoll_pwait:
int epoll_wait(int fd, struct epoll_event* events, int max_events, int timeout) {
  return epoll_pwait64(fd, events, max_events, timeout, nullptr);
}

(see https://android.googlesource.com/platform/bionic/+/master/libc/bionic/sys_epoll.cpp)
This is why users using libc rather than making direct syscalls will not be affected.
I believe the fix (at least for Android) is simply to use __NR_epoll_pwait rather than __NR_epoll_wait.
The only thing I'm not clear on here is why we don't see crashes on e.g. arm64 devices running Android 9. In theory this code should just not work on Android. I can only chalk that down to either seccomp not being enabled yet on those devices or Google have a bug in their implementation.
Note that the libuv code is also technically using the following illegal APIs:

__NR_epoll_create
__NR_inotify_init

However, these are only called as fallbacks if __NR_epoll_create1 and __NR_inotify_init1 fail for any reason. I might suggest also removing those fallbacks on Android as failures in those calls will crash Android apps.

Libuv 1.23.1 and up no longer make direct epoll syscalls.
(It still makes other direct syscalls so YMMV.)

Ah, my bad. I thought I had master checked out but I was actually on some tagged 1.x branch.

Can this be closed then?

Yes, I assume so. It'll take us a while to update to latest version but I'd assume that if you're not making direct sys calls then there'll be no issue 👍


Feature/company crud

closes #5
closes #9
Overview

Configure lint and coverage report
CRUD Company with users

Next step
#10

Is it possible to separate the subjects? I mean one PR for linter and one for Company crud? :)

Is it possible to separate the subjects? I mean one PR for linter and one for Company crud? :)

It is. But it's not necessary here.


KAFKA-7918: Inline generic parameters Pt. III: in-memory window store

Third (and final) PR in series to inline the generic parameters of the following bytes stores:
[Pt. I] InMemoryKeyValueStore
[Pt. II] RocksDBWindowStore
[Pt. II] RocksDBSessionStore
[Pt. II] MemoryLRUCache
[Pt. II] MemoryNavigableLRUCache
[x] InMemoryWindowStore
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@guozhangwang @vvcephei @mjsax @bbejeck

JDK 11 failed on kafka.api.PlaintextConsumerTest.testAutoCommitIntercept
JDK 8 failed on kafka.api.SaslSslAdminClientIntegrationTest.testMinimumRequestTimeouts
Failures unrelated. Retest this, please

Java 11 passed, Java 8 failure unrelated
kafka.api.AuthorizerIntegrationTest.testSendOffsetsWithNoConsumerGroupWriteAccess
kafka.api.AuthorizerIntegrationTest.testDeleteGroupApiWithDeleteGroupAcl

retest this please

Java 11 passed,
Java 8 failed results already cleaned up
retest this please

The jenkins machine seems failed. I've ran the tests locally and they pass, will merge now.


Refactor to UIImageView

@wangjwchn hey! I'm back 😎. So, I have a good news for us:
Testing on (iPhone 6s, 11.1, Xcode 9.1 (9B55))
Old implementation: https://puu.sh/ynWBq/affb71ab10.png
After small changes: https://puu.sh/ynWBT/8ecccbad27.png
Resource: https://wangjwchn.github.io/image/view.gif
@wangjwchn Could you test as you wish and review code ?
Refactor:

Change layer contects setting to image setting;
Remove decode image function;
Time fired function;
Add types to variables.

@Alex601t have you tested this?
I would look at the code again and remove force unwraps, rename the method image(from:at) to something different, and would check the prepareCache function since the calling to that function is not correct.
Also, I would update the readme and the demo comments up in the code to match the changes.


Fix issues discovered by ShellCheck

ShellCheck is happy now!  I didn't break anything... 🤞  😅

These scripts have since been deprecated


Replace mock data in PartyStatsContainer with subscription to the Data Model.

The puzzle 1-a6ea5ab6 from #1 has to be resolved:
https://github.com/codeforthailand/election-live/blob/843128114e9e7345e8a2d3156089c89d841bb352/src/pages/index.js#L31-L32
The puzzle was created by Thai Pangsakulyanont on 20-Mar-19.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

The puzzle 1-a6ea5ab6 has disappeared from the source code, that's why I closed this issue.

@0pdd 3 puzzles #58, #59, #60 are still not solved.

@0pdd all 3 puzzles are solved here: #58, #59, #60.


App crash when create new account

When I created a new account, the app crashed!

Can you copy and paste the logs?

app crash when sync BTC too
2024-07-16 23:16:06.042 [INF] DCR: Discovering used addresses for 1 account(s)
2024-07-16 23:16:06.125 [INF] DCR: Account 0 next child indexes: external:2 internal:23
2024-07-16 23:16:06.134 [INF] DCR: Updating DB with discovered addresses...
2024-07-16 23:16:06.183 [INF] DCR: Finished address discovery
2024-07-16 23:16:06.183 [INF] SHWL: Set discovered accounts = true for wallet 10
2024-07-16 23:16:06.198 [INF] DCR: Loading active addresses and unspent outputs...
2024-07-16 23:16:06.203 [INF] DCR: Registered for transaction notifications for 65 HD address(es)
2024-07-16 23:16:06.206 [INF] DCR: Registered for transaction notifications for all relevant outputs
2024-07-16 23:16:06.206 [INF] DCR: Rescanning block range [1432803, 1432809]...
2024-07-16 23:16:06.222 [INF] DCR: Rescan complete
2024-07-16 23:16:06.222 [INF] DCR: [10] Indexing transactions start height: 1432760, end height: 1432809
2024-07-16 23:16:06.222 [INF] DCR: [10] Transaction index finished at 1432809, 16 transaction(s) indexed in total
2024-07-16 23:16:12.302 [INF] BTC: Fetching block height=2683623 hash=000000000000289b321c1a406a0a24c01dcdd0d50d3e6b8a1b4d320f3c7d8133
2024-07-16 23:16:12.754 [INF] BTC: Recovered addresses from blocks 2681650-2683649
2024-07-16 23:16:12.833 [INF] BTC: Scanning 2000 blocks for recoverable addresses
2024-07-16 23:16:12.837 [INF] BTC: Fetching block height=2683659 hash=00000000000018147d0dc0d439d874721d2ee4ff0b0d97f8fd6ec941976d03cb
2024-07-16 23:16:12.883 [INF] BTC: Canceling sync. May take a while for sync to fully cancel.
2024-07-16 23:16:12.883 [INF] BTC: (b1) SPV wallet closed
2024-07-16 23:16:12.987 [INF] BTC: Stopping (b1) wallet and its neutrino interface
2024-07-16 23:16:13.508 [INF] BTC: Synchronizing wallet (b1) with network...
2024-07-16 23:16:13.508 [ERR] BTC: Neutrino rescan ended with error: rescan block subscription was canceled
2024-07-16 23:16:14.510 [INF] BTC: RECOVERY MODE ENABLED -- rescanning for used addresses with recovery_window=200
2024-07-16 23:16:14.514 [INF] BTC: Seed birthday surpassed, starting recovery of wallet from height=2685650 hash=0000000000002c892e340af1e7b4249837679474afe297536fed50f4d2a146a9 with recovery-window=200
panic: unable to synchronize wallet to chain: unable to perform wallet recovery: failed to store sync information<PHONE_NUMBER>002c892e340af1e7b4249837679474afe297536fed50f4d2a146a9: failed to fetch block hash for height 2685649: block not found

goroutine 7052 [running]:
github.com/btcsuite/btcwallet/wallet.(*Wallet).handleChainNotifications(0x14000cb4c60)
        /Users/justindo/go/pkg/mod/github.com/btcsuite/btcwallet@v0.16.9/wallet/chainntfns.go:117 +0xcf0
created by github.com/btcsuite/btcwallet/wallet.(*Wallet).SynchronizeRPC in goroutine 7035
        /Users/justindo/go/pkg/mod/github.com/btcsuite/btcwallet@v0.16.9/wallet/wallet.go:213 +0x388

Can you copy and paste the logs?

Here is some logs:
2024-07-19 17:23:01.816 [INF] DCR: Rescanning block range [928001, 930000]...
2024-07-19 17:23:01.839 [INF] DCR: Rescanning block range [930001, 932000]...
2024-07-19 17:23:01.865 [INF] DCR: Rescanning block range [932001, 934000]...
2024-07-19 17:23:02.342 [INF] DCR: Rescanning block range [934001, 936000]...
2024-07-19 17:23:02.378 [INF] DCR: Rescanning block range [936001, 938000]...
2024-07-19 17:23:02.404 [INF] DCR: Rescanning block range [938001, 940000]...
2024-07-19 17:23:02.429 [INF] DCR: Rescanning block range [940001, 942000]...
2024-07-19 17:23:02.454 [INF] DCR: Rescanning block range [942001, 944000]...
2024-07-19 17:23:02.742 [INF] DCR: Rescanning block range [944001, 946000]...
2024-07-19 17:23:03.013 [INF] DCR: Rescanning block range [946001, 948000]...
2024-07-19 17:23:03.049 [INF] BTC: Fetching block height=1879778 hash=0000000000038802c6e58a523f41750ec0bd0b75b15dfda9a24a4a47f5896f44
2024-07-19 17:23:03.295 [INF] DCR: Rescanning block range [948001, 950000]...
2024-07-19 17:23:03.307 [INF] BTC: Recovered addresses from blocks 1878001-1880000
2024-07-19 17:23:03.323 [INF] DCR: Rescanning block range [950001, 952000]...
2024-07-19 17:23:03.326 [INF] BTC: The wallet has been locked
panic: unable to synchronize wallet to chain: unable to perform wallet recovery: recovery: forced shutdown
goroutine 257 [running]:
github.com/btcsuite/btcwallet/wallet.(*Wallet).handleChainNotifications(0x14000938a20)
/Volumes/Storages/Projects/cryptopower/vendor/github.com/btcsuite/btcwallet/wallet/chainntfns.go:117 +0xd84
created by github.com/btcsuite/btcwallet/wallet.(*Wallet).SynchronizeRPC in goroutine 192
/Volumes/Storages/Projects/cryptopower/vendor/github.com/btcsuite/btcwallet/wallet/wallet.go:213 +0x364
exit status 2

This crash should be resolved with the btc upgrade PR

It seems to have been resolved, i don't see it occurring again


Conflict resolver not being passed to Fetch/Store-ValueCommand [JIRA: CLIENTS-565]

Conflict resolver is not being passed from the builder to the command, so it's never being called.
https://github.com/basho/riak-go-client/blob/437fcfc355517ccfce3093232a6967df70e1fcf3/kv_commands.go#L240
Should be:
return &FetchValueCommand{protobuf: builder.protobuf, resolver: builder.resolver}, nil

https://github.com/basho/riak-go-client/blob/437fcfc355517ccfce3093232a6967df70e1fcf3/kv_commands.go#L510
Should be:
return &StoreValueCommand{value: builder.value, protobuf: builder.protobuf, resolver: builder.resolver}, nil

I was wondering also if it would be possible to make the call to the resolver only when there are at least 2 values, as it seems that often -when there are no siblings- the call might not be necessary.
https://github.com/basho/riak-go-client/blob/437fcfc355517ccfce3093232a6967df70e1fcf3/kv_commands.go#L76-L78
https://github.com/basho/riak-go-client/blob/437fcfc355517ccfce3093232a6967df70e1fcf3/kv_commands.go#L311-L313

Hi Sergio,
Thanks for finding that, I'll get it resolved soon.
I checked the Node.js client and Java client, and the resolver, if present, is always called, no matter the result count. I'll leave the behavior the same in the Go client to remain consistent.

Hi Luke,
I understand, that makes sense, thanks.


Maven Central Mirror : Default Restricted File Types List Prevents Some Artifacts from being used in Gradle

What happened?
I've been working for a few days to resolve an issue that began when updating reposilite and with your knowledge here I believe it will likely be an easy resolution.
About a week ago, we began experiencing issues with certain library dependencies getting downloaded from our reposilite repository that mirrors maven central. The dependencies would get downloaded fine from the gradle log, but intellij and gradle would fail to build and show errors on any import for classes in the dependency. The most obvious dependencies were:
// All ktor versions 2.1.0 through 2.3.2 failed (we did not try any older, but all failures were consistent).
implementation("io.ktor:ktor-client-core:$ktorVersion")
implementation("io.ktor:ktor-client-cio:$ktorVersion")
implementation("io.ktor:ktor-client-content-negotiation:$ktorVersion")
implementation("io.ktor:ktor-serialization-kotlinx-json:$ktorVersion")

We first narrowed it down by removing all other repositories and finally figured out that swapping the mirror repo out for maven central directly made the difference. This made us check the Reposilite configuration for the repo.
After a lot more experimentation we found that the deciding factor was the "allowed extensions" configurations. We had never modified this from the defaults, so here is what it was before we modified it:

We removed all of the extensions to "Allow all" as the hint suggests and the dependencies immediately started working correctly.

Please let me know if there is anything else we can test to help resolve this issue.
Reposilite version
3.4.5
Relevant log output
No response

You should be able to see all request to your Reposilite instance in CLI in debug mode (command: level debug) or in the latest.log file. Each error response should also contain an error message, it might be helpful to investigate which extensions were dropped.
I've checked out some of the files of these Ktor libs:

https://repo1.maven.org/maven2/io/ktor/ktor-client-cio-jvm/2.3.2/
https://repo1.maven.org/maven2/io/ktor/ktor-client-core/2.3.2/
https://repo1.maven.org/maven2/io/ktor/ktor-client-content-negotiation/2.3.2/

It looks like our default cfg misses .module extension - it might be the issue. I also see they have some extra files called kotlin-tooling-metadata.json, but I don't think it's the cause of this problem.

I'll add all the extensions back and test it today sometime. I'll also try out debug mode on reposilite to find any others if there is still an issue. Kotlin has a lot of multiplatform related capabilities which his opening the door for more types of files to be stored for multiplatform gradle builds, etc. and ktor does support many platfroms so I could absolutely see this causing some difference from a typical java project.
I'll also see if I can get a list of the file types output for multiplatform kotlin release artifacts as well.

Sure, let me know if you'll find out more extensions that we should reconsider as defaults. To be fair, I didn't try to publish KMP projects to Reposilite yet.

We are trying that out for you and happy to do so.

It seems to be resolved ;)


Muliplication of observations

Proposed changes
Add multiplications, better timestamps and tweak params.
Close #58

Does this resolve #20 and #58 ?

Oh I forgot the other one. Yes it does.


Plugin doesn't work for class in astro

Dependency information
"@prettier/sync": "^0.3.0"
"prettier": "^3.1.1"
"prettier-plugin-astro": "^0.12.3"
"prettier-plugin-classnames": "^0.3.1"
"prettier-plugin-merge": "^0.4.0"
"prettier-plugin-tailwindcss": "^0.5.10"

my .prettierrc file
{
  "tailwindConfig": "./tailwind.config.ts",
  "tabWidth": 2,
  "useTabs": false,
  "singleQuote": false,
  "bracketSpacing": true,
  "plugins": [
    "prettier-plugin-astro",
    "prettier-plugin-tailwindcss",
    "prettier-plugin-classnames",
    "prettier-plugin-merge"
  ],
  "customAttributes": ["className", "class"],
  "customFunctions": ["classNames"]
}

Steps to reproduce
try to reformat .astro file with those configurations
The current behavior
nothing happened
The expected behavior
create a line break when class or className attribute is too long

This plugin and prettier-plugin-merge are currently only supported for JavaScript, TypeScript, and Vue code.
I am also planning to support Astro, but it will take some time to complete. :pray:

Thanks :)

https://github.com/withastro/astro/issues/4029
It seems like some problems still exist with multiline classes in Astro. A workaround is to use curly brackets:
<MyComponent class={`some-tailwind-class another-tailwind-class more-tw-madness
   once-again-tailwind-class more-tailwind-class absolute inset-0
   etc-tailwind-class foo baz bar`}
/>

@BrandenXia Finally released as v0.6.0! Check it out! :smile:


Grouped Curtains

Hi! I can't see a way to control a set of grouped curtains, even though it does report on grouped status. Is it possible to add this?

Thank you for the feedback.
As I only have one actual device, I have only conducted verification for individual Curtain devices.
The API specifications only define operations for individual devices, and I am not aware of how to control specific devices within a group or what commands to send when they are grouped.
I have a feeling that the first parameter (Index) in the setPosition command might indicate something...
Alternatively, it might be possible to achieve control by simply sending commands to the master and configured devices.
Could you try experimenting with various commands in a grouped state and see if you can find the right input to make it work?
I apologize for not being able to provide more assistance.

I tried various commands with these parameters, but didn't have any luck...


Parse.Cloud.httpRequest fails if body is not a string

Hey Fosco!
In one of my objects, I had a afterSave hook where I do a http post to another Parse app.
However, the body parameter passed is an object, not a string, and this causes the app to throw an exception:
_http_outgoing.js:430
    throw new TypeError('first argument must be a string or Buffer');
    ^

I looked into it, and locally was able to solve it in a easy way. When defining Parse.Cloud.httpRequest in index.js, line 145 I added:
if (typeof options.body === 'object') {
      options.body = JSON.stringify(options.body);
}

It simply converts the object to a string if it's necessary. Is this a correct fix? If so, I can open a PR.

Please open the PR.  Yeah, it looks like the 'request' module we use only supports a string body.

But it works fine when running it in the Parse hosted Cloud Code

Because Parse hosted cloud code doesn't use node or the 'request' module... It's a v8 engine with some node-like properties, but mostly custom built.

I see... Ok, will add it soon. Thanks!

Small JS question: if options.body was not defined, is this code I suggested still safe?

Should be, yeah.


Add a file to track latest release version

add a file to track latest release version
as mentioned in https://github.com/redhat-developer/odo/pull/593#issuecomment-409173415

@syamgk , as per @kadel suggestion, look into scripts/bump-version.sh
we should not manually edit this kind of version files

@kadel  @surajnarwade  as per https://github.com/redhat-developer/odo/pull/593#issuecomment-409173415
what I understood is that bump-version.sh will change the version before binary becomes available
Is that correct?

Good point @syamgk  you are right.
We will have to update this as the last step in the release process. Can you please update development.md docs and add this as the step in the release process? It might be also worth adding a message/reminder to the bump-version.sh script, I believe that something similar is already there regarding homebrew package.

Ah, I didn't notice that it is  scripts/.latest-release. Shouldn't it be better if this would be in root directory (.latest-release)?

2 LGTM's. Merging.


feat(chip): add element and clear button

add element and clear button

TASK:
Description, Motivation and Context

Types of changes

[x] ✨ New feature (non-breaking change which adds functionality)

Screenshots - Animations

@andresin87 do you know why there is no test env deployed for your PRs about the Chip ?


Title of the download page is broken

Hello,
The title page of the current online download page is not displayed at all. This bug appears from this commit.
I looked at the default layout and at the download page content, but I was not able to find any problem.
I am not familiar with ruby and nanoc, so I was not able to create a PR to fix it directly, sorry.

I pushed the site of off a wip/nanoc branch. The issues is now fixed by @Saisse in #210 and merged to master.

Thanks @eed3si9n :+1:


[Paid service]: I am willing to pay for anyone who can provide an instagram registration agreement.

I need to be able to complete the registration and skip the registration agreement of risk control to a certain extent

my tg @rabbitTokens


Hot reloading enabled by default and broken on master

I just created a new app by publishing what's on master to Sinopia locally, then did React Native init:
On iOS, HMR is enabled by default and consistently results in a redbox.
When disabling HMR, Reload JS works and the packager shows one request:
 <START> request:/index.ios.bundle?platform=ios&dev=true
 <END>   request:/index.ios.bundle?platform=ios&dev=true (78ms)

When enabling HMR via the menu:
The app makes a request to the packager, END is not logged and I consistently get a redbox
<START> request:/index.ios.bundle?platform=ios&dev=true&hot=true

I get the same on Android when enabling HMR, although HMR is disabled by default:

HMR setting is persisted on the simulator. If you enable it and kill-restart an app when it restarts it will still be enabled. This is the same behavior we currently have for live reload.

Makes sense, thank @martinbigio! Any idea why we get the error on master when enabling it?

Fix landed on https://github.com/facebook/react-native/commit/3a1634ec7b0dfa2b5d8890118565006d845e2039
I'll release a new version of the preset tomorrow morning :)


Incorrect indentation

Lines 39 in VPNDetector
have incorrect indentation causing the script to fail.

Resolved by adding a space on line 39

Issue resolved

Issue resolved


audit: new formulae should not require patches.

We may under some circumstances accept these anyway but it's better to nudge people into the right behaviours on local audits.

Should this also check for patch do not just END?

@ilovezfs My thought was that a patch do is more likely to be one that's actually upstream so might actually be OK. Thoughts?

That still seems like it should require someone here to take a second look since accepting new formulae with patches, even reported ones, is still sketchy. Also, the patch URL could be from a stale PR no one upstream ever acknowledged, a personal gist, an attachment to a bug report buried on Sourceforge, etc.

Changed to check for any patches to any spec.

Thanks!


There should be a warning message when the user is clicking on the apply button without selecting any action.

Steps To reproduce--
1. log in as a user
2. Click on the orders tab.
3. Click on the apply button without selecting any action.
Actual result: There is no warning message when the user is clicking on the apply button without selecting any action.. Please find the below screenshot.
Expected result: There should be a warning message when the user is clicking on the apply button without selecting any action.

@jitendra-webkul
Now, this issue is fixed. so I am closing this issue.
And I have attached a screenshot Here, for future reference.