[BUG] quantized_matmul gives non reproducible NaNs

Describe the bug
I have a model using quantized_matmuls. It works fine in inference. And it works in when I fine-tune the lora weights in that model in the first iteration, but starting from the second iteration, loss becomes NaN.
I cache intermediate values and set a breakpoint after the optimizer update. I found the output of the first quantuzed_matmul is NaN (the second operation in my model). Now I start trying different stuff in this pdb session:

I feed the same inputs (including parameters in the QuantizedLinear module) to mx.quantized_matmul using cpu stream, it works fine.
I dump the inputs to file and load bad, and run it on gpu, it works fine as well.
If I still use the same inputs and run quantized_linear on gpu again, NaNs still.

To Reproduce
It's hard to reproduce since the project is a bit complicated.. And as mentioned previously, it cannot be reproduced with standalone mx.quantized_matmul with saved tensors. However, it is reproducible consistently with my training script.
I will appreciate any general suggestions or tips to further debug this issue.
For example, this symptom seems to suggest that there is some state/cache in the gpu pipeline? If so, maybe there is a way to "force reset" any possible state?
Thank you in advance!
Expected behavior
Identical or similar result as the cpu quantized_matmul.
Desktop (please complete the following information):

OS Version: [MacOS 14.1]
Version [0.8.0]

Additional context
I'm using float16 for inputs and non-quantized parameters.

Coming up with a simple way to reproduce the issue will be the most useful thing to do to debug this both for yourself and for us to be able to help you.

I dump the inputs to file and load bad, and run it on gpu, it works fine as well.
If I still use the same inputs and run quantized_linear on gpu again, NaNs still.

That sounds bad, it should not be the case that any GPU op depends on the state of the machine other than the input values themselves.
Can you give more input on the op you are running:

op name
arguments (bits, group size, etc)

it is reproducible consistently with my training script.

Does it show up early? Can you share even a subset of the script that reproduces it?

Hi Awni, thanks for the prompt reply!
I tried to dig deep to carve a reproducible part of the script. I am able to get the following script. I do not know if it reproduces exactly the same issue, but at least:
(1) GPU consistently generate some NaNs, while CPU matches the no-NaN results from dequantized version.
(2) GPU output is all NaNs in some cases, and partially NaNs in other cases.
import mlx.core as mx

mx.random.seed(0)
dtype = mx.float16

weight = mx.random.randint(46077963, 46077963 + 100000, (1, 80), dtype=dtype)
scales = mx.array([[0.00012, 0.00016, 0.00014, 0.00015, 0.00014]], dtype=dtype)
scales = mx.array([[-0.014, -0.021, -0.017, -0.017, -0.018]], dtype=dtype)
x = mx.array([[-1.25195]], dtype=dtype)

y_gpu = mx.quantized_matmul(
        x,
        weight,
        scales=scales,
        biases=biases,
        transpose=False,
        group_size=64,
        bits=8,
        stream=mx.gpu)

y_cpu = mx.quantized_matmul(
        x,
        weight,
        scales=scales,
        biases=biases,
        transpose=False,
        group_size=64,
        bits=8,
        stream=mx.cpu)

print(y_gpu)
# some or all NaNs
print(y_cpu)
# meaningful result

weight_deq = mx.dequantize(weight, scales, biases, 64, 8)
y_deq = x @ weight_deq
print(y_deq)
# meaningful result
print(mx.max(mx.abs(y_cpu - y_deq)))
# ~ 1.5e-5

I have not been able to reproduce the "same pdb session, cpu / gpu with loaded variables work, gpu with original variables fails" scenario. But this script does suggest some issue within the quantized_matmul op? I'm hoping they share the same underlying root cause.
Also, while dumping these scale/bias numbers, I do notice the scales are fairly small for float16 (although they come from real models). Could that be an issue? My 2 cents are I can multiply the scales with a big number to put it into a reasonable range (which requires some tuning), or simply use float32 (which could lead to more computation). But I don't know if these are good solutions to this problem, or should I try to do something ad-hoc like this.

The code you sent doesn't run for me... quantized_matmul needs uint32 weights.  Could you double check it please?

Sorry for the copy paste. Try the updated version below:
import mlx.core as mx

mx.random.seed(0)
dtype = mx.float16

weight = mx.random.randint(46077963, 46077963 + 100000, (1, 80), dtype=mx.uint32)
scales = mx.array([[0.00012, 0.00016, 0.00014, 0.00015, 0.00014]], dtype=dtype)
biases = mx.array([[-0.014, -0.021, -0.017, -0.017, -0.018]], dtype=dtype)
x = mx.array([[-1.25195]], dtype=dtype)

y_gpu = mx.quantized_matmul(
        x,
        weight,
        scales=scales,
        biases=biases,
        transpose=False,
        group_size=64,
        bits=8,
        stream=mx.gpu)

y_cpu = mx.quantized_matmul(
        x,
        weight,
        scales=scales,
        biases=biases,
        transpose=False,
        group_size=64,
        bits=8,
        stream=mx.cpu)

print(y_gpu)
# some or all NaNs
print(y_cpu)
# meaningful result

weight_deq = mx.dequantize(weight, scales, biases, 64, 8)
y_deq = x @ weight_deq
print(y_deq)
print(mx.max(mx.abs(y_cpu - y_deq)))
# ~ 1.5e-5

Ok it runs now and I see the same issue.

Thanks! Let me know if there's anything I can do to help debug.


Semantic Release integration

As a developer, it would be helpful to not have any manual processes for releasing CommunityUI.
For this, we will use semantic-release.
The goal here is to make sure that whenever we merge our branch with main, a release takes place automatically, and updates to the proper version of the package. The semantic-release utility can be configured to do this automatically.
Acceptance Criteria:

semantic release integrated
configured to release to our project in npm
test release ran to be sure

I'd love to tackle this issue... I'm not able to assign myself, just wanted to flag it so we don't have overlap.

Yes absolutely. I'll add you to the org Soni can assign it tonyou

Integration was spot on.
We are on v1.0.0 now (lol oops), so now, we're switching to the develop branch until any churn that exists dies down, and then we will start merging to main again.


Learning to Love DDD - a Tale of Two Products

https://www.youtube.com/watch?v=QqvYIIobZ6A

Domain language is a significant part of DDD
Lessons learned

Created and Updated are almost never part of the domain language, and shouldn't be used for events
Ubiquitous language should be co-created with both the customer and the developer
Continuous Integration for Conversations


vision.py (ep 5)

File "main.py", line 31, in
points = vision_limestone.find(screenshot, 0.5, 'rectangles')
File "C:\Users\x\Documents\x\lx\vision.py", line 29, in find
result = cv.matchTemplate(haystack_img, self.needle_img, self.method)
cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-52oirelq\opencv\modules\imgproc\src\templmatch.cpp:1163: error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2 in function 'cv::matchTemplate'
Not sure what's causing this. My own version of the code causes it and then I tried directly copying the file structure and this still happens

This error means that OpenCV doesn't like one of the images you've given to matchTemplate(). Either it's not a valid image, or OpenCV just doesn't like how your image is formatted. So the problem could be anywhere along the chain from when you first capture or load the image, to where you give it to matchTemplate().
Most likely this will be your haystack_img that it has a problem with. When I was working on this project, I saw this error most often when I was calling find() before the window capture had successfully taken a screenshot. This usually happens when the program is first started.
So the first thing to try is adding something like:
if screenshot is None:
        continue

to your main while loop, so it'll only call find() once it has a screenshot.
If that doesn't solve it, I would try saving your screenshot image to a file to confirm it looks how you expect it to. If it looks fine, then there's some sort of conversion you'll need to apply/remove from the image. You can see in WindowCapture get_screenshot() where I've left a comment about a similar error and needing to do a conversion to remove the alpha channel.


Asset imports in ESM mode are generated with __dirname variable

Describe the bug
When importing an asset path (in this case, it's a Tray icon that'll be used with the Tray class from electron), electron-vite will generate a path using the __dirname variable, which is not available in a full ESM context.
Here's my TS import:
import trayIconPath from '../../../assets/logoTemplate.png?asset';

And here's the generated code in main.js:
const trayIconPath = join(__dirname, "./chunks/logoTemplate-IqcVkt31.png");

Which leads to the following error at runtime:
App threw an error during load
ReferenceError: __dirname is not defined in ES module scope
This file is being treated as an ES module because it has a '.js' file extension and '/Users/**/Desktop/**/**/application-desktop/package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.
    at file:///Users/**/Desktop/**/**/application-desktop/dist/main/main.js:32:27
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

Here's my electron.vite.config.ts:
export default defineConfig({
  main: {
    build: {
      rollupOptions: {
        input: {
          main: resolve(__dirname, "src/main.ts"),
          pipeline_process: resolve(__dirname, "src/pipeline_process.ts"),
        },
        output: {
          format: "es"
        }
      },
      outDir: 'dist/main',
      publicDir: 'assets',
    },
  },
  preload: {
    build: {
      rollupOptions: {
        input: {
          preload: resolve(__dirname, "src/preload.ts"),
        },
        output: {
          format: "es"
        }
      },
      outDir: 'dist/preload',
    },
  },
});

Electron-Vite Version
2.0.0
Electron Version
28.0.0
Vite Version
5.0.12
Validations

[X] Follow the Code of Conduct.
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

@nerdynomadd It seems that electron-vite 2 is not used. When esm is enabled in this way, the file suffix will be .mjs. In addition, your config file is wrong, it should be main.publicDir. Upgrade your electron-vite or use npm create @quick-start/electron@latest to create a project to test.

Is there any more problem？

Hi @alex8088, regarding the statement When esm is enabled in this way, the file suffix will be .mjs.
I also enable the ESM same way: using  build > output > format > 'es' and I have type="module" in my package.json but I get .js as file extension instead of .mjs for main process, but I get .mjs for preloader process.
Is this intended? And is this best-practice to enable native ESM this way?

I have the same problem as @undergroundwires. The preload is .mjs and uses imports, which isn't currently allowed in a preload. Not sure what settings to use to have it build a cjs for preload.


Add badges and some code review details

Add to the main README - however note that since we only have PR based
tests right now, if a PR fails, our badges will go red which isn't
really fair to us or the submitter - but we can improve this in #52.
Added a go report card link too - looks like our linting could use some
improvement but we should see how #19 goes (maybe switching out
kubebuilder) before we start following up on all of these.
Looks like:

Also adding some details about code reviews after taking with @tejal29
and @aaron-prindle, specifically leaving the reviews up for a bit longer
than we have been to try to give more folks a chance to comment. Also
added a link to some useful Go coding standards.
Fixed a couple typos too.

p.s.

/lgtm

/lgtm


Adding a condition to not run list-changed-file workflow for PRs from forks.

Although we'd like to list the changed files in PRs from forked repos, GitHub can't use security tokens across forks when posting the comment with the changed file output. Until we can find a solution to this, we should just turn off this workflow in PRs from forked repos.

This change is

TFTR, Jesse. Yeah, a big pain with these integrations comes from the inability to test before merging.


A question about data type

I just run train.py without changing. The error report shows information below.
Traceback (most recent call last):
File "D:/research/noofficial_cyclegan/cyclegan-pytorch-master/train.py", line 32, in
from dataset import CUDAPrefetcher, ImageDataset
File "D:\research\noofficial_cyclegan\cyclegan-pytorch-master\dataset.py", line 25, in
from imgproc import image_to_tensor, random_rotate, random_horizontally_flip, random_vertically_flip
File "D:\research\noofficial_cyclegan\cyclegan-pytorch-master\imgproc.py", line 150, in
center: tuple[int, int] = None,
TypeError: 'type' object is not subscriptable
Is my python version wrong? Thank you for answering.

I already solve this problem by updating python to 3.10

Fixed.


performance of SECOND on Waymo

I trained second (8GPUs) with config from pcdet  but got much worse performance than the results reported in README.  Will the version of spconv  be the reason?  I  use spconv 2 in my experiment and here is the results of 30 epoch.

Have you generated the new data info with pcdet v0.5+?

@sshaoshuai Not yet, I created the data with pcdet 0.3, so the new version of data can improve performance?

@sshaoshuai, I generated the new waymo data and train SECOND (cfgs/waymo_models/second.yaml) again with the latest code,  but I met a strange bug: every thing was good at the beginning, but after some epochs the loss became nan and finally the evaluated mAP is zero. I did not change any code, did you meet this problem before?

@pigtigger Please just re-run the experiment again. This is a known issue, but will not take place frequently.

@jihanyang Thank you, I will have a try, btw, do you know why this problem occurs frequently? I meet this nan bug in my own model but can not figure out what causes it.

Actually, we are still not clear about the reason, since this is problem not easy to trace and hard to reproduce. From my experience, even the same code, this error could take place frequently in some machines while not appear at all in other machines. If you have any suggestion to solve this, feel free to let us know.

Thank you :) @jihanyang , re-run the experiment does work, I will share my solution if find it.


Feature: Added open and close conditions on toggle

Hi there,
Thanks for the great plugin.
I wanted to have a more granular control on whether an element could be opened or closed. As far as I know the collapse attribute will make an accordion item unclosable, but it would apply on all elements.
new Accordion('.accordion-container', {
	openCondition: (element) => {
		const condition = !!element.dataset.openable;
		return condition;
	},
	closeCondition: (element) => {
		const condition = someRandomFunction(element);
		return condition;
	},
});

Hi @yahyafati
I really appreciate your contribution, but I don't think I will include these options in the package.


satellite_gps_center.npy files

Hello, thank you for your work.
It seems that some files are missing: "satellite_gps_center.npy and groundview_satellite_pair_18.npy"
groundview_satellite_pair_18.npy is generated from satellite_gps_center.npy, so satellite_gps_center.npy is required to run the code.
Could you upload the satellite_gps_center.npy or provide links to download?
Thank you.

Please find the required file at the following link:
https://drive.google.com/drive/folders/1DML0ryEERCs1uiM501I-MJxSCjXaFFdS?usp=sharing
Additionally, 'groundview_satellite_pair.npy' is generated using 'kitti_data_process/kitti_cross_view_pair.py,' and you can find the code for 'satellite_gps_center.npy' generation for Ford-CVL in 'ford_data_process/get_gps_coverage.py.'

Thanks for your kind explanation. I ran the code successfully !


[WIP] Added graylog as a logging target

Proposed changes

Added graylog as a logging target and its gelf format.  Allow for configuring UDP, TCP, and TLS protocols
Types of changes

New feature (non-breaking change which adds functionality)

Linked Issues

https://github.com/rancher/rancher/issues/14062
Further comments

Related backend PR: https://github.com/rancher/rancher/pull/24911

Closing PR due to age. Feel free to reopen if backend changes are merged. With the new Dashboard UI these changes may be better going into the new logging functionality there as well.


Create composed selector + createPathSelector

@eaglus : filing pull requests with no explanation isn't really helpful.  It looks like you're trying to add a significant new feature.  Can you please do a complete writeup on what this feature is, why it's being created, justification for actually adding it to the library, and add documentation on this to the PR?

@eaglus : whatever this is, it looks kinda interesting.  I really would appreciate it if you could add an explanation of what this is supposed to do.

Hi Mark! I will update Readme, and will make another PR
сб, 1 сент. 2018 г. в 17:46, Mark Erikson<EMAIL_ADDRESS>
@eaglus https://github.com/eaglus : whatever this is, it looks kinda
interesting. I really would appreciate it if you could add an explanation
of what this is supposed to do.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reduxjs/reselect/pull/367#issuecomment-417864380, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AK3Swqc3O5QH0MaFvBGLg1imqziq1Bxwks5uWp2ogaJpZM4WU9-1
.


EOS RPC push_transaction

Why I always do something wrong with the push_transaction operation, who can tell how to fill in these parameters，and what is mean?
curl  http://localhost:8888/v1/chain/push_transaction -X POST -d '{"ref_block_num":"100","ref_block_prefix":"137469861","expiration":"2017-09-25T06:28:49","scope":["initb","initc"],"actions":[{"code":"currency","type":"transfer","recipients":["initb","initc"],"authorization":[{"account":"initb","permission":"active"}],"data":"000000000041934b000000008041934be803000000000000"}],"signatures":[],"authorizations":[]}'

The ref_block_num and ref_block_prefix here are provided as a result of /v1/chain/get_block of the last_irreversible_block. You also need to use /v1/wallet/sign_transaction to get the right signature.
Serialize json to binary hex. The resulting binary hex is usually used for the data field in push_transaction.
In spite of this ，I was still wrong。Once you've resolved those issues,please enlighten me.

Looks like you are using 2.x interface. Dawn3's push_transaction is totally different from 2.x.
The request of push_transaction should be something like:
{
"compression": "none",
 "packed_context_free_data": "00",
    "packed_trx": "73e5db5a00008e955ed0c0000003666666",
 "signatures": [
    "EOSJws1o98U6Vc1ACqZiJvqQN5o8BpsKqzvMPkZRY7LeVnKKwwcJWvBxtkrsiumZ88d1tgQLhJpc38pVgB4d2ndUrQAnCrkwC"
  ]
}
Maybe EOSDevHelper will help you.

@LIU9293 Here is what you are looking for.

I am having similar problem. I use push_transaction rpc api but got the error:
{
   "code":500,
   "message":"Internal Service Error",
   "error":{
      "code":3010010,
      "name":"packed_transaction_type_exception",
      "what":"Invalid packed transaction",
      "details":[
         {
            "message":"Missing signatures",
            "file":"abi_serializer.hpp",
            "line_number":393,
            "method":"extract"
         },
         {
            "message":"Failed to deserialize variant",
            "file":"abi_serializer.hpp",
            "line_number":487,
            "method":"from_variant"
         },
         {
            "message":"Invalid packed transaction",
            "file":"chain_plugin.cpp",
            "line_number":455,
            "method":"push_transaction"
         }
      ]
   }
}

I give push_transaction the following params:
{
   "signatures":[
      "SIG_K1_KeH6wCsnUWRjrfGtqPpXFK4BJArfQ4FwuZARnpVyWkfgdrbrKMgH1msyvrvGyxABYj5UyNpMmipWhpyaruRaRFQbcuDBb4"
   ],
   "compression":"none",
   "packed_context_free_data":"",
 "packed_trx":"000000005c95b1ca00000000007015d6010000000000000000454f5300000000087472616e73666572"
}

And I got the packed_trx above by using v1/chain/abi_json_to_bin, input params is like below:
"code":"eosio.token",
"action":"transfer",
"args":{
   "from":"tester",
   "to":"user",
   "quantity":"1 EOS",
   "memo":"transfer"
}

And I got the signature by using:
eosjs.modules.ecc.sign('signing tx', privateKey)

Cannot figure out what the problem is. Needing help!

Same problem.
@spartucus  Could please post some details about send transactions using RPC API?

my friend,I have the same problem.
http://<IP_ADDRESS>:8888/v1/chain/push_transaction post data {
"compression" : "none",
"packed_context_free_data" : "",
"packed_trx" : "0000001e4d75af46000000000010a6595cca12000000000004454f53000000001330784142415346444644464441465344464446",
"signatures" : [
"SIG_K1_Kc7XNr3aPsZ3asLXucmDJjMG8ACyRBxgryjd9Rm2xXJt1visyPiRQV1d4LqtvXZJmWtXQPzaJZRqjPC3QUerZxU6GhJVZ1"
]
}
response:
{
"code": 500,
"message": "Internal Service Error",
"error": {
"code": 3010010,
"name": "packed_transaction_type_exception",
"what": "Invalid packed transaction",
"details": [{
"message": "Invalid packed transaction",
"file": "chain_plugin.cpp",
"line_number": 722,
"method": "push_transaction"
}, {
"message": "read datastream of length 52 over by -2",
"file": "datastream.cpp",
"line_number": 6,
"method": "throw_datastream_range_error"
}, {
"message": "error unpacking uint64_t",
"file": "raw.hpp",
"line_number": 616,
"method": "unpack"
}, {
"message": "Error unpacking field value",
"file": "raw.hpp",
"line_number": 368,
"method": "operator()"
}, {
"message": "error unpacking eosio::chain::name",
"file": "raw.hpp",
"line_number": 616,
"method": "unpack"
}, {
"message": "Error unpacking field actor",
"file": "raw.hpp",
"line_number": 368,
"method": "operator()"
}, {
"message": "error unpacking eosio::chain::permission_level",
"file": "raw.hpp",
"line_number": 616,
"method": "unpack"
}, {
"message": "Error unpacking field authorization",
"file": "raw.hpp",
"line_number": 368,
"method": "operator()"
}, {
"message": "error unpacking eosio::chain::action",
"file": "raw.hpp",
"line_number": 616,
"method": "unpack"
}, {
"message": "Error unpacking field context_free_actions",
"file": "raw.hpp",
"line_number": 368,
"method": "operator()"
}]
}
}
the packed_trx is from the response of abi_json_to_bin :
{"binargs":"0000001e4d75af46000000000010a6595cca12000000000004454f53000000001330784142415346444644464441465344464446"}
where is the problem? Looking forward for you answer. Thank you.

same to me.

+1  Can someone tell me where the problem is?

@leekunkka maybe "packed_trx"  is wrong

It is very likely that the way you pack the transaction is wrong. abi_json_to_bin returns you the serialized action which you put inside a transaction, which later will be signed by you and be packed as packed_transaction. If you are using javascript, recommend you to use eosjs because it helps you to do the packing, otherwise, it will be quite cumbersome.


Update logrus

Please provide a description of this PR:
To fix: PRISMA-2023-0056
Reference: https://github.com/sirupsen/logrus/issues/1370

update logrus as part of https://github.com/istio/istio/pull/46658


Task failed: postalign_bam.bds', line 278

Unsure why my task failed here. Any suggestion? There are 8 other runs like this too but here are the stdout and stderr files for the one that finished last.
Thank you very much for your help,
Aaron Horning
ahorning at stanford period
stderrorNBM_55-4.txt
stdoutNBM_55-4.txt
edu

Did you get a success message when running install_dependencies.sh?
Did you have enough space on your system?
$ df -h /YOUR/WORKING/DIRECTORY
$ free -h
$ df -h /tmp
$ df -h $TMPDIR
$ df -h $TMP

It's Segmentation fault in sambamba sort and it usually occurs when there are not enough resource (memory, disk space on temps /tmp, $TMPDIR). Set your $TMPDIR in your ~/.bashrc and try again.
export TMPDIR=/somewhere/fast/and/large/storage/

If you are working on SCG, it is actually migrating to a new one with SLURM (not SGE). And our pipeline currently does not support SLURM on SCG. Let me know which SCG server you are working on.
You may need to submit a shell script for BDS command to SLURM manually with bds ... -system local.

Thank you both very much for your help on this. This was actually a very low starting input so it would make sense to me that it is hard to detect the peaks. I will lower the threshold for now and see how my peaks look.
fordebugging.txt

Pipeline now supports SLURM on new SCG. Please read the following instruction carefully and try again.
There has been some important updates about SLURM support on Kundaje lab's genomic pipelines (ChIP-Seq and ATAC-Seq) for the new SCG cluster.

You don't need to re-install Conda and dependencies

Update BigDataScript (IMPORTANT)
cd $HOME
rm -rf .bds/
wget https://github.com/leepc12/BigDataScript/blob/master/distro/bds_Linux.tgz?raw=true -O bds_Linux.tgz
tar zxvf bds_Linux.tgz

Update pipeline code
Do git pull on the pipeline git directory.

Usage
Add -q YOUR_PI_ACCOUNT_ON_SCG to the pipeline run command line.
The parameter -q is used for sbatch --account on SCG and -p on Sherlock.

-ATAC-Seq: bds atac.bds ... -q YOUR_PI_ACCOUNT_ON_SCG
-ChIP-Seq: bds chipseq.bds ... -q YOUR_PI_ACCOUNT_ON_SCG
Thanks,
Jin

closing this due to long inactivity.

Hi,
coming back to the initial question. I run into the same issue. Actually I got is solved by using TMPDIR to my ~/.bashrc. However, we have a central installation and I am setting TMPDIR TMP_DIR and TMP in an additional shell script:
export TMP=/clscratch/${USER}/atac_temp
export TMPDIR=/clscratch/${USER}/atac_temp
export TMP_DIR=/clscratch/${USER}/atac_temp

bds atac ....

If I check  task.postalign_bam.dedup_bam_PE_1_rep1.line_278.id_11.stdout.cluster, I see the following:
declare -x TMP="/tmp/12990246.1.default.q" -- defined where?
declare -x TMPDIR="/scratch/kiefefl2/" -- defined in ~/.bashrc
declare -x TMP_DIR="/clscratch/kiefefl2/atac_temp" -- defined the ATAC caller script

Why is $TMP being overwritten? Ideally, I could define the variables upfront and not in user specific .bashrc files.
Any ideas how to solve this?
Thanks a lot!
Cheers,
Flo

Sorry I don't know. This looks like a BDS problem. Post this issue on BDS github site. or your system problem? Some linux system overwrites env vars.
In my experience, I could define these env vars (TMP and TMPDIR) in ~/.bashrc and pipeline successfully took them from it.

Hi,
thanks for the suggestion. We will use that workaround for the moment.
However, it would be handy to have a dedicated TMP variable (or use the java_tmp_dir option) to specify  TMP for sambamba. At least the sort option has a tmp option:
--tmpdir=TMPDIR
Use TMPDIR to output sorted chunks. Default behaviour is to use system temporary directory.
Btw, thank you very much to maintaining this awesome pipeline!
Best,
Flo

Thanks for your suggestion, but we don't want to add any extra parameters for defining temporary directories.
The only parameter for temporary directory in the pipeline is -java_tmp_dir. We added it because Java does not automatically choose TMPDIR as its temporary directory.
Please let users define TMPDIR in their own ~/.bashrcs even though they are using central installation of the pipeline.

Hi Jin,
thanks a lot. We will continue with this model.
Best,
Flo
On Wed, May 23, 2018 at 11:09 PM Jin Lee<EMAIL_ADDRESS>wrote:

Thanks for your suggestion, but we don't want to add any extra parameters
for defining temporary directories.
The only parameter for temporary directory in the pipeline is
-java_tmp_dir. We added it because Java does not automatically choose
TMPDIR as its temporary directory.
Please let users define TMPDIR in their own ~/.bashrcs even though they
are using central installation of the pipeline.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kundajelab/atac_dnase_pipelines/issues/98#issuecomment-391497152,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGEDPNCM7BgJJ5tOIcviyrym5bM43Ak8ks5t1c_-gaJpZM4TAwTC
.


Optimize configuration and layout to reduce build-time

Various changes in configuration and some of the templates to considerably reduce build time:

enable the --profile option to output more detailed information about build-times
inline side-menu include.
It was fully static, only used in a single place, but would be processed for each page.
Optimize configuration to reduce build-times

remove wildcard match for defaults (this caused a ripple effect and slowed down total build time with 60 seconds?)
skip generating metadata.json (which took 25+ seconds)
skip generating sitemap.xml (10+ seconds)

inline "body-landing" and "body" into layout
This reduces the time to build, as Jekyll doesn't have to render the body, then copy it into the layout.

Argh! I missed adding one commit for the landing page; double checking now, and will do a quick follow-up

Fix in https://github.com/docker/docker.github.io/pulls


[breaking change]Support multi-col primary key

for https://github.com/xdblab/xdb/issues/88

close because we will rewrite based on the new design


[CSE-190] Explorer backend test coverage

https://issues.serokell.io/issue/CSE-190
Tests for the explorer.

We can (finally?) mock out the external functions (easily) - see HasExplorerCSLInterface which is using ExplorerMockMode passed through ExtraContextT.
Now all that's missing is to add functional tests for benchmarking.

I believe explorer/bench/results/ServerBackend.html file is autogenerated? Do we really want to commit it to repository ?

Yes. That's the CPU benchmark result. The idea is that we should be able to track the performance easily.  This way we can track what (or who) changed the performance of specific endpoints.

@ksaric how can we compile/run these benchmarks and tests? (I am a bit out of loop)

stack bench cardano-sl-explorer
But as mentioned before, these tests are currently more a skeleton since they are using pure functions lifted in the required context.
"Now all that's missing is to add functional tests for benchmarking." - we need to add tests that use ExplorerTestMode since that can give a more realistic performance (ideally, we would need to use a "real" database, currently we are using memory, even in these functional tests).

I was trying to remove GeneratedTestArguments and use ExplorerMock(able)Mode instead, but it seems that we cannot derive a Generic instance from it (because of it's complex/rich structure).
Also, we need to take a look at ExplorerTestMode if we want to extend it and use it in benchmarking.


Future of the nftnl-rs project

Disclaimer: please do not take this as a project advertisement, this is a genuine message to discuss the future of this project.
Please tell us if you feel this is harmful, as this is absolutely not the way we intended this message to be taken.
Considering the lack of recent activity in the repository (until a couple of weeks ago when a PR or two were merged), and the fact that one of us had started experimenting with a different (higher level but slightly less efficient resource-wise) API, we (@lafleurdeboum and myself) took the decision a few months ago to build upon nftnl-rs to create an easier to use libnftnl abstraction.
That library aims at providing a practical and idiomatic library to build firewalls or network filtering tools.
We named that fork rustables and licensed it under GPLv3, thanks to the rights granted by nftnl-rs' MIT license.
(As https://github.com/mullvad/nftnl-rs/issues/42 was pointing out, users of this library were already bound to the GPLv2+ requirement of libnftnl, so choosing to use either nftnl-rs or rustables shouldn't have much practical impact on them).
We included your initial copyright notices in our source code as visible in the README and in the main lib.rs.
We did this without any ill intent toward the nftnl-rs project, and with a spirit of fairness, and we emphasize that we absolutely don't want to harm the project.
Please tell us if you feel that what we did (or the way we did it, without consulting with you earlier) was not OK.
Which brings us to the crux of the matter: the future of nftnl-rs.
If we may ask: how do you envision the future of the library?
Do you believe the library is currently near feature complete, or is there endeavors you would like to see the nftnl-rs project tackle?
Do you see nftnl-rs as a value to yourself or your company, or do you see it as a burden that you need to maintain for your software to work on Linux?
Depending on the answer to these questions, I see three main paths ahead for both libraries:

you wish to maintain nftnl-rs as-is and consider it is mostly feature-complete for your needs
-> having two projects with separate goals is probably the path forward
you wish to maintain nftnl-rs as-is and do not consider it feature-complete
-> maybe we could join forces on some aspects, and who knows, even merge the two projects?
you do not wish to maintain nftnl-rs but need it for other software parts
-> maybe you could help us by pointing users that seek a higher-level library towards the rustables project, e.g. by adding a link at the bottom of the README

In the even of cases 1/2, or if the new licensing doesn't fit your need, we are open to contributing back the merge requests we have currently open against the project, and thus double-licensing this part of our work, as a testimony of our gratefulness for your work.
We want to thank you again for creating and developing this project, and we would be grateful for any insight into your position about rustables or the current status of nftnl-rs.

Given that this issue has been open for over a year without a comment, I believe I can close this?


Handle custom types arrays in ts callers

not implemented in as-transformer yet

you should be able to try it using https://github.com/massalabs/as.git#326-array-of-integer-other-than-i32-or-u32-are-not-working
or directly with this SC AS1G75wsgo31KGLjQC4gUQmcfTfutzhkRutTijJ3w1fKeaVW1NJi (on buildnet)


Initial support for enabling MyPy in CI

This PR adds initial support for MyPy in the Jenkins pipeline.
At this point, it checks only python/tvm/runtime and python/tvm/auto_scheduler.

It adds a standalone stage in Jenkins to check MyPy
It add an option in Makefile for $ make mypy.

@jroesch @jwfromm

Thanks @mikepapadim ! Can we merge the mypy check to the ci lint stage? This is mainly to simplify the pipeline. We can also push the items to https://github.com/apache/tvm/blob/main/tests/scripts/task_lint.sh

Thanks @mikepapadim ! Can we merge the mypy check to the ci lint stage? This is mainly to simplify the pipeline. We can also push the items to https://github.com/apache/tvm/blob/main/tests/scripts/task_lint.sh

Sure, I will revert the changes and keep it in the lint stage.

@mikepapadim Thanks! It's really great to have type check for python codes. While seems the CI failed on mypy but it didn't throw any error?
In my understanding, mypy requires python files to specifie data type explicitly for function parameters and return values, but we have not yet add the missing data type to these two dirs in this PR?

@mikepapadim Thanks! It's really great to have type check for python codes. While seems the CI failed on mypy but it didn't throw any error?
In my understanding, mypy requires python files to specifie data type explicitly for function parameters and return values, but we have not yet add the missing data type to these two dirs in this PR?

@jcf94  you are right. This PR does not fix the missing data types, it is only enables MyPy.
At this point error logging is disabled ( by setting ignore_errors in mypy.ini). So, thats why at the end of the linting stage in Jenkins  there is the following:
Checking MyPy Type defs in auto_scheduler package.

Success: no issues found in 17 source files

Checking MyPy Type defs in runtime package.

Success: no issues found in 13 source files

These two dirs are serving as placeholders atm.
Ideally, this PR adds the basic functionality.
Then, subsequent PRs can start incrementally to fix data types in specifics  dirs and enable them.

@mikepapadim Thanks, that make sense to me. Looking forward to see all these python files have been filled with data types.
p.s. I just find a:
mypy: error: unrecognized arguments: --install-types

error in the CI log, does that need to be solved?

@mikepapadim Thanks, that make sense to me.
p.s. I just find a:
mypy: error: unrecognized arguments: --install-types

error in the CI log, does that need to be solved?
I been looking into this.
It looks like MyPy version issue. I will try to reproduce it locally.
https://github.com/python/mypy/issues/10596

It's better to specify all linting/format checking tool versions (i.e., pylint, mypy, black, clang-format) to make sure the results are consistent.

Finally!! Thanks for the great work!
As TensorIR is fully prepared for type annotations, we should probably check python/tvm/tir/schedule as well. CC: @Hzfengsy @tqchen

@junrushao1994 we can followup with a PR

Finally!! Thanks for the great work!
As TensorIR is fully prepared for type annotations, we should probably check python/tvm/tir/schedule as well. CC: @Hzfengsy @tqchen

@junrushao1994 https://github.com/apache/tvm/pull/8367 Please have a look in this one


Release 2024.5

With RVs and spitzer photometry from the UC sheet.

https://github.com/SIMPLE-AstroDB/SIMPLE-db/wiki/Release-Workflow

[ ] Update Versions table
[ ] Create Release on Github
[ ] If necessary, update SIMPLE-binary repo with new database file. (Creating the release auto-triggers the Generate Database action)
[ ] Run a codespace instance from SIMPLE-web repo to make sure the website works with new database.
[ ] Sync new file to website host.
[ ] Restart Application Server Nodes on website host
[ ] Start an instance of the app by visiting https://simple-bd-archive.org/.


into_offset_iter() - Incorrect range for Table event

Given the following:
Testing|This|Outtt
--|:--:|--:
Some Data|Other data|asdf

The table range starts at 18, but I think it should start at 0:
Event: Start(Table([None, Center, Right]))
Range: 18..57

Event: Start(TableHead)
Range: 0..19

That probably should start at 0 indeed. Thanks for reporting this!


link: fix old/bad Get Started links

This is self-approved maintenance work, but need a formal approval to merge.

A link is failing. Typo I think... ⌛


Pass GPG secrets to workflows.

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@6687ba5). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #25   +/-   ##
=======================================
  Coverage        ?   50.00%
  Complexity      ?        3
=======================================
  Files           ?        2
  Lines           ?       10
  Branches        ?        1
=======================================
  Hits            ?        5
  Misses          ?        3
  Partials        ?        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6687ba5...bfd73b3. Read the comment docs.


file type filter

Hi, thanks for making this node public!
It is more of a question, rather that issue. How can I add file type/extension filter to the node configuration, so that only particular file type (e.g. *.CSV) could be selected in Open File dialog box?
Thanks

An accept attribute should be added to
https://github.com/alexandrainst/node-red-contrib-ui-upload/blob/a048bd1adef60a69f45e0ee0f5ef1c65667a41e9/ui_upload.js#L51
See https://developer.mozilla.org/docs/Web/HTML/Element/input/file
Ideally, there should be a new option to define it from the Node-RED configuration interface
Pull request welcome

Hello,
Thank you for explanation and great guidance on how to make a fix. If i'll find enough time, i'll make PR.

See https://github.com/alexandrainst/node-red-contrib-ui-upload/pull/15
Tests welcome


Your Manifest file ScreenSaver.yaml has the wrong Required API version for Playnite Addon Database

Just thought Id let you know you set the wrong RequiredApiVersion in your manifest file so currently no one can install from the Playnite Add-ons menu. You have it set to 9.0.0 it should be set to I believe its 6.2.0 for P9 or 6.3.0 for P10.

Thanks for pointing it out. Fixed!


Use full Local Hack Day Logo with GitHub branding

What would y'all think about using the full Local Hack Day seen here?
I'm referring specifically to the banner beneath with "Hosted by GitHub".
I think having the Hosted by GitHub is powerful branding that could encourage better attendance.
I'm aware right beneath that logo we currently have 'Hosted by Hack CU', but perhaps we could change that to something along the lines of 'Presented by HackCU' or 'Organized by HackCU'.
I just think keeping the GitHub branding adds credibility to the event for people who are first seeing about it.
Thoughts?

P.s. I do have an email from MLH confirming that we can use that logo, along with an original SVG they provided for us to use.

@elliotwhitehead yeah, we can totally change the logo and the text below it. Do you want to do that? If you're busy, you can share the logo on slack and the one who is least busy can get to it ASAP.

Mmm. Even though I love github, given the fact that we have sponsors I wouldn't put Github there. It may be a conflict of interests for other sponsors who are paying us (and not MLH like Github). I agree on adding Github under our sponsors sections though

I suppose that's a good point. Hm. What if we used the full LHD logo with GitHub on printed posters, but not on the website?


Investigate tokio panic (assertion failed: self.ref_count() > 0)

Sentry link: https://neondatabase.sentry.io/issues/5076699787/?alert_rule_id=13489026&alert_type=issue&notification_uuid=821438c5-fe8b-4aef-94a9-4f932dd3c959&project=4504220031582208&referrer=slack
Slack thread: https://neondb.slack.com/archives/C033RQ5SPDH/p1710750099615559
Current theories are:

tokio bug
leaky_bucket bug

Call site: https://github.com/neondatabase/neon/blob/77f3a30440aba4845da3a5203a2764fed4d96648/pageserver/src/tenant/throttle.rs#L146
        std::future::poll_fn(|cx| {
            use std::future::Future;
            let poll = acquire.as_mut().poll(cx);
            did_throttle = did_throttle || poll.is_pending();
            poll
        })
        .await;

Backtrace:
panic{thread=compute request worker location=/home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/state.rs:592:9}: assertion failed: self.ref_count() > 0
Stack backtrace:
   0: utils::logging::tracing_panic_hook
             at /home/nonroot/libs/utils/src/logging.rs:206:21
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2029:9
   2: std::panicking::rust_panic_with_hook
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:783:13
   3: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:649:13
   4: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:171:18
   5: rust_begin_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
   6: core::panicking::panic_fmt
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
   7: core::panicking::panic
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:144:5
   8: tokio::runtime::task::state::State::transition_to_notified_by_val
   9: tokio::runtime::task::waker::wake_by_val
  10: leaky_bucket::AcquireState::drain_wait_queue
  11: leaky_bucket::AcquireState::drain_core
  12: <leaky_bucket::Acquire as core::future::future::Future>::poll
  13: <tokio::task::unconstrained::Unconstrained<F> as core::future::future::Future>::poll::{{closure}}
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/unconstrained.rs:25:57
  14: tokio::runtime::coop::with_budget
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/coop.rs:107:5
  15: tokio::runtime::coop::with_unconstrained
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/coop.rs:80:5
  16: <tokio::task::unconstrained::Unconstrained<F> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/unconstrained.rs:25:13
  17: pageserver::tenant::throttle::Throttle<M>::throttle::{{closure}}::{{closure}}
             at /home/nonroot/pageserver/src/tenant/throttle.rs:146:24
  18: <core::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/future/poll_fn.rs:151:9
  19: pageserver::tenant::throttle::Throttle<M>::throttle::{{closure}}
             at /home/nonroot/pageserver/src/tenant/throttle.rs:150:10
  20: pageserver::tenant::timeline::Timeline::get::{{closure}}
             at /home/nonroot/pageserver/src/tenant/timeline.rs:657:53
  21: pageserver::pgdatadir_mapping::Version::get::{{closure}}
             at /home/nonroot/pageserver/src/pgdatadir_mapping.rs:1654:63
  22: pageserver::pgdatadir_mapping::<impl pageserver::tenant::timeline::Timeline>::get_rel_page_at_lsn::{{closure}}
             at /home/nonroot/pageserver/src/pgdatadir_mapping.rs:199:37
  23: pageserver::page_service::PageServerHandler::handle_get_page_at_lsn_request::{{closure}}::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:1156:14
  24: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.37/src/instrument.rs:272:9
  25: pageserver::page_service::PageServerHandler::handle_get_page_at_lsn_request::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:1108:5
  26: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.37/src/instrument.rs:272:9
  27: pageserver::page_service::PageServerHandler::handle_pagerequests::{{closure}}::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:646:30
  28: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.37/src/instrument.rs:272:9
  29: pageserver::page_service::PageServerHandler::handle_pagerequests::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:548:5
  30: <pageserver::page_service::PageServerHandler as postgres_backend::Handler<IO>>::process_query::{{closure}}::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:1408:18
  31: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.37/src/instrument.rs:272:9
  32: <pageserver::page_service::PageServerHandler as postgres_backend::Handler<IO>>::process_query::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:1375:5
  33: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/future/future.rs:124:9
  34: postgres_backend::PostgresBackend<IO>::process_message::{{closure}}
             at /home/nonroot/libs/postgres_backend/src/lib.rs:747:75
  35: postgres_backend::PostgresBackend<IO>::run_message_loop::{{closure}}
             at /home/nonroot/libs/postgres_backend/src/lib.rs:485:80
  36: postgres_backend::PostgresBackend<IO>::run::{{closure}}
             at /home/nonroot/libs/postgres_backend/src/lib.rs:414:14
  37: pageserver::page_service::page_service_conn_main::{{closure}}::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:275:10
  38: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.37/src/instrument.rs:272:9
  39: pageserver::page_service::page_service_conn_main::{{closure}}
             at /home/nonroot/pageserver/src/page_service.rs:211:1
  40: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::future::future::Future>::poll
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic/unwind_safe.rs:297:9
  41: <futures_util::future::future::catch_unwind::CatchUnwind<Fut> as core::future::future::Future>::poll::{{closure}}
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/catch_unwind.rs:36:42
  42: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic/unwind_safe.rs:272:9
  43: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  44: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  45: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  46: <futures_util::future::future::catch_unwind::CatchUnwind<Fut> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/catch_unwind.rs:36:9
  47: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/task_local.rs:347:31
  48: tokio::task::task_local::LocalKey<T>::scope_inner
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/task_local.rs:217:19
  49: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/task_local.rs:343:19
  50: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::{{closure}}
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/task_local.rs:347:31
  51: tokio::task::task_local::LocalKey<T>::scope_inner
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/task_local.rs:217:19
  52: <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/task/task_local.rs:343:19
  53: pageserver::task_mgr::task_wrapper::{{closure}}
             at /home/nonroot/pageserver/src/task_mgr.rs:398:10
  54: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/core.rs:328:17
  55: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/loom/std/unsafe_cell.rs:16:9
  56: tokio::runtime::task::core::Core<T,S>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/core.rs:317:13
  57: tokio::runtime::task::harness::poll_future::{{closure}}
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:485:19
  58: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panic/unwind_safe.rs:272:9
  59: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  60: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  61: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  62: tokio::runtime::task::harness::poll_future
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:473:18
  63: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:208:27
  64: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.36.0/src/runtime/task/harness.rs:153:15
  65: tokio::runtime::scheduler::multi_thread::worker::Context::run_task

90% confident that it's not a neon bug

There are two similar upstream issues:

https://github.com/tokio-rs/tokio/issues/6422
https://github.com/tokio-rs/tokio/issues/5295

We were on tokio 1.17 when we had this panic, and are upgrading to latest tokio before looking for re-occurrences -- if/when that happens we'll continue to chase the issue upstream.

Turns out we are on the latest tokio version: https://github.com/neondatabase/neon/commit/e62baa97041e10ce45772b3724e24e679a650d69

@VladLazar I'm investigating this right now on the tokio side. We think this is a memory corruption issue unrelated to tokio which tokio is the first to yell about.
Have you folks done any other dependency updates recently that line up?
Also a possibility is that it's actually a recent neon issue, but I'm strongly leaning towards this being another common crate, likely in tokio's tree, given the number of reports.

Interesting!
Keep me in the loop here, if this ends up being an issue on your end that helps us narrow down what could be going on with our other reports, as our current theory is that there's a common issue

Interesting observation: Peter's benchmarks seem to repro this. We have seen the throttling rooted panic two times in staging in the past 7 days. Both times it was on Peter's tenants.

Got a new panic in prod eu-central-1@pageserver-4
	2024-04-23 20:14:23.837
2024-04-23T20:14:23.750193Z ERROR panic{thread=compute request worker location=/home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/state.rs:195:9}: current: 1, sub: 2
Stack backtrace:
   0: utils::logging::tracing_panic_hook
             at /home/nonroot/libs/utils/src/logging.rs:206:21
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2029:9
   2: std::panicking::rust_panic_with_hook
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:785:13
   3: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:659:13
   4: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/sys_common/backtrace.rs:171:18
   5: rust_begin_unwind
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:647:5
   6: core::panicking::panic_fmt
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/panicking.rs:72:14
   7: tokio::runtime::task::state::State::transition_to_terminal
   8: tokio::runtime::task::harness::Harness<T,S>::complete
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:335:12
   9: tokio::runtime::task::harness::Harness<T,S>::poll
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.37.0/src/runtime/task/harness.rs:168:17
  10: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
  11: tokio::runtime::scheduler::multi_thread::worker::Context::run
  12: tokio::runtime::context::runtime::enter_runtime
  13: tokio::runtime::scheduler::multi_thread::worker::run
  14: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
  15: tokio::runtime::task::core::Core<T,S>::poll
  16: tokio::runtime::task::harness::Harness<T,S>::poll
  17: tokio::runtime::blocking::pool::Inner::run
  18: std::sys_common::backtrace::__rust_begin_short_backtrace
  19: core::ops::function::FnOnce::call_once{{vtable.shim}}
  20: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2015:9
  21: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2015:9
  22: std::sys::pal::unix::thread::Thread::new::thread_start
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/sys/pal/unix/thread.rs:108:17
  23: start_thread
             at ./nptl/pthread_create.c:477:8
  24: clone
             at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

We think this is a memory corruption issue unrelated to tokio which tokio is the first to yell about.

This is my current understanding as well. I found a way to trigger both panics seen by memcpy-ing wakers and immediately dropping them (violating the Drop/!Copy contract).
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=5d2e870fb3e5bf36dfd11e0233c438d5
I cannot find any other way at the moment to trigger the same behaviour with any code in tokio 1.37.0. When investigating any dependencies, looking for potentially double-free bugs is what we should be looking for

Haven't seen any recent instances of this. Can we close this issue out @VladLazar?


In React 16 Upgrade, getting error: Uncaught TypeError: Cannot read property 'input' of undefined

I have a React project using Redux Form extensively (v7.4.2). I am trying to upgrade my Reactjs version from v15.6.2 to ^v16.0.0 and am getting the above error.
Clicking through the error in the console, I see that the _React$DOM is undefined on .input. I have   followed all the migration instructions on the React site and looked extensively, cannot seem to get rid of this error. I'm guessing that there is some issue with input fields in Redux Form 7.4.2 and and React 16 that is not present in React v15.6.2.
Any ideas what that issue is and how I can resolve it?
I cannot give all the code related to Redux Form in my project, but below are my package.json and webpack.config. They are pre-upgrade to v16, but I have tried adding core-js per Reactjs.org.
var _React$DOM = _react2.default.DOM,
    input = _React$DOM.input, <<< Here is the highlighted error
    select = _React$DOM.select,
    textarea = _React$DOM.textarea;

My dependencies in package.json:

  "devDependencies": {
    "babel-loader": "^7.1.5",
    "babel-preset-es2015": "^6.1.18",
    "babel-preset-react": "^6.1.18",
    "babel-preset-stage-0": "^6.24.1",
    "chai": "^4.1.2",
    "chai-jquery": "^2.0.0",
    "css-loader": "^1.0.0",
    "eslint-config-rallycoding": "^3.2.0",
    "mocha": "^5.2.0",
    "react-addons-test-utils": "^15.0.1",
    "style-loader": "^0.21.0",
    "webpack-cli": "^3.1.0",
    "webpack-dev-server": "^3.1.8"
  },
  "dependencies": {
    "actioncable": "^5.2.4",
    "axios": "^0.18.0",
    "babel-core": "^6.2.1",
    "babel-polyfill": "^6.26.0",
    "babel-preset-stage-1": "^6.1.18",
    "cloudinary-core": "^2.5.0",
    "cloudinary-react": "^1.0.6",
    "dotenv-webpack": "^1.5.5",
    "file-loader": "^2.0.0",
    "jquery": "^3.3.1",
    "jsdom": "^12.0.0",
    "lodash": "^4.1.0",
    "prop-types": "^15.6.1",
    "react": "^15.6.2",
    "react-bootstrap": "^0.32.1",
    "react-day-picker": "^7.1.6",
    "react-dom": "^15.6.2",
    "react-dropzone": "^5.0.1",
    "react-helmet": "^5.2.0",
    "react-redux": "^5.0.7",
    "react-router-dom": "^4.3.1",
    "react-stripe-elements": "^2.0.1",
    "redux": "^4.0.0",
    "redux-form": "^7.4.2",
    "redux-thunk": "^2.2.0",
    "sha1": "^1.1.1",
    "webpack": "^4.17.2"
  }

My webpack.config:
Here is webpack.config:
const webpack = require('webpack');
const Dotenv = require('dotenv-webpack');
const raf = require('raf')

const path = require('path');

module.exports = {
  mode: 'none',
  watch: false,
  entry: [
    'babel-polyfill', './src/index.js'
  ],
  output: {
    path: __dirname,
    publicPath: '/',
    filename: 'bundle.js'
  },
  module: {
    rules: [
      {
        test: /\.(jpg|png|svg)$/,
        use: [
          {
            loader: 'file-loader',
            options: {
              name: '[path][name].[hash].[ext]'
            }
          }
        ]
      },
      {
        test: /\.js?$/,
        exclude: /node_modules/,
        use:
          { loader: 'babel-loader',
        options: {
         presets: ['react', 'es2015', 'stage-0']
       }
      }
    }

    ] // end of loaders

  devServer: {
    historyApiFallback: true,
    contentBase: './'
  },
  plugins: [
    new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV)
    }),
    new Dotenv()
  ]
};

@chikarao why you decided this is a redux-form issue?

Hi, based on the error message, it appears _React$Dom is null for .input, .select, and .textarea. I use Redux Form on just about all of those elements. I cannot think of any other library that I use that would cause such an error. Any ideas on where else if not Redux Form?

@chikarao unfortunately this issue is not related to redux-form. react-dom is not used in redux-form


prepDE.py yielding more counts compared to what the fastqc estimates

Hi Developer!
I've been working with the pipeline described in the Nature Protocols paper (2016). I've done everything up to the ballgown step. Instead of using ballgown, I've used prepDE.py to get the counts to use DESeq2. However, I've made a colSums() on the counts matrix (for both gene and transcripts) from the output of prepDE.py and I find that I have an increased number of counts-read-paired. For example,  for sample1 I have 72,165,071 "total sequences" in my .fastqc output. But in the case of counts yielded by prepDE.py I have 109,846,849. The same for the other 3 samples that I am using.  Is this what is expected? or is it a bug? I can make my .gtf files available.
Many thanks for your help and support!!

Interesting observation. Not sure what is the best explanation for it, but let's start with the fact that the counts shown by prepDE are calculated from the coverage estimates  for each transcript/isoform. These coverage values are estimated based on a network flow distribution model and they do not precisely reflect the actual read counts per base -- the relative differences between such coverage estimates are more significant here. I guess it is unavoidable to overestimate such counts in the case of multiple isoforms where some alignments cannot be unambiguously assigned to one isoform or another (due to overlapping or common exons) so they might "appear" to be counted twice sometimes..
Ela should correct me here if I'm wrong, but I suspect the network flow algorithm sometimes might assume/infer higher flow values which should be there for some graph traversals, even though actual reads to account for the expected flow capacity values might be missing there in some parts of the graph - due to sequencing bias, quality filtering or other factors which could prevent us from getting/seeing physical reads at the expected level of expression for some of the predicted exons and isoforms..
Secondly (and perhaps less likely), this over-counting issue may also be related to multi-mapped reads - a read which maps in multiple places on the genome (with the same alignment score). StringTie has no choice but to assign some amount of "coverage" in all those places as in many cases it cannot decide which location is the "real" one. Even though fractional coverage is used in these situations, it is possible, due to rounding errors and other factors, to end up getting a higher total "coverage" for all those places, I suppose..

I have the same problem too! My fastq has 4482320 reads, but colSums() of transcript counts (or gene counts) is 11775080. I called prepDE.py by default length parameter, which is 75. However, my fastq is piared-end and each end is 150 bp. So should I set -l to 300? It seems in this way colSums of transcript counts would be 11775080/4 = 2943770 < 4482320, which seems acceptable.

Could you clarify as to the input numbers you are comparing with the prepDE estimates: when you say you have 4 million reads you really mean reads or you mean 4 million pairs , in your RNA-Seq sample (the latter meaning 8 million effective reads). StringTie estimates the coverage based on the effective number of reads per base, taken from the provided alignments, so it doesn't matter if they are paired or not -- it just counts how many reads overlap each base of the predicted exons/transcripts.
I guess there might be some interpolation and rounding errors for exons with incomplete coverage, multi-mapped reads etc. but such deviations should be similar for paired reads as for single reads. What you seem to be reporting here is that these raw count estimates are really off (almost double) when using paired reads vs. single reads -- and I want to clarify that this is indeed the case.
Also, in the example above, if you have 150bp reads you should definitely inform prepDE.py about that length (i.e. use -l 150) in order to get better raw count estimates per transcript.

@gpertea Thanks! I have 4.48 million pairs indeed. And each end is 150 bp. In light of your suggestion, I set -l to 150 and sum of transcript count (or gene count) is 5903709, which is more than 4.48 million but less than 8.96 million.

@gpertea Your answer has clarified the situation because indeed, I counted paired reads as a single read. This, however, raises further questions, because in my understanding RNAseq analysis tools like edgeR or DESeq do not expect paired-end reads to be counted independently. Paired-end reads come from the same cDNA fragment and they are not equivalent to two separate single-end reads, which come from different fragments. Some of the alternative tools like HTseq explicitly state in their documentation ([http://htseq.readthedocs.io/en/master/count.html]), that one read pair in paired-end sequencing is a single piece of evidence as it originates from a single cDNA fragment. From what you have said it seems that prepDE.py would treat the paired-end library as a single-end library with double read count.

Although two years have past since the last message on this thread I think the issue is not entirely solved. The question raised in the last message of @PNorvaisas is still open. @gpertea stated that "StringTie estimates the coverage based on the effective number of reads per base". This means that by using the conceptual formula "coverage * transcript_len / read_len", cited by @PNorvaisas, prepDE.py will calculate counts of the "effective reads", which in case of SE data corresponds to number of cDNA fragments per transcript but in case of PE data it corresponds to double the number of cDNA fragments per transcript because each fragment contributes two effective reads to a transcript coverage. If this is true than for PE data, the counts produced by prepDE are not compatible with what DESeq2 or edgeR are expecting, as they expect counts of cDNA fragments and not the counts of effective reads. This would mean that in case of PE data one should use prepDE with "-l 2 * read_len". I have experimented with that and it seems that the counts produced by prepDE for SE are always what you would expect, while for PE data using "-l read_len" gives always counts that after sumation give total count number higher than the read number prior the alignment; using "-l 2 * read_len" however gives count values that after sumation are somewhat lower than what you would expect given the alignment statistics.  Please could somebody comment on that. I can't believe that this issue has not been cleared for so long, or am I missing something fundamental here?

I think this is a rather complex issue and we don't claim to have a good answer right now, especially when it comes to PE reads.
I remind you that prepDE simply applies a simple formula to derive hypothetical read counts from the average base coverage estimated by StringTie for each assembled transfrag. Indeed, the model employed for that calculation is based on a simple independent read alignment model -- i.e. prepDE does not care if two read alignments are paired as part of a single fragment (because prepDE does not have access to that information, but StringTie did when it built the transcript models and estimated their coverage).
So indeed, if the downstream analysis tools need fragment count instead of read count (which seems counter-intuitive to me at this point, since the only concrete, guaranteed alignment unit is the read, not the fragment), a simple workaround would be to divide that read count by 2, as mentioned before (which is what "-l 2*read_len" effectively accomplishes). But of course this would be very coarse approximation if there is a significant number of reads which are not properly aligned together as pairs, without overlapping (redundancy).
Perhaps most relevant to the discrepancy you observed when trying to get fragment counts instead of read counts is the fact that in PE sequencing the paired reads can overlap, sometimes significantly. StringTie would not count "twice" the base coverage in the overlapping region, but a very naive read alignment counter might do that -- and actually prepDE would indirectly appear to make exactly that mistake when one uses "-l 2*read_len" in an attempt to get fragment counts instead of read counts.

Admittedly I don't have much experience with using read counts (or rather fragment counts, as I now realize) as a starting point for DE analysis, as I assumed that the programs requiring those counts would actually use them to calculate the average base coverage of the transcripts anyway, in the process of estimating the expression levels there.
If that's the case perhaps a better workaround here would be to use the read counts provided by prepDE but pretend that it is a SE experiment instead of PE (so don't use "-l 2*read_len" option with prepDE), assuming that the DE analysis program requires that PE vs SE information.
Again, I haven't used DESeq2 or similar programs lately but if all they do with those read counts is calculate transcript coverages, let's just give them the virtual SE read counts, as reverse-estimated by prepDE from StringTie coverage data, and tell those programs that it's a SE experiment.. That would be a much better way to deal with prepDE limitations (rather than attempting to get the correct fragment counts from prepDE, which seems to be an impossible task).

Yes for DE analysis we need to count the fragments and reads are simply proxies for that, for SE we have one read per fragment for PE we have ideally (and roughly) 2 reads per fragment. At no point deseq or edgeR would calculate the full coverage as the whole analysis is based on counts only.
I am not sure whether I understand your last suggestion, I think using "-l 2 * read_len" is in fact pretending that we have SE reads of double length - but this perhaps is an overkill. I think your point about the overlapping reads is key. This would suggest that if PE reads overlap they contribute "2 x read_len - overlap_len" to the coverage, which effectively is the length of a fragment. My question is whether the PE non-overlapping reads contribute also the length of a fragment, which is "2 x read_len + insert_len" or just two read lengths only. The 2015 Stringtie paper states: "StringTie also reports a read per base coverage for each exon of a predicted transcript", which suggests the latter. If this is the case than perhaps calculating "the average read-pair contribution to the coverage" would give us an optimal length value to be used as -l parameter in prepDE. This value would be lower than "2 * read_len" but higher than "1 * read_len" and its actual value would depend on the fraction of overlapped PE reads.

My question is whether the PE non-overlapping reads contribute also the length of a fragment, which is "2 x read_len + insert_len" or just two read lengths only. The 2015 Stringtie paper states: "StringTie also reports a read per base coverage for each exon of a predicted transcript", which suggests the latter. If this is the case than perhaps calculating "the average read-pair contribution to the coverage" would give us an optimal length value to be used as -l parameter in prepDE

And herein lies the problem, I think. Fragment length is not something that is used internally by StringTie in order to calculate that read-per-base coverage value.  Indeed StringTie calculates that coverage value for each exon, but that value is directly based on individual read alignments -- with the special exception of overlapping reads from the same fragment, in which case the overlapping portion of such reads is counted only once. That is happening on a case by cases basis (read pair by read pair) -- as the overlapping portion varies or might not be there at all.
To emphasize, StringTie approach to coverage calculation is read-centric, it does not use a "fragment length" value when computing the read-per-base coverage, though it does keep track of the average fragment length encountered for each bundle, which is needed for other calculations.
My suggestion was to forget about PE and fragments and just direct DESeq and edgeR to use the counts produced by prepDE as if it were a SE experiment, with the read length set to the actual read length as found in the alignment data (not the fragment length).
The end goal is estimating expression levels anyway, which should be (or so I thought) directly bound to the effective read-per-base coverage of each transcript (with the special correction for the overlapping region of paired reads, which StringTie applies). Now of course if the read length is not constant in the sequencing data (or there is a lot of soft-clipping going on), using that read length to recover the "virtual" read counts with prepDE may also be inadequate.
If you already tried that and the results were wildly inadequate then of course my suggestion was wrong and we still have to dig deeper to understand the issue here, thought at this time I am afraid I do not have more to add to this otherwise interesting discussion.


Switch from storage.tsdb.retention property to storage.tsdb.retention.time

This PR will substitute the deprecated storage.tsdb.retentionproperty to storage.tsdb.retention.time as mentioned in #296 .
I decided to go with the .time instead of _time to be more flexible to add storage.tsdb.retention.size to spec as soon as it leaves status experimental.

Thanks @benjaminguttmann-avtq!


Update dependency ts-loader to v6.2.1

This PR contains the following updates:

Package
Type
Update
Change

ts-loader
devDependencies
minor
6.0.3 -> 6.2.1

Release Notes

TypeStrong/ts-loader
v6.2.1
Compare Source

Output types alongside JS files, enable declaration maps - thanks @meyer!

v6.2.0
Compare Source

Emitting .tsbuildinfo when using watch api - thanks @sheetalkamat!

v6.1.2
Compare Source

don't emit declaration files for a declaration file (#1014) - thanks @gvinaccia!
Consume typescript apis from typescript nightly - thanks @sheetalkamat!

v6.1.1
Compare Source

Fix SolutionBuilder watches and related fixes (#998) - thanks @sheetalkamat!
fix: no errors reported if flagged with @ts-check (#1004) - thanks @reinholdk!

v6.1.0
Compare Source

Build upstream project references with SolutionBuilder (#851, #913) - thanks @sheetalkamat!

v6.0.4
Compare Source

Fix issue when handling files not included in tsconfig.json (#934) - thanks @davazp!

Renovate configuration
:date: Schedule: At any time (no schedule defined).
:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.
:recycle: Rebasing: Whenever PR becomes conflicted, or if you modify the PR title to begin with "rebase!".
:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

PR has been edited
:construction_worker: This PR has received other commits, so Renovate will stop updating it to avoid conflicts or other problems. If you wish to abandon your changes and have Renovate start over you may click the "rebase" checkbox in the PR body/description.


filter : textinput as results

ng-repeat="person in persons | filter : text as filteredCollection | orderBy : 'name' | limitTo : 2"

Why this filter is not working? Here is no reserved names or something else which could conflict. Just not working. Without filter all is working properly.
persons = array;
text = input[type=text];

@FlamesoFF impossible to say without a clear, minimal, runnable reproduce scenario using plunker, jsbin, jsfiddle or similar. Please provide one - without it this issue is not actionable really...

Here: http://jsfiddle.net/q0yfm6ap/

@FlamesoFF the reason is that as [variable name] is not an operator but rather a part of ngRepeat micro-syntax so it can be used only at the end (and not as operator, inside an expression), see:
http://jsfiddle.net/c2s7dLuj/
Closing, as it works as expected. If you want to capture intermediate results you can do so by moving filtering to a controller's function and do custom processing on the JS side.

Thanks, where I can find this in documentation?

https://docs.angularjs.org/api/ng/directive/ngRepeat

Jut mentioning another alternative for capturing intermediate results in the view is:
ng-repeat="p in (results = ps | filter : text) | orderBy : 'name' | limitTo : 2"


Router events runs multiple times.

When i'm using the Inertia Router start event, all previous events gets logged too.
I'm using the following code:
router.on("start", (event) => {
    console.log(event);

    console.log(`Navigated to ${event}`);

    console.log("--------------");
});

This code is running in the main Layout, so it shouldn't be loaded more than once.
The purple box on the image is the console output when i click any link first time, the red box is when i click it the second time (i get both previous event and current event) and if i click it again it will continue like that (recursively).
Is this a bug, or am i doing it wrong?

How are you calling your layout?
It’s looks like the layout setup is running with each page load, this is to be expected if your layout isn’t persistent.

In DEV mode maybe it's ok because everything runs twice...

How are you calling your layout?
It’s looks like the layout setup is running with each page load, this is to be expected if your layout isn’t persistent.

This is an example of my index page that uses the Layout component:
`

`
Pretty much the out of the box way of loading the DeveloperLayout. The DeveloperLayout is just the default Layout component renamed.
I've tried with a fresh Laravel-Inertia installation too, and the issue is the same.
The event is triggered in the DeveloperLayout component.

In DEV mode maybe it's ok because everything runs twice...

I just tested in production mode. The issue is the same.

Try following the instructions for Persistent Layouts instead of wrapping your layout around your page.
 This is likely causing the event listener to be registered multiple times.


some confusion about filePrinter

The xLog is very powerful and pretty, i am excited in using it.
but when i use the function of printing log to the sdcrad,  i meet some question.
for example,when i use a globalFilePrinter and a LocalFilePrinter in the same time,and set the file path same,and the filename generator by date as also, it would't create two log file, they will write in a same file. is it a BUG?
in another case, I just use a globalFilePrinter,then  it will create one log file, does the xLog has the ability to clear the log file automatically ？ If not ,the log file will be very large.

@softmicro929

Two printers, same file. I think you can specify different path for these two printers. For example, printer1 write logs in "/your_path/printer1/", printer2 write logs in "/your_path/printer2/", so even you use the same filename generator(by date) they won't write in a same file.
Currently, xLog doesn't provide a method for clearing log files. For your problem, single log file seems to be very large, you can specify a backup strategy(FilePrinter.Builder#backupStrategy) for your file printer, I think it can fix your problem.

@elvishew
Get it. I didn't notice the backupStrategy.Thank you.

CleanStrategy is supported since version 1.5.0


First step removed?

Older versions of the code used 2 steps, with only the second step using the laplacian? Why was the first step removed?

Here we have 2k iterations with laplacian?


Add raw AES-GCM key to document head instead of Keyset

Modifies the key stored in the cryptokeys script in head to go from sending an entire Keyset vs. just the raw symmetric key. This makes it so we can use different cryptographic packages to encrypt/decrypt.
Changes the prefix type of the encrypted messages from "TINK" to "RAW", meaning that no extra information about the keys gets appended to the encrypted text: https://github.com/google/tink/blob/master/proto/tink.proto#L70
Capitalizes names in JSON object so that golang exports them correctly.

Checked the tests and they all pass because we don't validate the encrypted text itself, just check to make sure that it is generated.