DataLoader timed out

Description
Whenever I set num_workers > 0 in gluon.data.DataLoader I get timed out.
Error Message
Worker timed out after 120 seconds. This might be caused by

            - Slow transform. Please increase timeout to allow slower data loading in each worker.
            - Insufficient shared_memory if `timeout` is large enough.
            Please consider reduce `num_workers` or increase shared_memory in system.

---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
<ipython-input-5-bf565db5d0a6> in <module>
      1 lr, num_epochs = 0.01, 1
----> 2 d2l.train_ch6(net, train_iter, test_iter, num_epochs, lr)

/usr/local/lib/python3.5/dist-packages/d2l/d2l.py in train_ch6(net, train_iter, test_iter, num_epochs, lr, ctx)
    419     for epoch in range(num_epochs):
    420         metric = d2l.Accumulator(3)  # train_loss, train_acc, num_examples
--> 421         for i, (X, y) in enumerate(train_iter):
    422             timer.start()
    423             # Here is the only difference compared to train_epoch_ch3

/usr/local/lib/python3.5/dist-packages/mxnet/gluon/data/dataloader.py in __next__(self)
    503         try:
    504             if self._dataset is None:
--> 505                 batch = pickle.loads(ret.get(self._timeout))
    506             else:
    507                 batch = ret.get(self._timeout)

/usr/lib/python3.5/multiprocessing/pool.py in get(self, timeout)
    602         self.wait(timeout)
    603         if not self.ready():
--> 604             raise TimeoutError
    605         if self._success:
    606             return self._value

TimeoutError:

To Reproduce
I'm using the mxnet docker image: mxnet/python:nightly_gpu_cu102_mkl_py3.
The notebook to reproduce can be find in d2l.ai - Ch. 7.1.
Environment
We recommend using our script for collecting the diagnositc information. Run the following command and paste the outputs below:
curl --retry 10 -s https://raw.githubusercontent.com/dmlc/gluon-nlp/master/tools/diagnose.py | python

----------Python Info----------
Version      : 3.5.2
Compiler     : GCC 5.4.0 20160609
Build        : ('default', 'Oct  8 2019 13:06:37')
Arch         : ('64bit', 'ELF')
------------Pip Info-----------
Version      : 20.0.2
Directory    : /usr/local/lib/python3.5/dist-packages/pip
----------MXNet Info-----------
Version      : 1.6.0
Directory    : /usr/local/lib/python3.5/dist-packages/mxnet
Num GPUs     : 1
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform     : Linux-5.3.0-51-generic-x86_64-with-Ubuntu-16.04-xenial
system       : Linux
node         : archmage
release      : 5.3.0-51-generic
version      : #44~18.04.2-Ubuntu SMP Thu Apr 23 14:27:18 UTC 2020
----------Hardware Info----------
machine      : x86_64
processor    : x86_64
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                6
On-line CPU(s) list:   0-5
Thread(s) per core:    1
Core(s) per socket:    6
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 158
Model name:            Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
Stepping:              10
CPU MHz:               3800.000
CPU max MHz:           4000.0000
CPU min MHz:           800.0000
BogoMIPS:              5599.85
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              9216K
NUMA node0 CPU(s):     0-5
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
----------Network Test----------
Setting timeout: 10
Timing for GluonNLP: http://gluon-nlp.mxnet.io, DNS: 0.0358 sec, LOAD: 0.8158 sec.
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0233 sec, LOAD: 0.9208 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.003752470016479492 sec.
Timing for GluonNLP GitHub: https://github.com/dmlc/gluon-nlp, DNS: 0.1257 sec, LOAD: 0.6051 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0428 sec, LOAD: 1.0215 sec.
Timing for D2L: http://d2l.ai, DNS: 0.0392 sec, LOAD: 0.5139 sec.
Timing for D2L (zh-cn): http://zh.d2l.ai, DNS: 0.0375 sec, LOAD: 0.3158 sec.
Timing for FashionMNIST: https://repo.mxnet.io/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0358 sec, LOAD: 0.9466 sec.

Have you tried "            - Slow transform. Please increase timeout to allow slower data loading in each worker."?

Close due to lack of response

Sorry for the late answer. Yes, I tried to increase the timeout to 300, but it didn't work.

have you resolved this issue yet ? I also got this issue after I wrote a gluon.Dataset by myself, but I can' t figure it out

@huzhouxiang I suggest you open a new issue containing a minimal reproducible example

I solved it. I have the same problem in d2l.ai - Ch. 7.7.
Use --shm 1024m to lauch docker, so docker run --shm 1024m <bla bla bla>
why?
gluon.data.DataLoader use python multiprocess, multiprocess need shared memory. The default shared memory is 64m in docker container. You can check the shm usage by using df -h and find shm.


Choose model type based on file extension

You can set the type of model based on file extension instead of forcing it to be always binary, making it a generic embeddings evaluation system.
(https://github.com/devmount/GermanWordEmbeddings/blob/master/evaluation.py#L329)
trained_model = gensim.models.KeyedVectors.load_word2vec_format(args.model.strip(), binary=True)

If it's a text file, then you can set binary as False (it's default value) and if not a text file, you can continue with current flow.

Thanks for the suggestion. You mean something like the following?
is_text = Path(args.model.strip()).suffix == '.txt'
trained_model = gensim.models.KeyedVectors.load_word2vec_format(args.model.strip(), binary=is_text)

And since fastText outputs a .vec file which has structure as any other .txt file
<vocabulary_size> <dimension_size>
<string> <vector>
...
...

Ok, feel free to create a pull request - I will check and merge it than.

Thanks for your request, I merged it and also added .model and no suffix at all as binary files for compatibility with my pretrained model.


Close inactive branches

Some branches seem far behind the develop branch, consider closing them for clarity.

There are no inactive branches at this moment


14

Codecov Report

Merging #16 (fcda92d) into main (7bf4166) will increase coverage by 12.23%.
The diff coverage is 87.50%.

@@             Coverage Diff             @@
##             main      #16       +/-   ##
===========================================
+ Coverage   62.32%   74.56%   +12.23%
===========================================
  Files           3        4        +1
  Lines         146      287      +141
===========================================
+ Hits           91      214      +123
- Misses         55       73       +18

Impacted Files
Coverage Δ

ifrn_estatistica/descriptive_table.py
89.69% <82.85%> (-8.14%)
:arrow_down:

ifrn_estatistica/plotting_graphs.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7bf4166...fcda92d. Read the comment docs.


How to implement filters?

I suppose, that it's often use-case, but still, I can't find an appropriate approach.
I have a model and the GET handler that have to return the list of objects.
This handler has to be able to filter (>, <, >=, <=, =, between, contains) the result by any field of the model.
For the GET handlers generally usable query parameters.
How to describe this functionality in swagger?
What the preferred URI schema?
Generally (without swagger/openapi), I used to use "name_of_field[operation]=value", but I didn't find a way to describe this in swagger 2.0.
Previously I also used the openapi 3.0, it allows using of the object type in query parameters, but 2.0 isn't.
Is there a best practice?
Or should I manually describe all permutations of fields and operations?

One way to model it is to use an array with a pipe delimiter or something.
so your query string becomes something like:
name_of_field=operator|value


[SPARK-15095] [SQL] remove HiveSessionHook from ThriftServer

What changes were proposed in this pull request?
Remove HiveSessionHook
How was this patch tested?
No tests needed.

Test build #57678 has started for PR 12881 at commit 9709675.

Test build #57678 has finished for PR 12881 at commit 9709675.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57678/
Test PASSed.

cc @rxin

Thanks - merging in master/branch-2.0.

Are there any planned hooks for SparkSession? If so, why were the Hive hooks removed prior to replacements?


Bug report，cannot run BeamDyn standalone

Bug description
HI, I trying to run BeamDyn standalone and to do some "FSI" thing, but the first step I try to run BeamDyn standalone. Follow the instructions given by the manual, I was unsuccessful to do it. The warning given in the DOS window is showed below.  If it is a very simple or basic question I must apologies for your times to help me.
Thanks for any help given.
P.S. I download the BeamDyn from the nrel website and all the files are exit.
DOS shows:
C:\Users\iammd>D:\Research_data\LUZEHUI\BeamDyn\bin\BeamDyn_Driver_Win32.exe D:\Research_data\LUZEHUI\BeamDyn\bin\Dvr_5MW_Dynamic.inp

BeamDyn Driver (v1.00.02, 5-Apr-2016)
Copyright (C) 2016 National Renewable Energy Laboratory
This program comes with ABSOLUTELY NO WARRANTY. See the "license.txt" file distributed with this
software for details.

Running BeamDyn Driver (v1.00.02, 5-Apr-2016)
linked with NWTC Subroutine Library (v2.09.00, 23-Jul-2016)
Running BeamDyn (v1.01.04, 26-Jul-2016).
BD_Init:BD_ReadInput:BD_ReadPrimaryFile:OpenFInpFile:The input file,
"Dynamic_BeamDyn_Input_5MW.inp", was not found.
BeamDyn Driver encountered simulation error level: FATAL ERROR
Aborting program.

The BeamDyn driver is not finding your input file.  Do you actually have one located at D:\Research_data\LUZEHUI\BeamDyn\bin\Dvr_5MW_Dynamic.inp?

About:" do you actually have one located at D:\Research_data\LUZEHUI\BeamDyn\bin\Dvr_5MW_Dynamic.inp? "Yes, I do have one. it shows like

About:"did you intend the input file to be on theC:\ drive instead?" No,i think it can run successful no matter the BeamDyn was in C or D disc. The Dos windows show C:\Users\iammd> is what it is, i mean when the dos was opened, the C:\User\iammd> show as well.
And then i try to move the program to C:\ and see if it can run. I install the BeamDyn in C disc and run it in Dos, and the warning shows as before.

BeamDyn in C disc

Dos warning:

The driver input file (Dvr_5MW_Dynamic.inp) does exist in your directory, but the primary input file you're passing in (Dynamic_BeamDyn_Input_5MW.inp) does not.
Just to be clear, the error message from BeamDyn is telling you that it cannot read the primary BeamDyn file. When using the driver, you have a driver input file and then another separate input file that corresponds to BeamDyn library itself. You need to put the path to the primary input file inside the driver input file.

Thank you rafmudaf and thank you Andrew
Follow your advice.
I have solve this problem by using txt-edit software to change the final line of (Dvr_5MW_Dynamic.inp)
"Dynamic_BeamDyn_Input_5MW.inp"    InputFile - Name of the primary input file
To
"D:\Research_data\LUZEHUI\BeamDyn\bin\Dynamic_BeamDyn_Input_5MW.inp"    InputFile - Name of the primary input file
I mean add the path of (Dynamic_BeamDyn_Input_5MW.inp) in the drive input file. Hope this can help people face this problem.
Thanks again.


Local pull request success status testing

Docs Build status updates of commit 69d989c:
:clock10: Pending: waiting for processors (11 builds ahead of you)
:warning: Docs Build is busy, currently there are 11 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit 69d989c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

E2E_DocsBranch_Dynamic/index.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


BTF section is miss but bpf_spin_lock need it

When use bpf_spin_lock or bpf_spin_unlock in tc BPF program and load by tc, the verifier will reports

35: (85) call bpf_spin_lock#93
map '' has to have BTF in order to use bpf_spin_lock

Currently, .BTF section that contains BTF type records should be removed before ELF object file is loaded by tc utility. Because, the BTF type records that are generated by rustc have illegal characters in identifiers from the point of view of the Linux kernel.
So the redbpf crate that is userspace runtime library, parses .BTF section and replaces illegal characters in the identifiers with the underscore character. But tc does not do that for us.

the cargo bpf doesn't support tc-BPF program
and
the section

readelf -S egress.elf
There are 10 section headers, starting at offset 0x4f8:

节头：
  [号] 名称              类型             地址              偏移量
       大小              全体大小          旗标   链接   信息   对齐
  [ 0]                   NULL<PHONE_NUMBER>000000  00000000
<PHONE_NUMBER>000000<PHONE_NUMBER>000000           0     0     0
  [ 1] tc_action/egress  PROGBITS<PHONE_NUMBER>000000  00000040
<PHONE_NUMBER>0002c0<PHONE_NUMBER>000000  AX       0     0     8
  [ 2] license           PROGBITS<PHONE_NUMBER>000000  00000300
<PHONE_NUMBER>000004<PHONE_NUMBER>000000   A       0     0     1
  [ 3] version           PROGBITS<PHONE_NUMBER>000000  00000304
<PHONE_NUMBER>000004<PHONE_NUMBER>000000   A       0     0     4
  [ 4] maps              PROGBITS<PHONE_NUMBER>000000  00000308
<PHONE_NUMBER>00001c<PHONE_NUMBER>000000  WA       0     0     4
  [ 5] .eh_frame         PROGBITS<PHONE_NUMBER>000000  00000328
<PHONE_NUMBER>000030<PHONE_NUMBER>000000   A       0     0     8
  [ 6] .symtab           SYMTAB<PHONE_NUMBER>000000  00000358
<PHONE_NUMBER>0000f0<PHONE_NUMBER>000018           9     6     8
  [ 7] .reltc_actio[...] REL<PHONE_NUMBER>000000  00000448
<PHONE_NUMBER>000020<PHONE_NUMBER>000010           6     1     8
  [ 8] .rel.eh_frame     REL<PHONE_NUMBER>000000  00000468
<PHONE_NUMBER>000010<PHONE_NUMBER>000010           6     5     8
  [ 9] .strtab           STRTAB<PHONE_NUMBER>000000  00000478
<PHONE_NUMBER>00007f<PHONE_NUMBER>000000           0     0     1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  p (processor specific)

doesn't contains .BTF, the bpf verifier need .BTF

the tc output is

8 bytes struct bpf_elf_map fixup performed due to size mismatch!
Map 'NAT_MAP' loaded as pinned!

Prog section 'tc_action/egress' rejected: Permission denied (13)!
 - Type:         3
 - Instructions: 88 (0 over limit)
 - License:      GPL

Verifier analysis:

0: (bf) r6 = r1
1: (61) r1 = *(u32 *)(r6 +80)
2: (61) r2 = *(u32 *)(r6 +76)
3: (bf) r8 = r2
4: (07) r8 += 14
5: (2d) if r8 > r1 goto pc+80
 R1_w=pkt_end(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=14,imm=0) R6_w=ctx(id=0,off=0,imm=0) R8_w=pkt(id=0,off=14,r=14,imm=0) R10=fp0
6: (15) if r2 == 0x0 goto pc+79
 R1_w=pkt_end(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=14,imm=0) R6_w=ctx(id=0,off=0,imm=0) R8_w=pkt(id=0,off=14,r=14,imm=0) R10=fp0
7: (71) r3 = *(u8 *)(r2 +13)
8: (67) r3 <<= 8
9: (71) r4 = *(u8 *)(r2 +12)
10: (4f) r3 |= r4
11: (55) if r3 != 0x8 goto pc+74
 R1=pkt_end(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=14,imm=0) R3=inv8 R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R8=pkt(id=0,off=14,r=14,imm=0) R10=fp0
12: (bf) r7 = r2
13: (07) r7 += 34
14: (2d) if r7 > r1 goto pc+71
 R1=pkt_end(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=34,imm=0) R3=inv8 R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7_w=pkt(id=0,off=34,r=34,imm=0) R8=pkt(id=0,off=14,r=34,imm=0) R10=fp0
15: (71) r3 = *(u8 *)(r2 +23)
16: (55) if r3 != 0x6 goto pc+69
 R1=pkt_end(id=0,off=0,imm=0) R2=pkt(id=0,off=0,r=34,imm=0) R3_w=inv6 R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7_w=pkt(id=0,off=34,r=34,imm=0) R8=pkt(id=0,off=14,r=34,imm=0) R10=fp0
17: (07) r2 += 54
18: (2d) if r2 > r1 goto pc+67
 R1=pkt_end(id=0,off=0,imm=0) R2_w=pkt(id=0,off=54,r=54,imm=0) R3_w=inv6 R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7_w=pkt(id=0,off=34,r=54,imm=0) R8=pkt(id=0,off=14,r=54,imm=0) R10=fp0
19: (69) r1 = *(u16 *)(r7 +0)
20: (55) if r1 != 0xf27 goto pc+65
 R1=inv3879 R2=pkt(id=0,off=54,r=54,imm=0) R3=inv6 R4=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7=pkt(id=0,off=34,r=54,imm=0) R8=pkt(id=0,off=14,r=54,imm=0) R10=fp0
21: (61) r1 = *(u32 *)(r8 +16)
22: (69) r2 = *(u16 *)(r7 +2)
23: (67) r2 <<= 32
24: (4f) r2 |= r1
25: (7b) *(u64 *)(r10 -8) = r2
26: (bf) r2 = r10
27: (07) r2 += -8
28: (18) r1 = 0xffff8f62acbd0000
30: (85) call bpf_map_lookup_elem#1
31: (bf) r9 = r0
32: (55) if r9 != 0x0 goto pc+1

from 32 to 34: R0=map_value(id=0,off=0,ks=8,vs=16,imm=0) R6=ctx(id=0,off=0,imm=0) R7=pkt(id=0,off=34,r=54,imm=0) R8=pkt(id=0,off=14,r=54,imm=0) R9_w=map_value(id=0,off=0,ks=8,vs=16,imm=0) R10=fp0 fp-8=mmmmmmmm
34: (bf) r1 = r9
35: (85) call bpf_spin_lock#93
map '' has to have BTF in order to use bpf_spin_lock
processed 37 insns (limit 1000000) max_states_per_insn 0 total_states 3 peak_states 3 mark_read 2

Error fetching program/map!
Unable to load program

Hello @Sherlock-Holo
Currently, cargo-bpf removes .BTF section only when the ELF object contains tc program.
https://github.com/foniod/redbpf/blob/163103210fcdeec98d8d49a32af4f81cec5e0c73/cargo-bpf/src/llvm.rs#L234-L242
Because .BTF section contains invalid identifier names. The Linux kernel rejects to load the .BTF section generated by rustc.
RedBPF supports BTF by replacing invalid identifier names with valid one when parsing and loading the BTF in runtime. But in tc program case, tc utility directly loads BTF without correcting identifier names so it fails to load the BTF. To prevent this situation, cargo-bpf removes .BTF section beforehand.
I understand the problem you struggle with and I think this problem can be solved. The solution would be that correcting the problematic .BTF section by cargo-bpf while compiling the ELF object file.
I will try this to fix the problem, but I can't make sure the due date.
And thank you for letting me know of this problem. I hope you find workaround for the present.. and RedBPF will support BTF for tc soon.

@Sherlock-Holo
I am working on this issue. I am currently tweaking RedBPF and I succeeded in using bpf_spin_lock with tc.
But there are some issues remained so the code is not ready to be committed yet.
The first issue is that tc command does not support recent version of BTF records. It has not updated BTF related code over for 3 years. Even if some identifier names generated by rustc are fixed by RedBPF to make the BTF pass the BPF verifier, tc rejects the BTF. In particular, tc does not know of BTF_KIND_VAR, BTF_KIND_DATASEC. These are relatively new BTF types but tc just rejects entire BTF info because of those types.
Currently, tc is integrating libbpf as a replacement of its lagacy BPF related code. But most linux distros do not support tc that makes use of libbpf yet. The new code just exists at its git repository.
To solve this issue, I am going to remove particular BTF records that are unknown to tc command.

@Sherlock-Holo
I just sent a PR that solves your issue.

#241


package:integration_test should be able to write data to somewhere Firebase TestLab can see it

For example, we may want to write screenshots, coverage or timeline information to the SD card.
We should make this easier to do in the plugin itself, and document how a user should use it.

@dnfield Could you clarify what FTL is? 😝

Firebase TestLab- updated title


v2 api "greater_than" filter doesn't work as expected

Thanks for submitting an issue to cloud_controller_ng. We are always trying to improve! To help us, please fill out the following template.
Issue
I am playing around with the v2 api with timestamp. When I query for result “greater than” SOME_TIMESTAMP, the api returns results including the event happened right at that  SOME_TIMESTAMP.
Context
Bosh-lite setup
#bosh deployments
RSA 1024 bit CA certificates are loaded due to old openssl compatibility
Acting as user 'admin' on 'Bosh Lite Director'
+-----------------+----------------------------+-----------------------------------------------------+--------------+
| Name            | Release(s)                 | Stemcell(s)                                         | Cloud Config |
+-----------------+----------------------------+-----------------------------------------------------+--------------+
| cf-warden       | cf/253                     | bosh-warden-boshlite-ubuntu-trusty-go_agent/3363.12 | none         |
+-----------------+----------------------------+-----------------------------------------------------+--------------+
| cf-warden-diego | cf-networking/0.18.0+dev.3 | bosh-warden-boshlite-ubuntu-trusty-go_agent/3363.12 | none         |
|                 | cf/253                     |                                                     |              |
|                 | cflinuxfs2-rootfs/1.57.0   |                                                     |              |
|                 | diego/1.10.1               |                                                     |              |
|                 | garden-runc/1.3.0          |                                                     |              |
+-----------------+----------------------------+-----------------------------------------------------+--------------+
Deployments total: 2
Steps to Reproduce
#cf curl "/v2/events?q=type:audit.space.create&q=timestamp>'2017-04-05T23:37:15Z'" | python -m json.tool
I also tried replace '>' with '%3E' but got the same result.
Expected result
"resources" should be an empty list in my case.
Current result
{
"next_url": null,
"prev_url": null,
"resources": [
{
"entity": {
"actee": "7d0471c8-092d-4c41-9ff6-ed614058db2e",
"actee_name": "tryout",
"actee_type": "space",
"actor": "be58c9e6-8b55-4bba-aa9f-3bf8a31cf5ba",
"actor_name": "admin",
"actor_type": "user",
"actor_username": "admin",
"metadata": {
"request": {
"allow_ssh": true,
"name": "tryout",
"organization_guid": "829e1861-344e-4bd6-b14b-14f37bc8aa16"
}
},
"organization_guid": "829e1861-344e-4bd6-b14b-14f37bc8aa16",
"space_guid": "7d0471c8-092d-4c41-9ff6-ed614058db2e",
"timestamp": "2017-04-05T23:37:15Z",
"type": "audit.space.create"
},
"metadata": {
"created_at": "2017-04-05T23:37:15Z",
"guid": "7c44934c-dcd5-4ba5-b92f-6fb55323d4a5",
"updated_at": "2017-04-05T23:37:15Z",
"url": "/v2/events/7c44934c-dcd5-4ba5-b92f-6fb55323d4a5"
}
}
],
"total_pages": 1,
"total_results": 1
}
Possible Fix
[not obligatory, but suggest fixes or reasons for the bug]
name of issue screenshot
[if relevant, include a screenshot]

@jiahaoliang,
Thanks for opening this issue. We analyzed how we are filtering our dataset based on the timestamp query parameter. Turns out, Postgres stores fractional seconds in the database, but we display timestamps only on the second precision. You can check the values of the timestamp column in your database record, and it will contain fractional seconds, and therefore, it will be selected by the ">".
Closing this issue as it is not a bug. Let us know if you have any other concerns.
Side comment: MySql does not store fractional seconds. This seems like just a behavior in Postgres.
Thanks,
CAPI Community Pair - @nicregez && @michaelxupiv


Bugfix:Content is truncated to send markdown msg on wecom webhook(#101)

This PR fixes #101

Thanks for this PR.
And I want to do some discussion because I have some thoughts.
we  have two options

fix this in the underlying markdown() function
fix that in Wecom send function.

because the markdown format need not escape the quote, but the WeCom need because the API wraps the markdown to JSON.
So, what do you think? @jordy1024

Thanks for this PR.
And I want to do some discussion because I have some thoughts.
we have two options

fix this in the underlying markdown() function
fix that in Wecom send function.

because the markdown format need not escape the quote, but the WeCom need because the API wraps the markdown to JSON.
So, what do you think? @jordy1024

Oh,that's true;
before solving this problem, I also thought about whether this problem should be reported to WeChat 😏; if I simply modify markdown(), it feels a bit intrusive and affects its versatility.


Better Flow types and checks

We are currently using Flow but not being too strict about it.
It is quite helpful and also easy to improve.
This is a non-exclusive task - multiple people can work on learning and improving the Flow definitions in the code. Contributions welcome.

Similar to #2052

Yeah, closing this because of duplication.


How to disable Index Titles for SSCoreDataSource?

Is there any way to disable it?
Example:
self.dataSource = SSCoreDataSource(fetchRequest: fetchRequest, inContext: moc, sectionNameKeyPath: "someKey")

You could:

Adjust your data model or fetched results controller to change or remove section titles
Make a subclass of SSCoreDataSource, override sectionIndexTitlesForTableView:, and return nil

@jhersh Thanks. Just as I expected


Keepmenu

Project description
For i3 it is convenient to use Dmenu and Rofi, however it lacks of support for keepass databases.
Keepmenu fully featured Dmenu/Rofi/Bemenu frontend for autotype and managing of Keepass databases.
Metadata

homepage URL: https://github.com/firecat53/keepmenu
source URL: https://github.com/firecat53/keepmenu
license: GPL-3.0 License

I'll try to build this app

Many thanks for the quick reply :)

Any updates on this request?

Waiting for reviewer


About the 2D backbone

Hi, I have some questions about training the TransFusion-LC.

You mentioned in the supplementary materials that a 2D backbone pre-trained on the autonomous driving datasets is required and frozen during training the TransFusion-LC. (i.e., DLA-34 and ResNet-50 pre-trained on the nuScenes and Waymo in repsectively.) However, I don't find relevant pre-trained models in the readme.md of this git, and relevant configuration terms in the config files (e.g., transfusion_nusc_voxel_LC.py).  Or maybe you have provided but I missed something important?

Could you please provide relevant pre-trained 2D backbone models, or relevant instructions of pre-training the 2D backbone models? Thanks a lot!

Hi, sorry it seems I didn't make it clear in the readme.

For DLA34 pretrained on 3D detection, I follow PointAugmenting to reuse the model provided by CenterNet. You can download the checkpoint from https://github.com/xingyizhou/CenterTrack/blob/master/readme/MODEL_ZOO.md#monocular-3d-detection-tracking.
For ResNet50+FPN pretrained on instance segmentation, I use the model provided by mmdet3d, you can download the checkpoints from https://github.com/open-mmlab/mmdetection3d/blob/v0.12.0/configs/nuimages/README.md (note that you should also use the checkpoints provided by mmdet3d v0.12.0). I choose the backbone from MaskRCNN that pretrained only on imagenet (the first one).
For ResNet50+FPN pretrained on 2D detection, I train the model by using the same config file with (2) except removing the mask head.

And I use a similar step as (3) to train a 2D backbone for the waymo dataset. I can send you the relevant processing code and config file if needed.
Best,
Xuyang.

Thanks a lot for your reply! It is really clear!
Could you please send me the relevant code for training the 2D backbone for the waymo dataset if that doesn't bother you? My email is<EMAIL_ADDRESS>

Hi, I have sent them to your email.

Thanks! I received your email.
I still have some questions about re-implementation.

In https://github.com/XuyangBai/TransFusion/blob/ab245436a096e3e4a2fd10d773d75bf1d78fb611/configs/transfusion_nusc_voxel_LC.py#L130, I notice that you comment out DLA-34 image backbone, and replace it with the ResNet-50. I am wondering whether the configuration parameters of DLA-34 is the full version of DLA-34 because I notice that "heads" parameter is set to empty.
 img_backbone=dict(        type='DLASeg',        num_layers=34,        heads={},        head_convs=-1,       ),
I am reproducing the TransFusion-L strictly following your config file and instructions, but the mAP on nuScenes validation set is only 0.5985 at 17-th epoch (the whole training process hasn't be finished yet). I don't know where I went wrong. Could you please send me the training logs for TransFusion-L and TransFusion-LC, and thereby I can compare it with my training log.

Sorry to bother you again. Sincere appreciation!

Hi,

R50 + FPN gives a slightly better result compared with DLA34 (as shown in Table 12 in the supplementary material). And I only use DLA34 as the image feature extractor, so I do not load the task head.
Did you adopt the fade strategy (disenable the copy-and-paste augmentation for the last 5 epochs)?  That can have a remarkable effect on the mAP by reducing the false positive.

Best,
Xuyang

Oh, I got it. I forget to adopt the fade strategy for the last 5 epochs.
Besides, I found that the NDS value is always lower than mAP in my present validation process.
e.g.,

mAP 0.5199; NDS 0.4856 at epoch 5;
mAP 0.5606; NDS 0.5244 at epoch 10;
mAP 0.5895; NDS 0.5453 at epoch 15
I don't know if this is a normal phenomenon. I observe that the NDS value is generally higher than the mAP values. Could you please provide some valuable suggestions or point out where am I possibly wrong?
Thanks!

Sincerely,
Jay

It is not normal, could you provide the full results such as mATE, mAOE, mASE?

It can have a very bad mAOE abd mASE if you use the newest version mmdet3d to generate the .pkl and then train TransFusion. mmdet3d has a large coordinate system refactoring in newer version. See https://github.com/open-mmlab/mmdetection3d/blob/master/docs/en/compatibility.md#coordinate-system-refactoring

OK, I list the TP metrics results as below:
at epoch 19 (without the fade strategy), mATE=0.2839; mASE=0.7090, mAOE=1.5609; mAVE=0.2707; mAAE=0.1913

It can have a very bad mAOE and mASE if you use the newest version mmdet3d to generate the .pkl and then train TransFusion. mmdet3d has a large coordinate system refactoring in the newer version. See https://github.com/open-mmlab/mmdetection3d/blob/master/docs/en/compatibility.md#coordinate-system-refactoring

I think this might be the key to my problem! Since I create the meta data of nuscenes with the newest released mmdet3d, and degrade its version after I find the version mismatching with the mmdet3d of the TransFusion github.
Thanks for your valuable advice! I will re-create the metadata and see what will happen.

Nice discussion above! Hi @XuyangBai, I have a follow-up question regarding the training of LC model.
To load the TransFusion-L model when training the -LC model, should we change the load_from key in the config file into the -L model checkpoint, or should we leave that part empty but change the pretrained key in the TransFusionDetector field instead?

Hi @YunzeMan I usually use the follow code to combine the pretrained TransFusion-L and the 2D backbone
img = torch.load('img_backbone.pth', map_location='cpu')
pts = torch.load('transfusionL.pth', map_location='cpu')
new_model = {"state_dict": pts["state_dict"]}
for k,v in img["state_dict"].items():
    if 'backbone' in k or 'neck' in k:
        new_model["state_dict"]['img_'+k] = v
        print('img_'+k)
torch.save(new_model, "fusion_model.pth")

And then set the load_from key to load both the pretrained 3D backbone and 2D backbone.

Hi,  @XuyangBai @SxJyJay, it needs 4 days for me to train a TransFusion-L (8 V100 GPUS, epoch=20, samples_per_gpu=2), which seems too long. How long did you spend training TransFusion-L？Thanks!!

@WWW2323 about 2 days for me using 8V100 GPUs

Hi, @XuyangBai @SxJyJay, it needs 4 days for me to train a TransFusion-L (8 V100 GPUs, epoch=20, samples_per_gpu=2), which seems too long. How long did you spend training TransFusion-L？Thanks!!

Also about 2 days for me using 8 RTX3090 GPUs.

@XuyangBai Hi, I have finished the whole training process of TransFusion. I make no modifications except for replacing the DLA-34 to ResNet50+FPN as you suggested. And the final results on the nuscenes validation set are:
mAP=67.25, NDS=70.89, mATE=28.09, mASE=25.30, mAOE=28.58, mAVE=26.26, mAAE=19.15
The mAP and NDS are a little lower than the results on the nuscenes test set reported in the paper. Conventionally, I think the results on the test set are lower than those on the validation set.
Besides, I find that the mAP drop may be caused by much lower AP of some classes such as trailer, traffic cone and barrier.. I list AP of my results (on val set) vs reported results (on test set) below:
car(87.9 vs 87.1), truck(64.0 vs 60.0), bus(74.1 vs 68.3), trailer(43.5 vs 60.8), construction_vehicle(29.8 vs 33.1), pedestrian(88.3 vs 88.4), motorcycle(74.3 vs 73.6), bike(63.5 vs 52.9), traffic cone(77.1 vs 86.7), barrier(70.1 vs 78.1)
I don't know whether my results are within an acceptable error margin. Or such results are caused by the bias of different image backbones (i.e., DLA-34 and ResNet50+FPN)?

Hi @SxJyJay, You can see the detailed results on val set below.
mAP: 0.6727
mATE: 0.2721
mASE: 0.2517
mAOE: 0.2740
mAVE: 0.2536
mAAE: 0.1902
NDS: 0.7122

Per-class results:
Object Class    AP      ATE     ASE     AOE     AVE     AAE
car     0.876   0.169   0.148   0.085   0.259   0.185
truck   0.620   0.302   0.182   0.102   0.228   0.221
bus     0.757   0.302   0.186   0.048   0.386   0.256
trailer 0.428   0.520   0.209   0.463   0.185   0.163
construction_vehicle    0.274   0.666   0.417   0.833   0.124   0.318
pedestrian      0.878   0.128   0.282   0.360   0.215   0.097
motorcycle      0.754   0.184   0.244   0.215   0.421   0.267
bicycle 0.631   0.150   0.263   0.300   0.212   0.016
traffic_cone    0.770   0.119   0.304   nan     nan     nan
barrier 0.739   0.182   0.281   0.059   nan     nan

I think it is within an acceptable error margin. The slightly worse performance might be coming from the training variance.  For the gap between validation and test set, it is normal because generally they are having different distributions. Also, you could try using more queries during inference to get a better result with a longer inference time (see Table 13 in the supplementary)

Hello @XuyangBai, I want to use your results on nuscenes validation set to do object tracking experiment, but I don't have enough computing power for training. I wonder if you could provide json files of the validation set results?  Here is my email<EMAIL_ADDRESS>Looking forward to your reply!

Thank you. On the validation set, the performance I re-produced seems close to yours.
 Hence, I close this issue.
Thanks again for your patience!

Hi, I have sent them to your email.

Hi, I also plan to train  2D backbone for waymo and nuscenes, Could you please send me the relevant code for training the 2D backbone? It will be helpful! My email is<EMAIL_ADDRESS>

Hi, @XuyangBai @SxJyJay, it needs 4 days for me to train a TransFusion-L (8 V100 GPUs, epoch=20, samples_per_gpu=2), which seems too long. How long did you spend training TransFusion-L？Thanks!!

Also about 2 days for me using 8 RTX3090 GPUs.

Hi, could you please provide the environment of your CUDA, PyTorch MMCV, mmdet, mmdet3d, because I am training on 4*A100 and the display takes 20 days, which makes me confused, I want to exclude the influence of the environment
`sys.platform: linux
Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0]
CUDA available: True
GPU 0,1,2,3: NVIDIA A100-SXM4-40GB
CUDA_HOME: /public/home/u212040344/usr/local/cuda-11.1
NVCC: Build cuda_11.1.TC455_06.29069683_0
GCC: gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
PyTorch: 1.8.0
PyTorch compiling details: PyTorch built with:

GCC 7.3
C++ Version: 201402
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.1
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.0.5
Magma 2.5.2
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0
OpenCV: 4.5.5
MMCV: 1.3.18
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.1
MMDetection: 2.11.0
MMDetection3D: 0.12.0+5337046`

Hi, @XuyangBai @SxJyJay, it needs 4 days for me to train a TransFusion-L (8 V100 GPUs, epoch=20, samples_per_gpu=2), which seems too long. How long did you spend training TransFusion-L？Thanks!!

Also about 2 days for me using 8 RTX3090 GPUs.

Hi, could you please provide the environment of your CUDA, PyTorch MMCV, mmdet, mmdet3d on 3090 GPUs, because I am training on 4*A100 and the display takes 20 days, which makes me confused, I want to exclude the influence of the environment `sys.platform: linux Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0] CUDA available: True GPU 0,1,2,3: NVIDIA A100-SXM4-40GB CUDA_HOME: /public/home/u212040344/usr/local/cuda-11.1 NVCC: Build cuda_11.1.TC455_06.29069683_0 GCC: gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5) PyTorch: 1.8.0 PyTorch compiling details: PyTorch built with:

GCC 7.3
C++ Version: 201402
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.1
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.0.5
Magma 2.5.2
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0 OpenCV: 4.5.5 MMCV: 1.3.18 MMCV Compiler: GCC 7.3 MMCV CUDA Compiler: 11.1 MMDetection: 2.11.0 MMDetection3D: 0.12.0+5337046`

Hi, @XuyangBai @SxJyJay, it needs 4 days for me to train a TransFusion-L (8 V100 GPUs, epoch=20, samples_per_gpu=2), which seems too long. How long did you spend training TransFusion-L？Thanks!!

Also about 2 days for me using 8 RTX3090 GPUs.

Hi, could you please provide the environment of your CUDA, PyTorch MMCV, mmdet, mmdet3d on 3090 GPUs, because I am training on 4*A100 and the display takes 20 days, which makes me confused, I want to exclude the influence of the environment `sys.platform: linux Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0] CUDA available: True GPU 0,1,2,3: NVIDIA A100-SXM4-40GB CUDA_HOME: /public/home/u212040344/usr/local/cuda-11.1 NVCC: Build cuda_11.1.TC455_06.29069683_0 GCC: gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5) PyTorch: 1.8.0 PyTorch compiling details: PyTorch built with:

GCC 7.3
C++ Version: 201402
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.1
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.0.5
Magma 2.5.2
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0 OpenCV: 4.5.5 MMCV: 1.3.18 MMCV Compiler: GCC 7.3 MMCV CUDA Compiler: 11.1 MMDetection: 2.11.0 MMDetection3D: 0.12.0+5337046`

Hi, my runtime environment is shown below:
 - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for
Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff68
3)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arc
h=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=com
pute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_
75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,co
de=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.0.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUD
NN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLA
GS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -D
NDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XN
NPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing
-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -
Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-funct
ion -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-s
trict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno
-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-ca
st -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno
-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-s
tringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WI
TH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PT
R=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, US
E_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0
OpenCV: 4.5.5
MMCV: 1.3.0
MMCV Compiler: GCC 7.5
MMCV CUDA Compiler: 11.1
MMDetection: 2.10.0
MMDetection3D: 0.11.0+

Besides, I think you can check time consumed on fetching data and run one forward pass to identify where is the bottleneck. Maybe your problem is caused by slow io.

thank you for your reply! The strange thing is that my GPU usage has been maintained at 100, basically will not jump back and forth, I don't know if this can mean that the Speed of CPU loading data is normal？

Hi, @XuyangBai @SxJyJay, it needs 4 days for me to train a TransFusion-L (8 V100 GPUs, epoch=20, samples_per_gpu=2), which seems too long. How long did you spend training TransFusion-L？Thanks!!

Also about 2 days for me using 8 RTX3090 GPUs.

Hi, could you please provide the environment of your CUDA, PyTorch MMCV, mmdet, mmdet3d on 3090 GPUs, because I am training on 4*A100 and the display takes 20 days, which makes me confused, I want to exclude the influence of the environment `sys.platform: linux Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0] CUDA available: True GPU 0,1,2,3: NVIDIA A100-SXM4-40GB CUDA_HOME: /public/home/u212040344/usr/local/cuda-11.1 NVCC: Build cuda_11.1.TC455_06.29069683_0 GCC: gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5) PyTorch: 1.8.0 PyTorch compiling details: PyTorch built with:

GCC 7.3
C++ Version: 201402
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.1
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.0.5
Magma 2.5.2
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0 OpenCV: 4.5.5 MMCV: 1.3.18 MMCV Compiler: GCC 7.3 MMCV CUDA Compiler: 11.1 MMDetection: 2.11.0 MMDetection3D: 0.12.0+5337046`

Hi, my runtime environment is shown below:
 - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for
Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff68
3)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arc
h=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=com
pute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_
75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,co
de=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.0.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUD
NN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLA
GS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -D
NDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XN
NPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing
-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -
Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-funct
ion -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-s
trict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno
-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-ca
st -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno
-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-s
tringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WI
TH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PT
R=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, US
E_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0
OpenCV: 4.5.5
MMCV: 1.3.0
MMCV Compiler: GCC 7.5
MMCV CUDA Compiler: 11.1
MMDetection: 2.10.0
MMDetection3D: 0.11.0+

Besides, I think you can check the time consumed on fetching data and running one forward pass to identify where is the bottleneck. Maybe your problem is caused by slow io.

thanks for your reply! The strange thing is that my GPU usage has been maintained at 100, basically will not jump back and forth, I don't know if this can mean that the Speed of CPU loading data is normal？

@SxJyJay Hi, can you provide the trained TransFusion and TransFusion-L model? My re-produced result is 63.9 mAP (Lidar) and 64.4 mAP(Lidar+Camera), which is strange. Thanks so much!

@wzmsltw Hi, you can leave me your email, and I will send checkpoints to you.

@SxJyJay my email address is<EMAIL_ADDRESS>Thanks so much for your help!

@SxJyJay Hi, when will you send checkpoints? Really looking forward to it. Thanks again~

@SxJyJay Hi, when will you send checkpoints? Really looking forward to it. Thanks again~

Sorry for the delay. I have something urgent yesterday. I have send you!
Best,
Yang Jiao

@SxJyJay Could you provide the trained TransFusion and TransFusion-L model? I only have one GTX 3080  and it is very difficult to train such a network. My email is<EMAIL_ADDRESS>Thanks so much

@SxJyJay Hi, when will you send checkpoints? Really looking forward to it. Thanks again~

Sorry for the delay. I have something urgent yesterday. I have send you! Best, Yang Jiao

hi，Could you shared the model of Transfunsion L and Transfusion LC  based on nuscenes? I have only a GPU device of 3080, so training may take a long time. Thank you very much<EMAIL_ADDRESS>

@SxJyJay Hi, when will you send checkpoints? Really looking forward to it. Thanks again~

Sorry for the delay. I have something urgent yesterday. I have send you! Best, Yang Jiao

hi，Could you shared the model of Transfunsion L and Transfusion LC based on nuscenes? I have only a GPU device of 3080, so training may take a long time. Thank you very much<EMAIL_ADDRESS>
Hi, I also plan to train 2D backbone for waymo and nuscenes, Could you please send me the relevant data processing code for training the 2D backbone? It will be helpful! My email is<EMAIL_ADDRESS>

@SxJyJay Could you provide the trained TransFusion and TransFusion-L model? I only have one GTX 3080 and it is very difficult to train such a network. My email is<EMAIL_ADDRESS>Thanks so much

Hello, I am also looking into this project but I don't have the hardware right now for the training. I would like to humbly ask if you can share with me the checkpoints for trained TransFusion and TransFusion-L models? It will be very helpful. Thank you so much! My email address is<EMAIL_ADDRESS>

Hi, sorry it seems I didn't make it clear in the readme.

For DLA34 pretrained on 3D detection, I follow PointAugmenting to reuse the model provided by CenterNet. You can download the checkpoint from https://github.com/xingyizhou/CenterTrack/blob/master/readme/MODEL_ZOO.md#monocular-3d-detection-tracking.
For ResNet50+FPN pretrained on instance segmentation, I use the model provided by mmdet3d, you can download the checkpoints from https://github.com/open-mmlab/mmdetection3d/blob/v0.12.0/configs/nuimages/README.md (note that you should also use the checkpoints provided by mmdet3d v0.12.0). I choose the backbone from MaskRCNN that pretrained only on imagenet (the first one).
For ResNet50+FPN pretrained on 2D detection, I train the model by using the same config file with (2) except removing the mask head.

And I use a similar step as (3) to train a 2D backbone for the waymo dataset. I can send you the relevant processing code and config file if needed.
Best, Xuyang.

Hello, thank you for the work. I want to reproduce the TransFusion LC, can you explain more about (3), what the "mask head" in the config file represents?  In other words, where can I find it and how can I remove it?

@heminghuang7  You can comment the following part out:
https://github.com/XuyangBai/TransFusion/blob/399bda09a3b6449313ccc302df40651f77ec78bf/configs/_base_/models/mask_rcnn_r50_fpn.py#L56-L66

@heminghuang7 You can comment the following part out:
https://github.com/XuyangBai/TransFusion/blob/399bda09a3b6449313ccc302df40651f77ec78bf/configs/_base_/models/mask_rcnn_r50_fpn.py#L56-L66

Thank you so much!

@maokp @kuangpanda @cxd520314wang I have send you my reproduced checkpoints! Please check up your email!

@maokp @kuangpanda @cxd520314wang I have send you my reproduced checkpoints! Please check up your email!

@SxJyJay Could you please also send a copy of the model of Transfunsion L and Transfusion LC on nuscenes? I am struggling to reproduce the performances. It couldn't be better if you could also share the training logs. Thank you very much<EMAIL_ADDRESS>

@maokp @kuangpanda @cxd520314wang I have send you my reproduced checkpoints!  Please check up your email!

hello, could you also sent me the checkpoints and training logs for TransFusion_L and TransFusion? Thank you very much!!!! My email is<EMAIL_ADDRESS>

@maokp @kuangpanda @cxd520314wang I have send you my reproduced checkpoints! Please check up your email!
@SxJyJay hello,I plan to reproduce results on nuscenes, Could you please send me the checkpoint for training the 2D backbone? It will be helpful! My email is<EMAIL_ADDRESS>

@maokp @kuangpanda @cxd520314wang I have send you my reproduced checkpoints! Please check up your email!

@SxJyJay  Hi, I am a PhD studenet aiming to study the lidar-camera detection models. I've tried many times but I still cannot reproduce satisfing results. Could you please send me your checkpoints? Really looking forward to it. Thanks!

your
Hi, I also plan to train 2D backbone for waymo and nuscenes. Could you please send me the relevant code for training the 2D backbone for the waymo and nuscenes dataset if that doesn't bother you? (specifically the waymo dataset) My email is<EMAIL_ADDRESS>

Hi, I have sent them to your email.

Hi, I also plan to train 2D backbone for waymo and nuscenes. Could you please send me the relevant code for training the 2D backbone for the waymo and nuscenes dataset if that doesn't bother you? (specifically the waymo dataset) My email is<EMAIL_ADDRESS>

Hi， I'm also trying to reproduce TransFusion-L but my mAP and NDS (60.34 & 66.46) are much lower than the author's. Could you please send me your training log of TransFusion-L? I notice an obvious drop of loss at epoch 16 when fade strategy is applied in other's training. But mine seems no difference between with and w/o fade strategy. Thank you! My mail is<EMAIL_ADDRESS>

@JamesHao-ml @yangsijing1995 @wangyd-0312 @Young98CN @zzj403 @jqfromsjtu Hi, I have sent checkpoints to u. Sorry for late reply, as I just finish a DDL.

@xpyqiubai @xxlbigbrother @kuangpanda Hi, I have sent data processing code for waymo and kitti to u. Sorry for late reply.

@xpyqiubai @xxlbigbrother @kuangpanda Hi, I have sent data processing code for waymo and kitti to u. Sorry for late reply.

Thanks!

@SxJyJay Hi SxJyJay, can you send the trained checkpoints on nuscenes and waymo to me? I need the trained TransFusion and
TransFusion-L model as well as the relevant data processing code. It would be greatly helpful for me since I may not have enough machines to train it by myself. Thank you very much! My email is<EMAIL_ADDRESS>

<EMAIL_ADDRESS>
I have sent relevant checkpoints and data processing code to your email.

@SxJyJay Hi SxJyJay, can you send the trained checkpoints on nuscenes to me? I need the trained TransFusion and TransFusion-L model as well as the relevant data processing code. It would be greatly helpful for me since I may not have enough machines to train it by myself. Thank you very much! My email is<EMAIL_ADDRESS>
I have sent relevant checkpoints and data processing code to your email.

Thank you very much!

Hi, @SxJyJay, I have reproduce the Transfusion-L with mAP 65.4, however, the reproduced Transfusion-LC model can only achive mAP 65.6, which has a large gap between yours(67.25). Can you send me your training log and checkpoint of both Transfusion-L and Transfusion-LC so I can check where went wrong, my email is<EMAIL_ADDRESS>Thank you!

Hi, @SxJyJay, I have reproduce the Transfusion-L with mAP 65.4, however, the reproduced Transfusion-LC model can only achive mAP 65.6, which has a large gap between yours(67.25). Can you send me your training log and checkpoint of both Transfusion-L and Transfusion-LC so I can check where went wrong, my email is<EMAIL_ADDRESS>Thank you!

Hi, I have sent you relevant pretrained weights.

Thanks a lot

@SxJyJay Hi SxJyJay, the reproduced Transfusion-LC model of mine is so low. Could you please send the trained checkpoints on nuscenes to me? I need the trained TransFusion and TransFusion-L model as well as the relevant data processing code. Thank you very much! My email is<EMAIL_ADDRESS>

@SxJyJay Hi, could you send me the checkpoint to me? I need the trained TransFusion-L,
TransFusion and 2D backbone as well as the data processing code. My email is<EMAIL_ADDRESS>

@SxJyJay Hi, can you provide the trained TransFusion and TransFusion-L model?
I am a novice and want to use 1 GPU to run through this verification and test model to learn.
Thanks so much! My email is<EMAIL_ADDRESS>

Hi, @SxJyJay, I have reproduce the Transfusion-L with mAP 65.4, however, the reproduced Transfusion-LC model can only achive mAP 65.6, which has a large gap between yours(67.25). Can you send me your training log and checkpoint of both Transfusion-L and Transfusion-LC so I can check where went wrong, my email is<EMAIL_ADDRESS>Thank you!

Hi, I have sent you relevant pretrained weights.
Hello, SxJyJay, I really need a well-trained and trained model. I would be very grateful if you could send it to me. I look forward to your help. My email is<EMAIL_ADDRESS>

Hi, @SxJyJay I am trying to reproduce the Transfusion-L. But I can‘t reach the results.
could you send me your checkpoints?
My email is<EMAIL_ADDRESS>Thank you so much for your work!

Hi, @minrui-hust @yichen928 @JamesHao-ml @yangsijing1995 @wangyd-0312 @Young98CN @zzj403 @jqfromsjtu @maokp @kuangpanda @cxd520314wang If you could please forward me the checkpoints for reproduced results I will be very thankful. My email is<EMAIL_ADDRESS>Thank you very much for your help

I upload my reproduced checkpoints to Google Drive. You can get access them using the following links:

Hi @SxJyJay Thank you very much!

@SxJyJay Thank you so much for you kind sharing!

@SxJyJay I am trying to reproduce the Transfusion-L. But I can‘t reach the results.
could you send me your train log or checkpoints? Thanks a lot.
<EMAIL_ADDRESS>

@maokp @kuangpanda @cxd520314wang I have send you my reproduced checkpoints! Please check up your email!

@maokp @kuangpanda @cxd520314wang @SxJyJay I am interested in training a 2D backbone on Waymo dataset. Could you share the relevant code and checkpoints with me on<EMAIL_ADDRESS>thanks in advance!

@heminghuang7 You can comment the following part out:
https://github.com/XuyangBai/TransFusion/blob/399bda09a3b6449313ccc302df40651f77ec78bf/configs/_base_/models/mask_rcnn_r50_fpn.py#L56-L66

Thank you so much!

After commenting out that part, I get an error
 '`cfg` or `default_args` must contain the key "type", '
KeyError: '`cfg` or `default_args` must contain the key "type", but got {\'num_classes\': 10}\nNone'

How did you solve that?

@xpyqiubai @xxlbigbrother @kuangpanda @SxJyJay I'm interested to train a 2D backbone on Waymo dataset. Can you please share the relevant code and checkpoints (if possible) to<EMAIL_ADDRESS>Thank you very much!

I upload my reproduced checkpoints to Google Drive. You can get access them using the following links: TransFusion-L: https://drive.google.com/file/d/1J7fTYsfqRovIdKPenEG5OHQObKq-tfrl/view?usp=sharing TransFusion-LC: https://drive.google.com/file/d/1mv_JH0gqC3SrUZ9ik9qPEBlgqeeCh9Tb/view?usp=sharing

This is fantastic, thank you so much for sharing!

我将复制的检查点上传到 Google 云端硬盘。您可以使用以下链接访问它们：TransFusion-L：https://drive.google.com/file/d/1J7fTYsfqRovIdKPenEG5OHQObKq-tfrl/view?usp=sharing TransFusion-LC：https://drive.google.com/file/d/1mv_JH0gqC3SrUZ9ik9qPEBlgqeeCh9Tb/view?usp=sharing

Hello, thank you very much for sharing, this is very helpful for me who only has a GPU, I also want to see the parameters after training, so can you send me a Transfusion work_dir file, thank you very much<EMAIL_ADDRESS>

@xpyqiubai @xxlbigbrother @kuangpanda @SxJyJay Could you please provide the necessary code and any available checkpoints for training a 2D backbone on the Waymo dataset? If possible, send the information to<EMAIL_ADDRESS>Your assistance is greatly appreciated!

I upload my reproduced checkpoints to Google Drive. You can get access them using the following links: TransFusion-L: https://drive.google.com/file/d/1J7fTYsfqRovIdKPenEG5OHQObKq-tfrl/view?usp=sharing TransFusion-LC: https://drive.google.com/file/d/1mv_JH0gqC3SrUZ9ik9qPEBlgqeeCh9Tb/view?usp=sharing

are these the checkpoints trained on waymo? if not could also sent the checkpoints to<EMAIL_ADDRESS>Thank you!


Thanks for open source. Two issues.

The background color of the horizontal scroll bar is incorrect, it should be transparent.

In the sidebar, the background color of some special directories feels a bit awkward and not very good-looking. Can you provide cancellation or customization.

The background color of phpstorm just started does not match the theme color, I hope it can be fixed.

Thanks for the comments! Could you clarify the build number of your IDE?
You can find it in the menu: Help - About.
For example:
Build #PS-221.5591.58, built on May 12, 2022

Thanks for the feedback! Could you clarify the build number of your IDE? You can find it in the menu: Help - About.
For example: Build #PS-221.5591.58, built on May 12, 2022

@varrcan Thank you for taking the time to answer. I also want to try to make a set of colors that I like. I want to use your project as a starting template. I have installed IntelliJ IDEA, and successfully modified the color and previewed it successfully, but I am a PHPer, I don't know Java at all, no I know how to export the modified result, or build it into a plugin that can be installed. If you have time, I hope you can guide me roughly how to build the project.

@varrcan Thank you for taking the time to answer. I also want to try to make a set of colors that I like. I want to use your project as a starting template. I have installed IntelliJ IDEA, and successfully modified the color and previewed it successfully, but I am a PHPer, I don't know Java at all, no I know how to export the modified result, or build it into a plugin that can be installed. If you have time, I hope you can guide me roughly how to build the project.

Unfortunately, it will not be possible to build a plugin through PHPStorm. But you can download the free version of IntelliJ IDEA Community Edition, it only supports Java. To build a plugin from source files, knowledge of Java is not necessary, everything is done directly from the IDE interface. You need a Gradle plugin and java installed on the system for this.
And there you can immediately view the changes you have made in live-mode.
After the plugin is built (it will be built .jar file), you can install the plugin in the IDE directly from the disk

.

@varrcan thanks, Where can I get the latest and complete settings for theme.json。all parameters of the ui object.

@varrcan  Hello, I have reported these three problems, have you found a solution? If you can solve these three problems, then it will be perfect, thank you very much.

@varrcan Hello, I have reported these three problems, have you found a solution? If you can solve these three problems, then it will be perfect, thank you very much.

Hello. There will be a small update in a couple of days, which should fix the problem with the sidebar.

On the second point. This is the standard behavior of the IDE, you can turn off the highlighting of the tests folder in the settings

On the third point. I didn't find an option to change the background color when starting the IDE. If I succeed in the future, I will definitely change it.
Thank you for your feedback!

@varrcan The issue that the background color of the horizontal scroll bar is not transparent has not been fixed.

@varrcan The issue that the background color of the horizontal scroll bar is not transparent has not been fixed.

hmm, maybe this is a feature of the OS. are you using a Mac? can you tell the version?

@varrcan


chore(dashboard): improve api performance

SUMMARY

The fields table_names and datasource were originally included in these dashboard APIs to avoid breaking changes, but they cause significant performance bottlenecks on the API so after further testing it seems we can remove them.
TESTING INSTRUCTIONS

Full test of all dashboard UI elements
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

Closing in favor of #16110


Moves the 'stopping machine' log below error log

As it doesn't make sense to first tell that the machine will be stopped for upgrade, but then throw an error that it isn't possible at all, I moved the 'stopping machine' info log below the error log.
This seems to be a legit change, as the machine isn't stopped at all...
Fixes #1032.

Fixed it @ehazlett :blush:

Thanks!
LGTM


Erroneous example in cron trigger documentation

The Cron Trigger Tutorial gives the following example:

So cron expressions can be as simple as this: * * * * ? *

The special character ? is not legal in the fifth field (month). It should probably be

So cron expressions can be as simple as this: * * * * * ?

Fixed by PR #371


Update for new hosting bundle installer link

@Rick-Anderson @shirhatti
Looks like the new hosting bundle installer is under ...
https://go.microsoft.com/fwlink/?linkid=832756
... if that's the correct link, I'll get a pair of update PR's in now for Publishing to IIS and the ANCM doc.

Your comment has the 1.1.0-preview1 installer.
The correct link is https://aka.ms/dotnetcore_windowshosting_1_1_0

Ah ... gotcha. That link that I show came from the ANCM repo README. I'll use the link you sent and get the PR's in shortly.

Fixed by https://github.com/aspnet/Docs/commit/ce841fa678ab3f1405847e42fd68d2504e35d0b1 and https://github.com/aspnet/Docs/commit/2387ba5964cdc4753027c0ee460aa0387c1c4cfb


No wifi in melcloud app

Although I have access via ping to the network card of the air conditioner, the melcloud app tells me that the unifad cannot be updated and in the reports section, if I ask for a report of the strength of the wifi signal it tells me that there is no signal. But if I ping the internal ip, it does respond to me. I know this isn't a plugin problem but I'd like to know if this happens to anyone else (I hope it does and that it's a widespread problem)

My be firewall lock

No.IEl 20 abr 2024, a las 15:22, Grzegorz @.***> escribió:
My be firewall lock
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

No. I reset the router and the problem persist.
I think is a problem with melcloud service.

Definitely a service issue. Lots of ppl having problems

Thanks for the information. It's reassuringEl 21 abr 2024, a las 12:12, cars11 @.***> escribió:
Definitely a service issue. Lots of ppl having problems
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

https://community.home-assistant.io/t/melcloud-not-working/718920

Thanks again.
The people is very frustrated with Mitsubishi, and is normal.
We still waiting...
Thank you for the link. Very usefull.

It's interesting to take a look at the website you've passed by to see how things are going. Mitsubishi is not giving too much information and users are starting to get fed up and they are starting to speculate about the causes and possible 'homemade' solutions. It seems that this incidence is affecting the whole of Europe. Some speculate that it was a hack because this issue is taking a long time to resolve. Whatever it is, the only thing that is certain is that Mitsubishi is demonstrating a lot of shortcomings.

yeah im surprised there is no statement. usually they post when they have server maintenance or wahteve ri think

Seems the problem is solved!


Support for file encoding

Describe the solution you'd like
Right now when you specify a secret it is always downloaded using utf-8 encoding. Something like objectencoding could be added in order to be able to download correctly other file encodings.
This would be useful to upload/download  binary files or jks files which cannot be converted to utf-8 More info

cc @chrisamert who is going to work on this


chore: update to scikit-learn

sklearn is being deprecated (see: https://pypi.org/project/sklearn/) and  this dependency should be updated to point to scikit-learn to avoid installation failures during brownout/deprecation windows.
fixes #5

Hi, following up on this - is this okay to be merged in?

Yes, please merge this

Please merge, pip install fails without this merge. Thank you!


docs(hub) introduce decK

Add decK to Kong Hub.

I couldn't get the site running locally due to an issue with bundle.
I'm hoping the netlify deploy will help us make sure the new content renders correctly before merge. Thanks!

@levfishbluefish Do you know why the decK doesn't actually show up in the preview of this PR?
https://deploy-preview-1418--kongdocs.netlify.com/hub/

NVM, just saw the comment in the review.

Okay, so it shows up in the preview now:
https://deploy-preview-1418--kongdocs.netlify.com/hub/hbagdi/deck/
Thanks @levfishbluefish!


[Feature Request] increment hover messages

Context

[X] This issue is not a bug report. (please use a different template for reporting a bug)
[X] This issue is not a duplicate of an existing issue. (please use the search to find existing issues)

Description
Hello,
I'm using provideHover for the hover feature of my editor. As a result, when we hover on a text in the editor (or select a text then press Ctrl+K Ctrl+I), we see its hover message.
Previously, the content (let's say Content A) of the hover message was very fast to generate. Now, I would like to add more content (let's say Content B) on top of Content A. Content B requires several seconds to generate. So an expected scenario is that when people select a text then press Ctrl+K Ctrl+I, we see first Content A (and maybe Calculating Content B...), then several seconds later, we see Content A and Content B in the hover window.
Is it achievable in the current Monaco Editor?

Hello,
I guess you can register 2 hover provider, it seems to do exactly what you want


DNS cache doesn’t persist between sessions

Operation System: Linux, macOS, and Windows
Beaker version: 0.8.8

Go to dat://beakerbrowser.com
Choose to seed the website
Give it some time to download the site. Check in the Library to confirm you’ve got the whole site.
Quit Beaker.
Disconnect from the internet.
Restart Beaker
Go to dat://beakerbrowser.com again.

Expected: page should load just fine as you have a local copy.
Actual: “It doesn't seem like anybody is sharing this page right now.”
beaker://dat-dns-cache/ confirm that the DNS cache fails and doesn’t persist between sessions.

I'm actually not 100% sure this is DNS cache that's failing. You'd get a message specific to DNS if it was.
My guess is there's some code that tries to check in with the network before loading the dat.

beaker://dat-dns-cache/ shows “false” so I figured that was the reason why the archive failed to load even though there is a local copy of it. As far as I can tell, there is no other mapping between dat://beakerbrowser.com and the raw dat archive key. beakerbrowser.com’s dat has a TTL of 1 hour so it should at the very least work for one hour. (Ideally the TTL should be configured to days and weeks and not just an hour – see the proposed changes in /datprotocol/DEPs/pull/59.) Reusing an expired stale response is better than not showing the page especially when a copy of the site is cached locally.

It sounds to be linked to my repport here https://github.com/beakerbrowser/beaker/issues/1354#issuecomment-515114790


System log

This one is really not helpful, system log says:
Login Succes
IP: some_IP_here
Success login

It would be benefitial if we had time and date of the action (add user account also), it tells me very little this way..

thanks, fixed now

Thank you, clising this one


Prevent initial page load from triggering onSet event

If I have the rateyo Plugin embedded in a template which is then injected in the page, my onSet event handler will trigger with a value of 5, when rating = 5;
For my purposes, I only want onSet event to be called when the user actually clicks on the rating, not initially. Any ideas on how I can make this happen?

hi @sachinlogiciel , In the latest version of the plugin, this is solved ,
you can download here:
JS - https://cdnjs.cloudflare.com/ajax/libs/rateYo/2.1.1/jquery.rateyo.min.js
CSS - https://cdnjs.cloudflare.com/ajax/libs/rateYo/2.1.1/jquery.rateyo.min.css


MULTI-GITTER PR: migration github-config-and-gitignore-changes

Summary by Sourcery
Revise GitHub configuration files to improve contribution guidelines, streamline code of conduct, and enhance issue templates. Update CI workflows to refine release conditions and add a demo deployment workflow.
CI:

Modify the testPublish.yml workflow to refine the conditions for triggering releases and adjust node version matrix formatting.
Add a new publishDemo.yml workflow to handle demo deployments on pull requests to the main branch.

Documentation:

Update the CONTRIBUTING.md to clarify guidelines for submitting issues and pull requests, emphasizing the need for consultation with the Auro team before starting work.
Revise the CODE_OF_CONDUCT.md to streamline language and remove specific contact details for reporting issues.
Enhance the ISSUE_TEMPLATE for feature requests and general support by adding an 'exit criteria' section to define closure conditions.

:tada: This PR is included in version 2.2.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add NCSU alert to PPU data tables

Example PPU data table page:

If a given presentation is NCSO in that month, add an icon (perhaps a red exclamation-in-triangle - one of the glyphicons from bootstrap) with a popover explaining "Savings may not be achievable long term as this item was in the Price Concessions/NCSO for ".
Definition of "NCSO in that month": we should scrape the NCSO list for November any time in December (but not before). That list then applies to the data for the whole of November.
Wording: it's actually no longer called NCSO but "Price Concessions" - we should probably call it "Price Concessions/NCSO"

Technically NCSO was different - so we should simply call it "price concessions".  That's my fault for continually using NCSO incorrectly.
(The point being that NCSO required pharmacies to endorse "no cheaper stock obtainable" on a prescription before getting reimbursed at higher rate.  This was phased out in favour of a national price concession which meant pharmacies automatically got the higher reimbursement.)

I went for Price Concessions/NCSO based on PSNC practice?


About U-Net model and data pre-processing

I am curious about the differences from the paper model

Why did you choose to use ConvTranspose instead of Upsampling
You used RELU in all parts and Leaky RELU after the last layer. But from the paper I think the author meant Leaky RELU everywhere except the last layer. And In the last layer only linear activation

Unrelated question
3) Did you use any pre-processing for the images e.g. means subtraction, normalization etc. I think that would be needed since we don't have BN layers
I am trying to implement the model in Tensorflow and having the problem of INF loss that starts in the 2nd epoch. So I hope your answers would help me. Thank you in advance!
Update: I seem to have solved the problem by adding Batch Norm layers to the UNET model. But still I am puzzled how the authors managed to get a stable training without Batch Norm

Hi Shakarim,

Upsampling works just as well as ConvTranspose. Use the one you like.
You might be right for this. The phrasing in the paper is kind of confusing. Seems to work okay with the current setup, but that might just solve the problem I had with Monte Carlo denoised output and tonemapping.
I am not doing anything to the images beside cropping them. No normalization is done to the images, and no batch norm is used (as written in the paper). This part was also weird to me; it doesn't hurt to use BN in this case.

I'm gonna mark this as closed, but feel free to open a new ticket if you find something else that feels off.

From this official release https://github.com/NVlabs/noise2noise/blob/master/network.py
Probably Shakarim94 is right.
I've trained on imagenet val set and could not get 32dB on Kodak
Maybe this modification would help


Error 500 when viewing some of repo's subdirectories

Gitea version (or commit ref): 1.7.6
Git version: <IP_ADDRESS>
Operating system: Windows Server 2008 R2
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[ X] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[ X] No
[ ] Not relevant

Log gist:

Description
I get error 500 when I try to view contents of some directories inside my repo. One subdirs are opened successfully but another gives an error. I can not find any quick reason to it.
All subdirs have simple names with only ASCII encoding, no special symbols or spaces.
In my gitea.log I see:
2019/05/03 09:01:08 [...routers/repo/view.go:55 renderDirectory()] [E] GetCommitsInfo: exit status<PHONE_NUMBER>

For example I get 500 when I follow:
http://sc.lers.ru/LersUchet/LersUchet/src/branch/master/Client
But following http://sc.lers.ru/LersUchet/LersUchet/src/branch/master/Server works as expected.

Can you reproduce this on try.gitea.io?

Unfortunately no. I can't upload my repo as it is commercial software. This repo is quite big - over 500MB in size and has 35K commits over last 7 years , so it is hard to reproduce it.
Also I get 500 with same error code when I try to view some old commits by SHA-1.
2019/05/06 14:17:22 [...uters/repo/commit.go:196 Diff()] [E] Repo.GitRepo.GetCommit: exit status<PHONE_NUMBER>

Not a big deal for me anyway, but it's quite strange.

Hmm... Two possible encoding problems come to mind:

Do you have filenames in client that are not in UTF-8 encoding?
Does client have commit messages that are not in UTF-8 encoding?

Git internally just thinks of filenames as a byte array of non NULs and likely we're just enforcing UTF-8 on that because that's the most likely case. Similarly for the commit messages but AFAIU there's supposed to be an encoding marker for these - which I also suspect we're ignoring.

No, all source file names are in ASCII encoding.
Right now all commit messages are encoded UTF-8 by default. Initially it was SVN repo, so I suppose that during migration "git svn" should have used UTF-8 by default too.

Anyway I can't be sure about it. In this comment I pointed that 500 occurs when viewing older commits, so maybe it is commit encoding that causes 500.

Ok that's good to know. I think there are some Chinese users who have used big5 as their commit messages too and are also having problems.
https://www.git-tower.com/help/mac/faq-and-tips/faq/encoding
Might be of some help.

In particular:
git log -1 --pretty='format:%h: "%B" (Encoding: "%e")' SHA

Which is just for testing one commit's stored encoding but should be extendable to look at all or a particular file.
Now iirc svn is actually quite good at dealing with encodings - so I expect that the encoding will be stored rather than just the data being dumped in the message, however it may not be. If it's the first case that should be relatively easy to fix - assuming go git handles this correctly. If it's the second case well that's more difficult but again should be solvable.
If you could find a commit which doesn't have the utf8 encoding that would be good to know.

Oh would it be possible to check this is still a problem on master? We made substantial changes to our git infrastructure between 1.7 and 1.9 so it would be good to know if it's still a problem.
You certainly should move off 1.7.6 though.

Hello!
I have already upgraded to the latest 1.8.1 and 500 is still there.
All commits in my repo are encoded UTF-8 , ie "Encoding: %e" shows empty string **Encoding: ** As I understand it defaults to UTF-8.

I was meaning master not 1.8.
The lack of encoding headers doesn't rule out misencoded commits, it just means that we're unlikely to be able to easily fix them without catching an error first. But it does make it less likely.
Are you able to find the commit that causes the 500? It sounds like you know a SHA that causes the problem. Is there anything at all you can tell me about that SHA that might help?

Sorry, right now I can't try master, because it's preferable to stay on stable release.
Actually all commits that was made earlier than 04.07.2016 causes 500. I know last commit causing 500. There is nothing special about it, only couple of line changes in one file. No moves or renames.
What particular information about this commit would you like to know?

Hmm. I'm trying to work out how to get the information without you revealing your internal data. I'm guessing that the commit message is not in English but likely in Russian - my suspicion is still that the message is not stored as utf8 even though it claims it is.
Basically I would like to check that the commit and the tree make sense.
git show --format=raw $SHA should show us the commit raw but what we want is to definitely prove that its message is in utf8 rather than Windows-1251 or iso-8859-5.
In some ways without being able to check whether this bug is still present on master means that we don't know if it's fixed already.

Having a similar problem myself, using MySQL as the DB though. Running on v799f5e0.
Can't view here: https://gitea.bnolet.me/brandon/personal-blog/src/branch/master/themes/hello-friend-ng
But can view here: https://gitea.bnolet.me/brandon/personal-blog/src/branch/master/themes
Haven't done much troubleshooting yet, but if I find anything interesting I'll post here. Feel free to download and mirror the source.

@BNolet I mirrored your repo to local and https://gitea.com/lunny/test-personal-blog/src/branch/master/themes/hello-friend-ng , but it's OK. Is your database setting as utf8mb4?

@lunny Strange! I didn't actively change the encoding for the DB and looking at some threads, it doesn't seem that's the default either, so I'm inclined to say no

The following is from the logs at the moment where I get a 500 error code:
2019/07/15 15:13:32 [...les/context/panic.go:36 1()] [E] PANIC:: runtime error: slice bounds out of range
/usr/local/go/src/runtime/panic.go:54 (0x4340da)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:176 (0x90b6e0)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:280 (0x90c101)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:232 (0x90bc3c)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:133 (0x90ae03)
/go/src/code.gitea.io/gitea/routers/repo/view.go:52 (0x1226a16)
/go/src/code.gitea.io/gitea/routers/repo/view.go:437 (0x122befd)
/go/src/code.gitea.io/gitea/routers/repo/view.go:377 (0x122b69e)
/usr/local/go/src/reflect/value.go:447 (0x4cb940)
/usr/local/go/src/reflect/value.go:308 (0x4cb3c3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:177 (0x9a1266)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:137 (0x9a0b5b)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x11661a8)
/go/src/code.gitea.io/gitea/modules/context/panic.go:40 (0x116619b)
/usr/local/go/src/reflect/value.go:447 (0x4cb940)
/usr/local/go/src/reflect/value.go:308 (0x4cb3c3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:177 (0x9a1266)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:137 (0x9a0b5b)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x9efc76)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/session/session.go:192 (0x9efc61)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:79 (0x9cfbc0)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:157 (0x9a0f20)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:135 (0x9a0c4a)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x9e177f)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/recovery.go:161 (0x9e176d)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/logger.go:40 (0x9d39b3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:157 (0x9a0f20)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:135 (0x9a0c4a)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x9e0aa0)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/logger.go:52 (0x9e0a8b)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/logger.go:40 (0x9d39b3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:157 (0x9a0f20)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:135 (0x9a0c4a)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:187 (0x9e29c6)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:303 (0x9dc435)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/macaron.go:220 (0x9d4d8c)
/go/src/code.gitea.io/gitea/vendor/github.com/gorilla/context/context.go:141 (0xce32ca)
/usr/local/go/src/net/http/server.go:1995 (0x6f6213)
/usr/local/go/src/net/http/server.go:2774 (0x6f94e7)
/usr/local/go/src/net/http/server.go:1878 (0x6f51d0)
/usr/local/go/src/runtime/asm_amd64.s:1337 (0x464c60)

@BNolet It seems you have a version of v1.8.x or before.

welp, upgrade fixed the issue. I was running 1.8.0 upgraded to 1.10.
Thank you @lunny. I'll try remember to upgrade before posting here next time

Yes, upgrade to 1.9.0 RC2 fixed issue.
Thanks!


Problems editing parentheses (and other paired characters) (low priority)

Currently when you edit and you press "left parenthesis", the tool adds "right parenthesis". This makes sense in some contexts, but it's annoying in others - in many cases you are certain to need to delete the right parenthesis, and their insertion is more trouble than help.
I think using them should be an editor setting (on/off), so if you hate them they stop getting added.
If it's on, it'd be nice if they were inserted using a smarter algorithm. Here's a simple one (algorithm 1):

Insert the matching closing "paren" after the opening if the expression is at the end or the expression is followed by a closing "paren-like" symbol.

Algorithm 1 is at least simple to simple to implement.
Another algorithm (algorithm 2) would be:

only add the right paren after the left one if the whole statement would be syntactically valid (once the "(...)" expression was completed).

However, this second one is complicated, I'm not sure how to do that quickly & I doubt it's worth the trouble.

The easy solution is to just add a setting to enable/display smart parentheses. Then people like it, they can use it, otherwise they can turn it off.

I am thinking of implementing the following:

when you edit a statement, near the text field, there is a button to turn on/off parenthesis autocompletion. So you don't have to move your mouse too far. Also it should be possible to turn it on/off with a keyboard shortcut.
There will be a global setting on the settings tab to indicate how you want this autocompletion to be (on or off) when you start editing a statement. So depending on what you do more often (editing existing or adding new) you may change this global setting.

Cool! That would be nice.

@david-a-wheeler
This is implemented on dev. However, not exactly as I planned:

when you edit a statement, near the text field, there is a button to turn on/off parenthesis autocompletion. So you don't have to move your mouse too far. Also, it is possible to turn it on/off with the keyboard shortcut Alt+P.
There is no a global setting to indicate how you want this autocompletion to be (on or off) when you start editing a statement. Instead, the last used state (on or off) is used when you start editing another statement.

This was easier to implement (no need to add a new global setting) and, as for me, it is convenient as well.

Overall that's fine, but I recommend NOT using "Alt-P", as that usually means "print". Is "Alt+(" a possibility?

As I know "Ctrl+P" means "print". What browser do you know where "Alt+P" means "print"?
I thought about "Alt+(", but it would actually be "Alt+9", that's why I didn't want to use this shortcut.

Nevermind, you're right, wrong meta key. Control-P is print on Windows, command-P on Mac, which isn't Alt-P or Opt-P.

So carry forth, your current plan makes sense to me!

This is available in version 15 https://expln.github.io/lamp/v15/index.html


[WIP] Support CentOS 8.

This PR is after the PR: https://github.com/junaruga/rpm-py-installer/pull/237 .
Possibly the py36 case on CentOS 8 fails with the following error. I checked it on local.
>                   installer.install_from_rpm_py_package()
E                   Failed: DID NOT RAISE <class 'install.RpmPyPackageNotFoundError'>
tests/test_install.py:716: Failed

def test_installer_install_from_rpm_py_package(
    installer, is_debian, is_centos, monkeypatch
):
    with pytest.helpers.work_dir():
        # The RPM Python binding for python3 is not provided on CentOS.
        # rpm-python3 does not exist on CentOS.
        # http://mirror.centos.org/centos/7/os/x86_64/Packages/
        if is_debian or \
           not installer._predict_rpm_py_package_names() or \
           (is_centos and sys.version_info >= (3, 0)):
            with pytest.raises(RpmPyPackageNotFoundError):
                installer.install_from_rpm_py_package() # <= the line 716.

The reason is because there is python3-rpm-4.14.2-37.el8.x86_64.rpm package on http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/ . python2-rpm and python-rpm do not exist.
The solution is checking VERSION_ID in the process of the test.
Here is the /etc/os-release  on CentOS 8.
$ podman run --rm -it centos:8 bash
[root@1acda2c8fb56 /]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="8"

Possibly the py36 case on CentOS 8 fails with the following error. I checked it on local.

The test failed as expected.
https://travis-ci.org/github/junaruga/rpm-py-installer/jobs/741230891#L853

Note:
Related to the logic to get values in /etc/os-release file, I found the following logic to load the values as environment variables. We can use the logic instead of the current logic parsing the values in the file by python manually.
https://podman.io/getting-started/installation - Ubuntu
. /etc/os-release
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list


set_id issue with included

Picture simple relationship like this:
class ASerializer
  include FastJsonapi::ObjectSerializer

  has_many :bs
end

class BSerializer
  include FastJsonapi::ObjectSerializer

  set_id :custom_id
end

The problem is that fast_jsonapito serialize A objects with B included will generate structure like this:
{
  data: [
    { id: AID,
      type: :a,
      attributes: {},
      relationships: {
       bs: [
         { id: B1ID, type: :b },
         { id: B2ID, type: :b }
       ]
      }
  ],
  included: [
    { id: B1CUSTOMID, ... },
    { id: B2CUSTOMID, ... },
  ]
}

So that unless we specifically set id_method_name in relationship and implement it accordingly this makes no sense
I suggest to add additional checks to prevent this situation, i.e. have included ids not matching relationships ids

This is backlog feature request, in general, as it improves API but not changes functionality and may affect performance

This is interesting. I believe this could be a bug. Do you mind writing a failing test to better capture this issue? I can try to fix it if you could do that.

Sure, please take a look at https://github.com/ababich/fast_jsonapi/compare/master...set_id_fail

hi folks - just bumping this as we're running into this same issue as well.

@yale there is a workaround for this as @ababich mentioned you need to set both id_method_name and set_id. We would need to figure out how to set_id so that it can be used both in the serializer and all the places the record is used in a relationship

Released in Version 1.4

@shishirmk I don't think this issue is resolved by #274 since I'm still running into the same problem with 1.4. As manojmj92 pointed out, #274 only solves the issue with id_method_name not being set with block.


FYI: Webbooost's Research on CDN Usage and JS Libs Link is Broken

Hey, Mr. Mig,
I just wanted to let you know that ever since your changed your pretty url-rewrites from slash to hyphen (or when you changed to Ghost CMS) your Webbooost's Research on CDN Usage and JS Libraries link is currently broken--and perhaps others too. Most users may not care much about the inner mechanisms but developers and power users sure do, don't you think?
I recommended you to a Dev. a friend of mine who (like me and you too right?) love pushing the short and long-tail of tweaks and tech that makes for a "faster Internet" experience.
Спасибо!

Thanks for pointing my attention to this!