Is it possible to get data faster than 5 seconds?

I need to get the data as quickly as possible, what is the fastest way to do this?
No matter how I try, I always get only once every 5 seconds.
Perhaps there are some restrictions in the browser itself?

There is nothing we do to "slow down" the data getting to the component in case the user has allowed the Geolocation API to be used, so maybe there is in fact som slow-down in the browser.


401 Content access is protected by token

Has anyone encountered this issue?
https://community.sonatype.com/t/401-content-access-is-protected-by-token-authentication-failure-while-performing-maven-release/12741/15
I have already published several releases of a library without having any problem (thank you for the plugin!)
But now, as I am trying to publish a new release, I have been hit by exactly this.
The process is able to create a staging repository but then fails when trying to upload the bundle.
I succeeded in generating a new token (user and password) from https://s01.oss.sonatype.org, but then how I use it?
I tried to substitute the user and password in the https://www.scala-sbt.org/release/docs/Using-Sonatype.html#step+3%3A+Credentials, but it does not seem to work...

I haven't seen this issue, probably because I only use https://oss.sonatype.org (legacy host).
Sonatype also has recently migrated to the new authentication system.
I believe the only settings you will need are sonatypeCredentialHost:
import xerial.sbt.Sonatype.sonatype01
ThisBuild / sonatypeCredentialHost := sonatype01

and credentials:
credentials += Credentials("Sonatype Nexus Repository Manager",
        "s01.oss.sonatype.org",  // I guess this needs to be s01.oss.sonatype.org
        "(Sonatype user name)",
        "(Sonatype password)")

As I don't have an user account for s01.oss.sonatype.org, I can't test it by myself.

I release to s01 and can confirm this works. The (Sonatype user name)  and (Sonatype password) have to be the values provided as tokens as described here.

I had the same issue today for the first time, no changes to my repo, the plug-in etc. It seems something has been changed in sonatype, now you need to use a user token, as described here: https://central.sonatype.org/publish/generate-token/ - only that it shows a maven XML and not how to do it in sbt.
I managed to fix the 401 issue (on legacy oss.sonatype.org): In ~/.sbt/1.0/sonatype.sbt, I changed
credentials += Credentials("Sonatype Nexus Repository Manager", "oss.sonatype.org", "<sonatype-user>", "<sonatype-password>")

to
credentials += Credentials("Sonatype Nexus Repository Manager", "oss.sonatype.org", "<token-user>", "<token-pass>")

Thanks for the reports.
Sonatype no longer accepts user and password-based authentication, so we need to generate a token as described in https://central.sonatype.org/publish/generate-token/ and use it for authentication.
We may be able to improve the error message for the 401 error (unauthenticated). Still, there is no reliable way to know whether the given username and password are tokens, so it would be difficult to report this case properly.
Let me close the issue. Thanks.


stable/velero

Recent updates indicate the addition of compatibility with helm 3, however, I get when I try it:
masoud@LAPTOP-5MS843IU:/mnt/c/Users/masoud$ helm install --namespace velero  velero -f Desktop/velero-values.yaml stable/velero --version 2.6.0
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
NAME: velero
LAST DEPLOYED: Fri Dec  6 20:30:26 2019
NAMESPACE: velero
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Check that the velero is up and running:
kubectl get deployment/velero -n velero

Check that the secret has been created:
kubectl get secret/velero -n velero

Once velero server is up and running you need the client before you can use it

wget https://github.com/vmware-tanzu/velero/releases/download/v1.2.0/velero-v1.2.0-darwin-amd64.tar.gz
tar -xvf velero-v1.2.0-darwin-amd64.tar.gz -C velero-client

More info on the official site: https://github.com/vmware-tanzu/velero#install-client
masoud@LAPTOP-5MS843IU:/mnt/c/Users/masoud$  kubectl get deployment/velero -n velero
Error from server (NotFound): deployments.extensions "velero" not found

It seems these warnings were immaterial to my issue, which was malformed values.yaml


NCCL Timeout Error when applying NCE Loss

Hi! Sorry to bother you again. In my further experiments I want to explore the potential of contrastive learning. But I encounter an error when setting with_contrastive=True:
[E ProcessGroupNCCL.cpp:414] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might
run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=66, OpType=ALLGATHER, Timeout(ms)=1800000) ran for 1801979 milliseconds b$
fore timing out.
[E ProcessGroupNCCL.cpp:414] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might
run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=67, OpType=ALLGATHER, Timeout(ms)=1800000) ran for 1802085 milliseconds b$
fore timing out.

I try to debug the problem, and I find the issue probably comes from line 236-241 in common/model.py where the allgather operation executing.
if self.cfg.with_contrastive:
      dense_feat = allgather(dense_feat) # B, C, D, H, W
      reduced_orig_joint_img = allgather(reduced_orig_joint_img.div(32).sub(1)) # (-1, 1), B, 15, 3
      joint_img_cur = allgather(joint_img[layer_idx].mul(2).sub(1)) # (-1, 1) B, 15, 3
      joint_valid_cur = allgather(reduced_orig_joint_trunc) # B, 15, 1
      is_3D_mask = allgather(meta_info['is_3D'])
      is_3D_mask = is_3D_mask.flatten() == 1 # B,

When the program goes to here, it turns out no response and then occurs timeout error. It looks like a DDP operation error but I am not so familiar with it, so I am looking for help if you have any solutions to debug. Thanks very much!

Can you provide more information? For example, what GPU are you using, which distribution of the operating system, PyTorch version, CUDA version, and so on.
In addition to NCCL, you can also try using gloo. More information can be found here.

OK, here's my settings:

GPU: Nvidia RTX 3090
Operating System: Ubuntu 18.04 x86_64
Python Version: 3.7
Pytorch Version: 1.12.1+cu113
CUDA Version: 11.4
NCCL Version: 2.10.3

and here's the command to start train:
CUDA_VISIBLE_DEVICES=6,7 \
python -m torch.distributed.launch \
--master_port 29521 \
--nproc_per_node 2 \
train.py \
--cfg ./assets/yaml/fine_tune.yml \
--distributed \
--lr 5e-5 \
--lr_backbone 5e-5 \
--weight_decay 0.0001 \
--exp_id "fine_tune" \
--resume_ckpt output/JOTR/checkpoint/snapshot_18.pth.tar \
--continue

It looks like gloo is for CPU communication. Except this issue, the NCCL connection for DDP operation seems work well. It looks like gloo is for CPU communication. So I want to figure out whether there are any problems in my settings or code under GPU settings. Thanks for your help!

I haven't tested it on a 3090 and Ubuntu machine. My environment is CentOS and V100/A100 and runs well. I have written a unit test that you can try in your environment.
import torch
import torch.distributed as dist
class AllGather(torch.autograd.Function):
    """An autograd function that performs allgather on a tensor."""

    @staticmethod
    def forward(ctx, tensor):
        output = [torch.empty_like(tensor) for _ in range(dist.get_world_size())]
        torch.distributed.all_gather(output, tensor)
        ctx.rank = dist.get_rank()
        ctx.batch_size = tensor.shape[0]
        return torch.cat(output, dim=0)

    @staticmethod
    def backward(ctx, grad_output):
        return (
            grad_output[ctx.batch_size * ctx.rank : ctx.batch_size * (ctx.rank + 1)],
            None,
        )
allgather = AllGather.apply

if __name__ == '__main__':
    # init process group
    dist.init_process_group(backend='gloo')
    torch.cuda.set_device(dist.get_rank())
    print('rank:', dist.get_rank(), 'world_size:', dist.get_world_size())
    x = torch.randn(32, 3, 256, 256, requires_grad=True).cuda()
    y = allgather(x)
    print('rank:', dist.get_rank(), 'y:', y.shape)

Running scripts
torchrun --nproc_per_node 2 test_allgather.py

Thanks very much for your scripts! I'm sorry for late reply. After I deeply research for the reason of error, I found that it is the incompleteness of my dataset cause this error. I have set collect_fn parameter in dataloader in order to deal with the missing data, but this will cause a stuck when conducting the allgather operation. As shown in pytorch manual, this error can be visable by export TORCH_DISTRIBUTED_DEBUG=DETAIL.
Reference in discuss.pytorch.org
RuntimeError: Detected mismatch between collectives on ranks. Rank 0 is running inconsistent collective: CollectiveFingerPrint(OpType=ALLGATHER, TensorShape
=[41, 256, 8, 8, 8], TensorDtypes=Float, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (
default), pinned_memory=false (default), memory_format=(nullopt))
RuntimeError: Detected mismatch between collectives on ranks. Rank 1 is running inconsistent collective: CollectiveFingerPrint(OpType=ALLGATHER, TensorShape
=[40, 256, 8, 8, 8], TensorDtypes=Float, TensorDeviceTypes=TensorOptions(dtype=float (default), device=cuda, layout=Strided (default), requires_grad=false (
default), pinned_memory=false (default), memory_format=(nullopt))

To solve this problem, I think one way is to complement the data, or set an try: ... expection: ... to handle the error. So I'd like to share with you about this detail and express my apologies for bothering you about this.
However, even I have dealt with this problem, it goes to CUDA Memory Error. I set train_batch_size=4 and use 2 GPUs for about 24G memory each, but it still goes memory error. Thus I want to know how much the memory costs in your experiments approximately? Or how can I reduce my computation costs by adjusting the hyperparameters in NCE Loss?
Thanks again for you considerating reply.

I run my code with train_batch_size=16 with 8 V100 or train_batch_size=32 with 4 A100. However, based on my experience, a 24GB graphics memory should be sufficient to run experiments with a batch size of 4, there may be other issues as well.

OK I got it! I'll try to figure out the issue. Thank you so much!


os_defines.h:44:19: error: missing binary operator before token "("

Hi!
I'm trying to build an old Intel PIN tool using the current GitHub's version of the single include file json.hpp. I noticed that it fails building in my system (Ubuntu 16.04 x86_64, tested with g++ 5.0 and 4.9). This the very first error I get:
In file included from /usr/include/x86_64-linux-gnu/c++/5/bits/c++config.h:482:0,
                 from /usr/include/c++/5/initializer_list:41,
                 from json.hpp:41,
                 from Tracer.h:24,
                 from Tracer.cpp:5:
/usr/include/x86_64-linux-gnu/c++/5/bits/os_defines.h:44:19: error: missing binary operator before token "("
 #if __GLIBC_PREREQ(2,15) && defined(_GNU_SOURCE)
                   ^

I tried including features.h before "json.hpp", defining myself __GLIBC_PREREQ to trick it, but it fails in other ways, so I guess I'm doing something wrong :)
Any ideas?

Could you try to apply a patch similar to this one?

@theodelrieu I honestly don't know how to apply a similar patch. That patch is a for a kernel and is changing directly the part where that macros were used. I'm coding a user-land component and it's failing at a standard include file.

Oh sorry I thought this was in the library's code... This SO post looks a lot like your issue, maybe it can help?

I tried most of the workarounds there: None worked for me. Before opening an issue I actually tried applying workarounds from that Stack Overflow's post, as it's the very first Google result. Alas, it didn't work for me.

Hmm, seems that including <initializer_list> on your system breaks. What is the stdlibc++ version you're linking against?

And only including <nlohmann/json.hpp> breaks?

No, I tried with another old project where I use it (a rather complex one) and everything builds OK. It seems the problem is specific to Intel PIN. If I try to build the most simple of the Intel PIN tools (just enter into $DIR/source/tools/MyPinTool and type 'make'), it compiles OK, but as soon as I include "json.hpp" in MyPinTool.cpp, it breaks. It didn't before with previous versions of Intel PIN. I wonder why...

And including <initializer_list> with Intel PIN's headers break, right?
I've looked at this issue, and it might be related to your problem (if Intel fiddles with glibc headers).
I have no notion of Intel PIN, so I'm afraid I won't be very helpful here :/

Yes, you're right, by simply adding that include file it breaks. Thanks a lot for the help!

I wish you good luck!

So this is not an issue of this library (other than it including <initializer_list>?

Nope, my issue has nothing to do with this library. I'm closing the issue. Again, thanks for the help!

No, I tried with another old project where I use it (a rather complex one) and everything builds OK. It seems the problem is specific to Intel PIN's headers. If I try to build the most simple of the Intel PIN tools (just enter into $DIR/source/tools/MyPinTool and type 'make'), it compiles OK, but as soon as I include "json.hpp" in MyPinTool.cpp, it breaks. It didn't before with previous versions of Intel PIN. I wonder why...

I had the same issue yesterday. and I solved it by setting CPLUS_INCLUDE_PATH to the right value, it was "/usr/include/jsoncpp/json" which is wrong, the right value should be "/usr/include/jsoncpp".


Shopper Login (SLAS) missed getPasswordResetToken and resetPassword methods

Methods getPasswordResetToken and resetPassword are missed for Shopper Login (SLAS) (v.2.8.0) but mentioned in the release changelog  and B2C Commerce API

@johnboxall https://github.com/orgs/SalesforceCommerceCloud/teams/commerce-cloud-services can you guys please have a look into this? Thanks!

Hi @a-dzhus, thanks for creating this issue! The team is taking a look into this

Hi @a-dzhus we've been able to confirm and reproduce this issue. I'll go ahead and create a work item in our backlog and we'll loop you in once this issue gets resolved!


Remove all code that still use IDS REST

We should remove all code areas in the other folder that still use IDS REST:

copy-file-from-s3-to-s3

SHould be done either directly in or after PR #493

done


Clear holography algorithms

This issue is part of my JOSS review at https://github.com/openjournals/joss-reviews/issues/6315
Your paper writes,

The package implements the hologram generation algorithms of the Lee hologram (Lee, 1978)
and its improved predecessors (Shih et al., 2021; Zupancic et al., 2016), targeting the digital
micromirror device (DMD) based SLM with binary amplitude controls. It also implements the
Gerchberg-Saxton algorithm (Gerhberg & Saxton, 1972) suitable for liquid crystal on silicon
(LCoS) based SLMs with pure phase controls.

And later,

Currently, pySLM2 supports two type of the spatial light modulator (SLM), liquid crystal on
silicon (LCoS) SLM and digital micromirror device (DMD). The LCoS SLM modulates the
phase profile purely without modifying the amplitude. As the time of writing, Gerchberg-Saxton
(GS) (Gerhberg & Saxton, 1972) algorithm and the mixed-region amplitude freedom (MRAF)
algorithm (Gaunt & Hadzibabic, 2012) are included.

On the other hand, DMDs use micromirrors to locally turn on and off the light by toggling the
micromirrors between two directions. This allows binary amplitude control. By periodically
turning on and off the micromirrors across the DMD to form grating patterns, diffracted beams
with controllable phase and amplitude can be engineered to have the desired beam profiles. As
the time of writing, a randomized algorithm (Zupancic et al., 2016) and an iterative Fourier
transformation algorithm (Motlakunta et al., 2023; Shih et al., 2021) are provided for hologram
generation.

These seem somewhat contradictory; the first quote implies Lee's hologram algorithm is present in the package, the latter pair imply it is not.  I was not able to find it.  All I could find is in _lib.py, the def calculate_dmd_grating function and the routines it calls.  I was surprised that none of the code in _lib.py indicates which function is which algorithm.  Could you please name the functions or add to their documentation which is which?
Unrelatedly, a grammar issue: ...improved predecessors is improper English.  To be a predecessor, Shih et al or Zupancic et al would have had to publish their algorithms before Lee.  These could perhaps be considered "successors" but are more aptly "improved alternatives."

These seem somewhat contradictory; the first quote implies Lee's hologram algorithm is present in the package, the latter pair imply it is not. I was not able to find it. All I could find is in _lib.py, the def calculate_dmd_grating function and the routines it calls. I was surprised that none of the code in _lib.py indicates which function is which algorithm. Could you please name the functions or add to their documentation which is which?

Documentation for the algorithms: https://pyslm2.pages.dev/algorithm
The Lee hologram corresponds to the ideal and the simple methods.

Unrelatedly, a grammar issue: ...improved predecessors is improper English. To be a predecessor, Shih et al or Zupancic et al would have had to publish their algorithms before Lee. These could perhaps be considered "successors" but are more aptly "improved alternatives."

Fixed!


python3 is required to run abi tests, generate cmake error if python3 not found

currently "make test" fails without good explanation, this could be caught with better configuration checks during cmake.

A number of us ran into this recently.  The plan is to add a dependency check and auto install for python3.  The current work-around is to install python3.


[Bug] 1.1 version Column prune incorrect result

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
1.1
select t1.l_orderkey,
       t2.s_phone
  from (
        select l_suppkey,
               l_orderkey
          from lineitem
       )t1 left join(
        select s_suppkey,
               s_phone
          from supplier
       ) t2
    on t1.l_suppkey=t2.s_suppkey
 group by t1.l_orderkey,
          t2.s_phone

For this sql, when FE generate a HashNode, its output expr could contains TupleIsNullPredicate.

The tupleId is 2, it stands for left table.

In Doris be,  s_suppkey column couldn't be insert value because it's pruned.
But s_suppkey could be used to eval TupleIsNullPredicate, then the output value isn't true;
Here is my temporary solution, #12404.
But we need to solve it in FE finally.
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Solution
1 generate a _column_to_check for HashJoinNode, by using HashJoinNode's _row_desc_for_other_join_conjunt or _row_descriptor.
2 We fill column by


Tuple literals

Using (tuple ...) seems a bit much, what about literal {}?

While it's not {}, we already have a literal.
#('ok "I'm a tuple") will work.
If you need to match on or insert values into a tuple literal, you simply quote it and unquote the individual values, e.g.:
(let ((value "I'm a tuple"))
  `#(ok ,value))

However, (tuple ...) is required for some matching situations, if I recall correctly.

Yes, literal tuples are written with #(ok 1 2 3). It  is a literal tuple so nothing in it is evaluated, it is just itself. If I use it in the "right-hand-side" in an expression it will just return itself exactly as it is, no need to quote anything. If I use it in the "left-hand-side" in a pattern then it will match exactly that tuple.
To build a tuple you use the (tuple 'ok (+ 100 23)) form which dynamically evaluates its arguments and builds the tuple. This form evaluates to the tuple #(ok 123). If I want to match against a tuple and extract values the I need this form. So the pattern (tuple 'ok value) will match a tuple of 2 elements whose first element is the atom ok and bind the variable value to the second element.
Using the backquote macro ` together with , allows me to write a literal tuple with #(...) which expands into a tuple form. So  `#(ok ,value) expands into (tuple 'ok value). This is very practical and can make things easier to read.
There have been suggestions to use { } as syntax for literal tuples, so {ok 123} would be equivalent to #(ok 123). Or perhaps as alternative to (tuple ...) form, so then {'ok value} would be equivalent to (tuple 'ok value). We would have to pick one or the other as they are not equivalent, or none of the above.
So far it has been non of the above.

@rvirding all the forms which evaluate the forms inside the syntax seem a bit verbose. E.g to get a neat form which evaluates its terms you need to syntax-quote and syntax-unquote. Often people do this in Common Lisp with literal assoc lists and it's always a smelly bit of code when you see this. What's the cons for adding a form like {} to do the same thing?

The smell is definitely subjective.
I can think of a couple reasons not to want {...}:

Comma means unquote, which could lead to unnecessary confusion.
We borrow from Clojure a bit (libs more than lfe proper for now) and {...} (with no commas) is a map there.
We already have #(...).

The quote/unquote you see in CL actually expand to forms which evaluate internal forms. You have the two versions there as well: the literal form and the evaluating form. For example in SBCL:
* (setq a 42 b 84)
84
* (vector 'a a 'b b)

#(A 42 B 84)
* `#(a ,a b ,b)

#(A 42 B 84)
* (macroexpand '`#(a ,a ,b b))

(SB-IMPL::|Vector| 'A A 'B B)
T

The SB-IMPL::|Vector| is just an implementation level form for constructing a vector. Here #( ... ) is the literal and (vector ... ) is the evaluating form. The ` is a quote which allows unquoting inside using the , prefix. They exist for all composite types. This is how lisp works.
The only question is whether to use { ... } for the literal.

Cool, I didn't know that about SBCL. I vote :-1: for { ... }, I'm pretty sure.

My 2 cents.
While I fully understand LFE isn't common lisp #() is somewhat confusing because as we all now this is standard syntax for vectors in CL. Probably can source many annoying errors for people coming from CL.
and {} syntax is for tuples in erlang already.

We would have to pick one or the other...

I'd prefer {'ok value} => (tuple 'ok value) over{ok value} => #(ok value) => (tuple 'ok 'value)

I chose the #( ) syntax for tuples just because it is used for CL vectors
and tuples and vectors are pretty much the same thing. :-)
On 22 April 2016 at 01:25, Ilya Khaprov<EMAIL_ADDRESS>wrote:

My 2 cents.
While I fully understand LFE isn't common lisp #() is somewhat confusing
because as we all now this is standard syntax for vectors in CL. Probably
can source many annoying errors for people coming from CL.
and {} syntax is for tuples in erlang already.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rvirding/lfe/issues/225#issuecomment-213157446

If we go with using { } it would be as a replacement/alternative to #( ). Otherwise it would be inconsistent with the other types for which we have a special literal syntax, binaries and maps, which have a constructor to build them, binary and map.

Even in Lisp, datastructures having literals is always a win. See: clojure.

The problem is not whether they have literals or not, #(...) is a literal, but that you also need constructors. So you need (tuple ...) (or its equivalent) when you build a tuple which is not just a literal value, for example it contains the values of variables or expressions. Then you cannot write them as literals. Yes, you can use the ` macro and , to make it look more like a literal but it is not a literal. So doing  `#(ok ,a) actually expands to (tuple 'ok a).

Clojure's sugar works a bit differently from the way #(...) works now, i.e. [a b c] is analogous to `#(,a ,b ,c) not #(a b c). I agree introducing a { ... } syntax that works as I described above would be incongruous.

Ah ok, because in the doc page you get the impression that vector works like tuple and [ ] like #( ). Have to download and test it properly I see.

Just tested, it correct. But in that case why have two ways of writing the same thing? I do prefer having a literal and a constructor instead of just a constructor. Also a bit inconsistent:
user=> (list 1)
(1)
user=> (1)
ClassCastException java.lang.Long cannot be cast to clojure.lang.IFn  user/eval45 (NO_SOURCE_FILE:31)

Vectors only have constructors no real literals while lists need an explicit constructor.

Yeah.. I'm fine with the quasiquoting and mostly indifferent to the addition of { ... }.
FWIW, in practice I rarely use (vector ...) in Clojure.
One interesting bit too is that Clojure uses :keywords far more often than 'symbols', which I think alleviates some of the issues we have in LFE wrt ,varvs'var`.

I like the idea of having the {} literal syntax for tuples as it's one less thing to remember when switching between LFE and other popular BEAM languages.

I'm actually not a fan of special syntax for litteral forms. One of the nice things I like about Lisps in general, is their syntaxlessness. Admittedly without reader macros, LFE is less open to language customization than other Lisps.
Although I do enjoy Clojure a great deal, one of the ways in which it is not classically Lisp-like is all the custom syntax. I wiould vote for staying closer to LFE's Lisp roots and not trying to do many things like Clojure.
What I would propose instead would be to add a mechanism to LFE (whether that be reader macros or something else) which allowss users, devs, library maintainers, etc., to more easily augment LFE with their own syntaxes. At which point we could then bring alternate, Clojure-like syntaxes for literals into the clj library (once it gets merged into LFE-proper).
This would help keep the core language itself lighter and less prone to feature accretion and subsequently, exponential maintenance burdens.

The problem is whether {...} should be equivalent to the existing #(...) or (tuple ...).  We could not agree. LFE follows the CL and scheme conventions here which I feel are more lispy, at least more what I am used to.
If you write a clj library with some special syntax and merge it into LFE proper then you will have extended the core syntax. Or did you mean the ability to extend the syntax? The LFE parsing does not work int the same way as classic lisp parsets do.

I mean that we'd first have to add to LFE the ability to support special syntaxes generically (e.g. reader macros), and then at that point we could add Clojure-inspired syntax to the clj module in LFE.
We've talked a little about reader macros in the past, and that they might be a bad fit for LFE/BEAM ... but even if we couldn't support them fully in LFE, we might be able to do a little with a workaround.
For instance, assuming that, as part of that work, LFE supported a # literal marker (excepting the ones that already exist), the clj library could add macros to support #{...} ... not as nice as having complete control over new syntax, but perhaps better than nothing?

:+1: to reader macros in general. Seems like a good way to please many people too. Maybe we could even do something per file like #lang clj.

I don't understand this POV. Reader macros and custom syntaxes not included in the base language are literally always avoided in Common Lisp. How can you be for this when all it brings is incompatibility between codebases which use reader macros and those which don't?
On one hand you say you aim for making things simple but then advocate using reader macros for people wanting additional simplification? When the common opinion is reader macros make codebases much more complex by adding special rules to individual files rather than having a common language used between them.

When the common opinion

I'm sorry but I don't think there a such thing as "a common opinion" in CL community.
Particularly on this topic: lots of libs provide their reader macros. Never had any problem with that.

@deadtrickster I'm not sure where you interact with the community but definitely the consensus is that reader macros are for very unique situations.

this will never be resolved.


Rename limitedScope option to scope or deploymentScope for clarity

When talking about federation deployment, we talk about 'cluster-scoped' or 'namespace-scoped'.   The option to configure the scope , though, is limitedScope in FederationConfig, with a value of true indicating namespace-scoped and false indicating cluster-scoped.  It would be easier to understand if the argument was scope or deploymentScope and the values were cluster and namespaced (as per crd scope).

/assign
I will handle this.


How to change /data path

I'm trying to use this project outside of home assistant, as the script mentions it should be more generally useable, which rocks.
I notice it's looking to create and read files in /data. The script wouldn't run until I created that directory.  I'm assuming that with Home Assistant that directory normally exists.
If there a way to set that path to some place else? I looked in the code to find where it was set, but haven't found it so far. All I found was that in theory it should save to the home directory. Perhaps /data is baked into the Chip library?

If you start the server with the --help flag you will see the command line options.
This is how it looks for me:
(python-matter-server) x$ python -m matter_server.server --help
usage: __main__.py [-h] [--vendorid VENDORID] [--fabricid FABRICID] [--storage-path STORAGE_PATH] [--port PORT] [--log-level LOG_LEVEL] [--log-file LOG_FILE]

Matter Controller Server using WebSockets.

optional arguments:
  -h, --help            show this help message and exit
  --vendorid VENDORID   Vendor ID for the Fabric, defaults to 65521
  --fabricid FABRICID   Fabric ID for the Fabric, defaults to 1
  --storage-path STORAGE_PATH
                        Storage path to keep persistent data, defaults to /home/martin/.matter_server
  --port PORT           TCP Port to run the websocket server, defaults to 5580
  --log-level LOG_LEVEL
                        Provide logging level. Example --log-level debug, default=info, possible=(critical, error, warning, info, debug)
  --log-file LOG_FILE   Log file to write to (optional).

How are you running the server?

This is customized as part of our CHIP wheels build (see https://github.com/home-assistant-libs/chip-wheels/blob/main/0007-Use-data-as-platform-storage-location.patch). You can build the CHIP wheels from Matter SDK on your own and customize for your needs.
However, ideally this should be a runtime configuration we can control from Python side. But that would require changes to the Matter SDK C++ code, from what I can tell... But  they usually are receptive of PR :wink:

@agners Ah, so that's where it is. I was searching all through the code :-)
Thanks for the clear explanation. I've created a feature request with Matter, as a PR is slightly above my skill level at the moment. For now I'll just hack around it.
@MartinHjelmare I'm running the example script.
Thanks you both for your great work in making experimenting with Matter more accessible.

The patch was renamed (?) from 0007 to 0002: https://github.com/home-assistant-libs/chip-wheels/blob/main/0002-Use-data-as-platform-storage-location.patch


Handle problematic correlated columns group bys

Since this is part of the implicit group by generation functionality I don't see this as a bug per se.
Almost all DBs except for PostgreSQL require that all columns from (complex) expressions need to be listed in GROUP BY although the (complex) expression containing these very columns is already listed in the GROUP BY clause. Try if we can workaround that by using select aliases or ordinales. If not, I am not sure how we could retain the same semantics. Any ideas?

Since this was required to work correctly for the SIZE to COUNT transformation, I implemented it.
Note that DB2, SQL Server and Oracle were problematic because

When parameters occur in an expression, the DBs can't match the expression because of potential different parameter values even if the parameters get the same values assigned
When a subquery occurs in an expression, the DBs fail to group by that per definition. Instead, surrounding expressions and correlated columns have to be put into the group by
When non-deterministic expressions are used like e.g. CURRENT_TIMESTAMP, SQL Server fails to group by that with a cryptic error

Additionally MySQL was problematic in the sense that the HAVING clause strictly required that all columns which are used in it's expressions to appear in the GROUP BY clause. Other databases could match more complex expressions like CASE WHEN and others which already appear in the GROUP BY and thus allow these expressions to be used in the HAVING clause.


:package: missing osx-temperature-sensor

What
/lib/cpu.js has
        try {
          osxTemp = require('osx-temperature-sensor');
        } catch (er) {
          osxTemp = null;
        }

So, it might be better to declare osx-temperature-sensor as a dependency

@chengcyber I excluded this dependency by purpose because I then had a lot of issues on other machines. See also notes in the docs: https://systeminformation.io/issues.html
Hope this is OK for you.


Attempted import error: 'useState' is not exported from 'react'

Describe the bug
I tried to build a nextjs application that uses the tanstack v8 react-table component but I came across the following error:
Failed to compile
<EMAIL_ADDRESS>Attempted import error: 'useState' is not exported from 'react' (imported as 'React').
This error occurred during the build process and can only be dismissed by fixing the error.
Now I'm not sure if this is caused by next, node or the component itself.
Your minimal, reproducible example
https://stackblitz.com/github/sek-consulting/tanstack-usestate-error-mre
Steps to reproduce
open the stackblitz link or use the github repo directly:
https://github.com/sek-consulting/tanstack-usestate-error-mre
Expected behavior
Since this is just a copy/paste of the Basic Example (with stripped down data/columns) I expect it to like similar to this:
https://tanstack.com/table/v8/docs/examples/react/basic
How often does this bug happen?
Every time
Screenshots or Videos

Platform
Windows
Firefox
Node 18.14
react-table version
^8.7.9
TypeScript version
4.9.5
Additional context
No response
Terms & Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

I played around with another project of mine that has the tanstack-table working just fine and step by step tried it to move it the above configuration.
And the error starts to appear when I move from the old pages setup to the new app directory feature.

Hi! did you found out a way to fix it? I'm having the same issue

Unfortunately not.
I set my project on hold until I get some more info on how to fix it here.
In the meantime I switched to t3-stack for a bigger project which doesn't use the app folder and therefore doesn't break the table.

Also experiencing this and unsure how to resolve. Next 13.2.4 and Tanstack Table 8.7.6

@sek-consulting btw, I was able to fix your error:
You're trying to use Tanstack Table in a server component, which doesn't have access to hooks like useState. In order to fix this, you need to let Next know that you want a Client component. You can do this by adding 'use client' to the top of page.tsx.
The trouble for me comes when I create a separate Client component <Table /> that renders the table, then import that into a server component. This should work according to Next's docs but I can't seem to get it working.
The error I get is:
Attempted import error: 'useState' is not exported from 'react' (imported as 'React').```

It seems like createColumnHelper needs to be on the client side component. Make sure you have "use client" on top of any files that uses tanstack tabl

thanks to everyone for helping me out with this! ❤
I now have my table.tsx in a client component and a seperat client component for all my needed tables to wrap the columnHandler stuff and the typing.
Then I can fetch the data in a server component and display it in the client component.
Still need to get used to the new layout and usage of the app directory 😆

I solved this issue by creating a new function which returns the createColumnHelper function. This new function resided in a component that had a "use client" declaration.
export function _createColumnHelper<
  TData extends unknown,
>(): ColumnHelper<TData> {
  return createColumnHelper();
}


Enforce LF line ending in the repository, instead of letting git chose

What I'm trying to achieve
Have the LF line ending enforced in the git repository instead of letting git choose depending on the user's local or global git configuration ; which, on Windows, git line endings are defaulting to CRLF.
Describe a proposed solution
In .gitattributes, add:
# force LF EOL
* text=auto eol=lf

Bump.


Cannot reach coredns server after adding default bgpConfiguration in K8S

After I added a default bgpConfiguration as in https://docs.projectcalico.org/reference/resources/bgpconfig
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  logSeverityScreen: Info
  nodeToNodeMeshEnabled: true
  asNumber: 64512
  serviceClusterIPs:
  - cidr: <IP_ADDRESS>/12
  serviceExternalIPs:
  - cidr: <IP_ADDRESS>/32
  - cidr: <IP_ADDRESS>/32
  - cidr: <IP_ADDRESS>/32

all pod in k8s cannot reach coredns anymore.
/ # nslookup google.com
;; connection timed out; no servers could be reached

Your Environment

Calico version

Client Version:    v3.12.0
Git commit:        84a21b3b
Cluster Version:   v3.11.2
Cluster Type:      k8s,bgp,kdd

Orchestrator version (e.g. kubernetes, mesos, rkt):

Kubernetes 1.17 with 3 nodes

Operating System and version:

Ubuntu 18.04

Are the Calico pods healthy?
What are the CIDRs being used in your cluster?
If you remove the BGPConfiguration does everything go back to working?

The Calico pods are healthy. CIDR I have tried <IP_ADDRESS>/16 and in another setup <IP_ADDRESS>/16.
It did not go back to working even after I removed the bgpconfiguration

Any update on this @shaozi ?

Don't have the opportunity to reproduce it. you can close it for now. Thanks!


Split the ActiveRecord-style Model from the bare Entity

This way, one can choose to use bare entities with an EntityMapper
class (like Doctrine does it) instead of the ActiveRecord style models
that Fluent provides.

Makes sense. I like it!

@Prince2k3, awesome! Does that mean you will merge it? Or can only @tannernelson do that?

@Evertt if @tannernelson agrees. We can merge it. I'd rather a discussion and thoughts before immediate merges/changes to the repo.

@Prince2k3 good point. :-)

This actually makes a lot of sense. Voting for it.

This looks good to me. Getting Fluent updated is going to be top priority when I start working full time next week. It's been lagging behind Vapor.

I'm on the Fluent team now? Wow, so much power, so much responsibility. Do you guys have some kind of rules around merging PRs? For example, so far 4 of the 13 people on the team like this PR, is there a rule that says at least 50% of the team needs to like it before the PR should be merged?

Haha. Yes, with great power comes great responsibility. We don't have any hard rules yet, because so far people have been really thoughtful about when it's appropriate to merge. But that is something we should make more well defined.
I'll make an issue on the main Vapor repo for us to discuss this further.
In terms of this PR, you're definitely good to merge it!

Let's discuss the approval process here:
qutheory/vapor#242


docs/guide/README.md - "Database access objects" [ci skip]

for consistency sake

Nope. https://en.wikipedia.org/wiki/Data_access_object

I see.
But what about the title of the linked section?

Should be changed to Data Access Objects, I think.

@qiangxue it was introduced by you if I remember correctly. Which one was meant to stay?

OK, I'll wait and see.
Besides, I'll delete the branch, anyway.


Sensitive data redaction support in libsinsp

Motivation
When capturing syscalls, we capture all data including sensitive information that should be removed prior to forwarding to a central registry. There is no filter to exclude or redact passwords, secrets and other sensitive data from being saved. The data should be removed at the source.
Feature
Regex capabilities to match sensitive sensitive data
Capabilities to redact (replace with ***) or remove the sensitive data
Alternatives
Considered removing the sensitive data after write - not secure enough
Additional context

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Hi @0snug0 !
i agree with you, we somewhat need this.
I think we can try to give this a shoot for the 0.32 release cycle.
Thanks for the input!
/remove-lifecycle stale

/remove-lifecycle rotten

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/remove-lifecycle rotten

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

First of all @0snug0 yes this is very relevant from a data privacy point of view.
These are the technical hurdles:
Generic regex matching capabilities based on end user input is not yet supported and it may be a bigger ask than it sounds, because of possible performance impacts.
Even if we did already support above, the end user may still encounter significant challenges when attempting to correctly redact logs when applicable as applications constantly change.
By the way do you have concrete examples? For the most part it may be rare cases that the clear text data in question makes it to one the field outputs?
In summary, I do not yet have a good idea on how to support this feature in a sustainable way. Open to hearing more suggestions from the community.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/remove-lifecycle rotten

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


rpc: generate/expose unique auth identifier to grpc handlers

In order to support the future RDK concept of sessions (if approved) in conjunction, we need to expose a unique identifier with an authentication context. This PR provides that along with a tiny bit of cleanup.

Your PR description: "... we need to expose a unique identifier with an authentication context.".
Why does the "control" session need to be tied to the authorization system?


Squash warnings on gcc and clang

@jkwak-work might want to merge this before https://github.com/shader-slang/slang/pull/4659


restart workers after executing 10 tasks to mitigate memory leaks

Fixes #https://github.com/nebari-dev/nebari/issues/2418

Description

This pull request:

restarts celery workers after they execute 10 tasks (docs here).  This mitigates memory leaks which are occurring.

Pull request checklist

[x] Did you test this change locally?
[ ] Did you update the documentation (if required)?
[ ] Did you add/update relevant tests for this change (if required)?

Additional information

How to test

@peytondmurray @kcpevey @trallard Could I get a review on this?

I'm happy to make this configurable via a traitlet as well if desired, but the worker restarts seem happen quickly so I can't see too much down side to just applying it for all conda store deployments.

I can't speak to the technical aspect of this, but I'm very glad to see it. I see the memory leakage on all my deployments. As far as I can tell it leads to an erroneously failed build due to OOM - which also generally fails with errors that make no sense. All around, this is a big admin management and end user UX improvement.

Huh, with this option it looks like celery will just automatically restart the worker after 10 tasks are completed. My one complaint with this is it doesn't strictly speaking fix nebari-dev/nebari#2418 as we are only treating a symptom not the memory usage itself.

I agree it does just treat the symptom without fixing the underlying issue.  That said, I think this will work robustly.

But in any case from what I have read it seems like this is exactly the kind of use case the option was intended for, so I guess let's just use it. Is there an easy way to test this? If so it would be nice to ensure that it works.

You could check out this branch, then run conda standalone https://conda.store/conda-store/how-tos/install-standalone or you can run it in [docker containers](docker-compose up --build -d as explained here).  After you startup, you can visit localhost:8080/conda-store/admin and rebuild the environment it comes with a few times.  If you watch the docker memory usage (docker stats) you'll see that the memory usage does not continue to climb with each new build as it does without this option set.

To test, you could check out this branch, then run conda standalone https://conda.store/conda-store/how-tos/install-standalone or you can run it in [docker containers](docker-compose up --build -d as explained here). After you startup, you can visit localhost:8080/conda-store/admin and rebuild the environment it comes with a few times. If you watch the docker memory usage (docker stats) you'll see that the memory usage does not continue to climb with each new build as it does without this option set.

Ahh, what I meant was is there a simple way of adding automated testing?

Ahh, what I meant was is there a simple way of adding automated testing?

I am +1 with Peyton here. It is good that we can manually verify (once or so), but I would prefer having some sort of automated tests (akin to load tests) to ensure this is, in fact, a useful fix.
Those tests could later be used to also do some benchmarking/memory checks.
So while the fix itself seems fine I would suggest holding on merging until the tests have been added.

I've followed up on where the memory leak is occurring using memray.  I'm new to memary, but the flamegraph seems to point to line 20 in list_conda_prefix_packages at /opt/conda-store-server/conda_store_server/_internal/action/add_conda_prefix_packages.py which is prefix_data.load() here.
I'm not sure what that code is doing or why that would cause a memory leak, but I'm recording it here for now.  I'll try to find out more later or how to do a more targeted fix.

Thanks, this is actually enough to start properly narrowing this down to something we can properly troubleshoot. I still think it would be worth merging this feature as it doesn't seem like there's a real cost to this. I'll open another issue with the goal of actually eliminating the memory leak.

While I can see why add_conda_prefix_packages.py could lead to memory growth (and potential issues if the json.load() and its in-mem representations are not handled accordingly, I cannot see a definite indication that this is indeed the culprit.
add_conda_prefix_packages.py  definitely spends a lot of time in CPU but that does not necessarily mean it is leading to memory leaks or bloat.
So we'd definitely need to do more profiling and mem analysis over time.
@Adam-D-Lewis can you share a minimally reproducible example of how you got to the flame graph you shared.
As per merging, since this PR has been marked as needing tests I think we should still hold until we have a test in place.

I plan to write what I did to generate the memray flamegraph soon.
As far as a test showing the leak, what I'm planning on doing is running a test which builds many conda envs and show that the memory used by the conda store worker docker container continues to rise with each build.


Translation doesn't work anymore

As you can see, I used the example twet :

But translation doesn't seems to work anymore

Translations accidentally broke due to a downstream change I made and fixed with this commit: https://github.com/FixTweet/elongator/commit/337357f46a4d607793c3eaee225cdc8ced24db5f
Translations should be working again. If you are still experiencing issues, feel free to reopen the issue and provide examples!

Seems to be working! Thanks a lot!

Hello, can this issue be reopened? It doesn't work anymore at the time i'm posting this.

Yep, this does appear to be broken atm.
https://fxtwitter.com/DQ_X/status/1722993457654816832/en

Currently looking into why translations are broken now, as they were working as soon as a couple days ago

I pushed the fix above, which should fix translation again. Let me know if you still have any problems. Btw, if you want to add a query string to invalidate the cache, add it after the language code. If all is well then I can close this again.

Confirmed this is working -- thank you!

Also working for me, thanks!

Translations stopped working again

Confirmed, if you try translating, it says post not found. Working on the underlying cause

Last night, i was able to confirm translations had stopped working, but they started working again today. It seems the issue might have been a server-side one.

Still not working for me

If further confirmation is needed: not an isolated case, it doesn't work for me either.

Behavior has changed: tweets are showing, but aren't translated

Seems fixed now?
(I tried earlier and it works, maybe temp so will continue monitor.)


Error while getting a YAML output of a secret

Expected behavior
Get the output colorized
Current behavior
Got stuck after 'data:' key
Package manager
Homebrew
What operating system are you running?
No response
Kubectl and kubecolor version
Client version: v1.29.2
Kubecolor version: v0.4.0

Additional context
After executing kb get secret -n kyverno sh.helm.release.v1.kyverno.v1 -o yaml; with kubecolor the command got stuck.
Running with kubectl I had no problem at all.

Verified bug. Thanks for reporting this @epoch-chrono !
This bug may cut deep, as I believe this is a side effect of us using bufio.Scanner, and could therefore take some time to fix.
The workaround is sadly to not use kubecolor's coloring for looking at Helm secrets, such as:
# via disabling coloring:
kubecolor get secret foo -o yaml --plain

# or if using alias, then you can bypass it via:
\kubectl get secret foo -o yaml


Feature: Interactive Mode

Added command line argument --interactive to use interactive mode for parameter selection to make it more user friendly for novice users :)

It also allows for Smashing Enter to keep everything default
Also, made the stars printed in print_version dynamic :)

I like this interavtive mode!

@agwosdz can you upmerge it? Than i reintegrste it 😀


Update assets to comply with the New NSF Branding Theme

Update documentation site, pages, etc. to reflect the NSF branding policy changes (e.g. NSF NCAR) wherever applicable, e.g. logos,  text, etc.

Similarly to viz, it looks like we just need to update the citation info.

I just went ahead and put in a PR (NCAR/geocat-viz#191) for the corresponding issue on geocat-viz.  Let me know if you'd like me to make those changes here as well once that's reviewed/approved.  I requested a review from you so we can make sure we're all on the same page.

Yes, that PR looked good, please feel free to do that similarly here.


Use case of <base href="/> if i customize this to / test it is not reflecting on url

May it is a silly question, but when should i customize the  tag and why?

Got information from @danroth27
https://www.tutorialspoint.com/html/html_base_tag.htm
https://blazor.net/docs/host-and-deploy/index.html#app-base-path


Support for cancelling all open orders

This adds support for the API call here:
https://binance-docs.github.io/apidocs/spot/en/#cancel-all-open-orders-on-a-symbol-trade
so that you don't have to iterate through the list of open orders cancelling individually.

Looks good to me. Thank you for your contribution.


Possible Bug in I2C [SHTC3 Sensor Module]

Hello, I have successfully run the SHTC3 I2C sensor on STM32, but it does not work with PICO. I only replaced the STM32 HAL I2C commands with PICO SDK I2C commands. I should mention that I just have a problem with this sensor, other I2C staff which I have around work without any issue, but the reason for this weird problem is still unknown. The sensor itself is fine and works with other hardware. no hardware problem.
Header File
#ifndef __SHTC3_H
#define __SHTC3_H
#include "pico/stdlib.h"

// The shtc3 provides a serial number individualized for each device
// Params:
//  - `hi2c` I2C bus
// Returns device id or 0 in case of error.
uint16_t shtc3_read_id();

// Put sensor into sleep mode
// Params:
//  - `hi2c` I2C bus
// Returns device id or 0 in case of error.
uint32_t shtc3_sleep();

// Wake up sensor.
// You must wait for 240us to let sensor enter into IDLE mode.
// Params:
//  - `hi2c` I2C bus
// Returns zero in case of error
uint32_t shtc3_wakeup();

// Performs full cycle: starts temperature/humidity measurements using "clock stretch" method.
// Params:
//  - `hi2c` I2C bus
//  - `temp` measured temperature, in C multiplied by 100 (e.g. 24.1C -> 2410)
//  - `hum` measured relative humidity, in percents
// Returns zero in case of error
uint32_t shtc3_perform_measurements();

// Start temperature/humidity measurements using "clock stretch" approach, in low power mode.
// After completed - values can be obtained by shtc3_read_measurements()
// Params:
//  - `hi2c` I2C bus
//  - `temp` measured temperature, in C multiplied by 100 (e.g. 24.1C -> 2410)
//  - `hum` measured relative humidity, in percents
// Returns zero in case of error
uint32_t shtc3_perform_measurements_low_power();

#endif

Source
#include "shtc3.h"
#include "hardware/i2c.h"

#define SHTC3_ADDRESS_READ          (0x70 << 1) | 0x01
#define SHTC3_ADDRESS_WRITE         (0x70 << 1)

#define SHTC3_PRODUCT_CODE_MASK     0x083F
#define SHTC3_SENSOR_ID_MASK        0xF7C0

// NOTE: all commands are "byte swapped" (ntohs), meaning:
// 0x3517 -> 0x1735

#define SHTC3_CMD_WAKEUP                                    0x1735
#define SHTC3_CMD_SLEEP                                     0x98B0
#define SHTC3_CMD_SOFT_RESET                                0x5D80
#define SHTC3_CMD_READ_ID                                   0xC8EF

// Clock stretching based commands
#define SHTC3_CMD_CLK_STRETCH_READ_HUM_FIRST                0x245C
#define SHTC3_CMD_CLK_STRETCH_READ_HUM_FIRST_LOW_POWER      0xDE44

// Polling commands
#define SHTC3_CMD_POLL_HUM_FIRST                            0xE058
#define SHTC3_CMD_POLL_HUM_FIRST_LOW_POWER                  0x1A40

uint16_t shtc3_read_id()
{
  uint8_t data[2];
  int16_t command = SHTC3_CMD_READ_ID;

  i2c_write_blocking(i2c0, SHTC3_ADDRESS_WRITE, (uint8_t *)&command, 2, false);

  i2c_read_blocking(i2c0, SHTC3_ADDRESS_READ, (uint8_t *)data, 2, false);

  // SHTC3 16 bit ID encoded as:
  // xxxx 1xxx xx00 0111
  // where "x" are actual, sensor ID, while the rest
  // sensor product code (unchangeable)
  uint16_t id = data[0] << 8 | data[1];
  uint16_t code = id & SHTC3_PRODUCT_CODE_MASK;
  if (code == 0x807) {
    // Sensor preset, return actual ID
    return id & SHTC3_SENSOR_ID_MASK;
  }

  return 0;
}

uint32_t shtc3_sleep()
{
  uint16_t command = SHTC3_CMD_SLEEP;
  uint32_t res = i2c_write_timeout_us(i2c0, SHTC3_ADDRESS_WRITE, (uint8_t*)&command, 2, false , 5000);

}

uint32_t shtc3_wakeup()
{
  uint16_t command = SHTC3_CMD_WAKEUP;
  uint32_t res = i2c_write_timeout_us(i2c0, SHTC3_ADDRESS_WRITE, (uint8_t*)&command, 2, false, 5000);

}

static uint32_t checkCRC(uint16_t value, uint8_t expected)
{
	uint8_t data[2] = {value >> 8, value & 0xFF};
	uint8_t crc = 0xFF;
	uint8_t poly = 0x31;

	for (uint8_t indi = 0; indi < 2; indi++) {
		crc ^= data[indi];
		for (uint8_t indj = 0; indj < 8; indj++) {
			if (crc & 0x80) {
				crc = (uint8_t)((crc << 1) ^ poly);
			} else {
				crc <<= 1;
			}
		}
	}

	if (expected ^ crc)	{
    return 0;
	}
  return 1;
}

static uint32_t _read_values(uint8_t* data, int32_t* out_temp, int32_t* out_hum)
{
  // Check CRC
  uint32_t raw_hum = data[0] << 8 | data[1];
  uint32_t raw_temp = data[3] << 8 | data[4];

  if (!checkCRC(raw_hum, data[2])) {
    return 0;
  }
  if (!checkCRC(raw_temp, data[5])) {
    return 0;
  }

  // Convert values
  if (out_hum) {
    *out_hum = raw_hum * 100 / 65535;
  }
  if (out_temp) {
    *out_temp = raw_temp * 17500 / 65535 - 4500;
  }

  return 1;
}

static uint32_t _perform_measurements(uint16_t command, int32_t* out_temp, int32_t* out_hum)
{
  uint8_t result[6];

  uint32_t res = i2c_write_timeout_us(i2c0, SHTC3_ADDRESS_WRITE, (uint8_t*)&command, 2, false, 5000);

  res = i2c_read_timeout_us(i2c0, SHTC3_ADDRESS_READ, result, 6, false, 15000);

  return _read_values(result, out_temp, out_hum);
}

uint32_t shtc3_perform_measurements(int32_t* out_temp, int32_t* out_hum)
{
  return _perform_measurements(SHTC3_CMD_CLK_STRETCH_READ_HUM_FIRST, out_temp, out_hum);
}

uint32_t shtc3_perform_measurements_low_power(int32_t* out_temp, int32_t* out_hum)
{
  return _perform_measurements(SHTC3_CMD_CLK_STRETCH_READ_HUM_FIRST_LOW_POWER, out_temp, out_hum);
}

To run
shtc3_perform_measurements(&temperature, &humidity);

I suspect that the Raspberry Pi engineers are all too busy to help with debugging random I2C devices? So you may be better off asking for help at https://forums.raspberrypi.com/viewforum.php?f=143
If you do want help from a Raspberry Pi engineer, you'll probably want to provide logic-analyser traces for both the working (STM32) and non-working (RP2040) I2C transactions, so that they can see what any differences might be.

Alright, I captured the signal using an oscilloscope. in both cases, I just call the function shtc3_read_id()
STM32

PICO

Why are you shifting the addresses by one?
#define SHTC3_ADDRESS_READ          (0x70 << 1) | 0x01
#define SHTC3_ADDRESS_WRITE         (0x70 << 1)

Have you tried not doing that? Pico SDK takes care of setting the R/W bit for you. Therefore, you have to pass the 7 bit address to the SDK functions:

i2c_write_blocking
[...]
addr: 7-bit address of device to write to

Why are you shifting the addresses by one?

I tried your suggestion, and wow, thank you very much, how did you know that? I read everywhere
I read everywhere, but I was not aware of your point, below worked, so as you said, no need to differentiate the address for read and write
#define SHTC3_ADDRESS_READ          (0x70)
#define SHTC3_ADDRESS_WRITE         (0x70)

Ah great, so this can be closed now?

I tried your suggestion, and wow, thank you very much, how did you know
that?

Perhaps a little bit of background info helps: I2C (usually) has 7-bit
addresses. When starting a transaction, the address is immediately
followed by a bit that indicates a read or write command. Apparently,
with your previous hardware, you had to include this bit in the address.
This is why you had the left shift (and "| 0x01" in case of the read).
However, with the Pico SDK, and as described in its documentation, you
only pass the 7-bit address of the I2C device. The Pico SDK already
knows from the function call whether it is a read or write command.


gCNV merge + annotate

Draft Stages

throws a quick BCFtools merge on the already co-called single-sample VCFs
abstracts out the SV VCF annotation from the GATK-SV stages, both gCNV and GATK-SV stages call the same method

numerous rebases since opening, dealing with overlapping parts of the codebase - filtering workflow, billing labels

Re the most recent commit, I actually like the technique of using add_extension rather than artificially bundling files into a group via  declare_resource_group…

Re the most recent commit, I actually like the technique of using add_extension rather than artificially bundling files into a group via declare_resource_group…

Yees, but you're missing the indexes - the index files are created as an implicit path (other file + tbi), so you can't write them to a nominated output path, which means you can't read them into permanent storage

FYI I am unable to review this PR. It makes unrelated changes to functions that it doesn't actually affect, for example formatting changes (which are presumably fallout from black), so there is too much to meaningfully review.

cancelled in favour of fragmented PRs


Merge test fixes/test matrix to vs2015

@KevinRansom @jaredpar

@KevinRansom  it looks like appvoyer didn't run the build correctly. Here is a passing one from my account:
https://ci.appveyor.com/project/otawfik-ms/visualfsharp/build/<IP_ADDRESS>


Semantic Release Test for Workflow

This PR is made to test out the semantic versioning in the github action workflows.

Codecov Report

Merging #41 (30aaec5) into main (2765f9a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #41   +/-   ##
=======================================
  Coverage   97.05%   97.05%
=======================================
  Files           5        5
  Lines          34       34
  Branches        1        1
=======================================
  Hits           33       33
  Misses          1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2765f9a...30aaec5. Read the comment docs.

The Github Actions are working for semantic release
Can you help me in reviewing this one @vinayaksh42


Make function to create a playlist from MusicItemIDs

I have a use case where I only have access to the ID of songs and not the full Song object

Awesome! Thank you for using MusadoraKit!!


Add technical debt log items and template

Adds a new default log type for technical debt.
Debt items act similarly to Caveats.

Codecov Report

Merging #12 (98376fe) into main (8529fe9) will decrease coverage by 0.29%.
The diff coverage is 42.85%.

@@            Coverage Diff             @@
##             main      #12      +/-   ##
==========================================
- Coverage   50.00%   49.70%   -0.30%
==========================================
  Files           4        4
  Lines         154      167      +13
==========================================
+ Hits           77       83       +6
- Misses         77       84       +7

Flag
Coverage Δ

pytests
49.70% <42.85%> (-0.30%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

assumptions/cli.py
18.51% <0.00%> (-2.32%)
:arrow_down:

assumptions/log_items.py
63.63% <55.55%> (-1.28%)
:arrow_down:

assumptions/log.py
45.71% <100.00%> (+0.78%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8529fe9...98376fe. Read the comment docs.


Can  you write a guide that works with Plesk onyx on linux?

Just trying to get this to work with Plesk Onyx, and it would be super awesome if your guides worked well with plesk onyx. Thanks.

I don't have time and I don't use Plesk Onyx. Sorry

@Pangamma What are you trying to achieve?

Well it would be super cool to be able to have this console logging on my Linux server, but Plesk does not allow you to modify the root location in the nginx configuration file. Also, it has an application for running node apps but need to figure out how to set that up and which settings to give it. That's what I was trying to accomplish. I eventually gave up.

@Pangamma Why do you need this logging? Can't you just view or tail the logs directly on the server?

The whole point of this is to log front end client sided console logs right?

The whole point of this is to log front end client sided console logs right?

Well, you can use it on server with CLI to send logs to your consolere sever
npm install console-remote-client -g

@futureweb if you don't have root shell access I don't think you can install in via Plesk UI. If you have shell root access, install dockers and use this guide to run it as a docker https://github.com/kurdin/console-remote-server/tree/main/docker

@kurdin - nice, thx! 👍🏻
https://hub.docker.com/r/yavuzslmtpl/console-remote-server


No module named 'mmdet.models.dense_heads.atss_rpn_head'

Could you upload this file? Thanks.

Could you upload this file? Thanks.

I will upload it soon, thanks~

Could you upload this file? Thanks.

I have fixed it. Please have a check.

I'm sorry, there are still many new errors. It may require you to successfully run it once to see what files are missing.

I'm sorry, there are still many new errors. It may require you to successfully run it once to see what files are missing.

I have fixed the errors and run it simply. Maybe you can have a try now.
If you meet the file missing problem, just try deleting this error sentence in init.py

Thank you, it has been successful training.


Failed command (python3) ensurepip is not available on upgrade.

Description
Upgrade script fails
What's the behavior that you expect?
Upgrade script succeeds
What's happening instead?
+ pushd /opt/tinypilot-updater
/opt/tinypilot-updater ~
+ python3 -m venv venv
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt-get install python3-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/opt/tinypilot-updater/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

What are the steps to reproduce this behavior?

ssh into raspberrypi
sudo to root
run /opt/tinypilot/scripts/upgrade

Logs
root@raspberrypi:~# uname -a
Linux raspberrypi 5.10.17-v7l+ #1414 SMP Fri Apr 30 13:20:47 BST 2021 armv7l GNU/Linux

root@raspberrypi:~# cat /etc/issue
Raspbian GNU/Linux 10 \n \l

root@raspberrypi:~# /opt/tinypilot/scripts/upgrade
+ set -u
+ set -e
+ curl --silent --show-error https://raw.githubusercontent.com/mtlynch/tinypilot/master/quick-install
+ bash -
+ readonly DEFAULT_TINYPILOT_SETTINGS_FILE=/home/tinypilot/settings.yml
+ DEFAULT_TINYPILOT_SETTINGS_FILE=/home/tinypilot/settings.yml
+ [[ -n '' ]]
+ set -u
++ mktemp --tmpdir=/var/tmp --suffix .yml
+ readonly TINYPILOT_SETTINGS_FILE=/var/tmp/tmp.iBLBLtHvm3.yml
+ TINYPILOT_SETTINGS_FILE=/var/tmp/tmp.iBLBLtHvm3.yml
+ [[ -f /home/tinypilot/settings.yml ]]
+ echo 'Using settings file at: /home/tinypilot/settings.yml'
Using settings file at: /home/tinypilot/settings.yml
+ sudo cp /home/tinypilot/settings.yml /var/tmp/tmp.iBLBLtHvm3.yml
+ readonly<EMAIL_ADDRESS>+<EMAIL_ADDRESS>+ add_setting_if_undefined ustreamer_port 8001
+ local key=ustreamer_port
+ local value=8001
+ grep --silent '^ustreamer_port:' /var/tmp/tmp.iBLBLtHvm3.yml
+ add_setting_if_undefined ustreamer_persistent true
+ local key=ustreamer_persistent
+ local value=true
+ grep --silent '^ustreamer_persistent:' /var/tmp/tmp.iBLBLtHvm3.yml
+ USE_TC358743_DEFAULTS=
+ grep --silent '^ustreamer_capture_device:' /var/tmp/tmp.iBLBLtHvm3.yml
+ '[' -f /home/ustreamer/config.yml ']'
+ grep --silent 'capture_device: "tc358743"' /home/ustreamer/config.yml
+ [[ '' == \y ]]
+ add_setting_if_undefined ustreamer_encoder hw
+ local key=ustreamer_encoder
+ local value=hw
+ grep --silent '^ustreamer_encoder:' /var/tmp/tmp.iBLBLtHvm3.yml
+ add_setting_if_undefined ustreamer_format jpeg
+ local key=ustreamer_format
+ local value=jpeg
+ grep --silent '^ustreamer_format:' /var/tmp/tmp.iBLBLtHvm3.yml
+ add_setting_if_undefined ustreamer_resolution 1920x1080
+ local key=ustreamer_resolution
+ local value=1920x1080
+ grep --silent '^ustreamer_resolution:' /var/tmp/tmp.iBLBLtHvm3.yml
+ echo 'Final install settings:'
Final install settings:
+ cat /var/tmp/tmp.iBLBLtHvm3.yml
ustreamer_encoder: hw
ustreamer_format: jpeg
ustreamer_persistent: true
ustreamer_port: 8001
ustreamer_quality: 50
ustreamer_resolution: 1920x1080
+ HAS_PRO_INSTALLED=0
++ dirname bash
+ SCRIPT_DIR=.
+ '[' . = . ']'
+ SCRIPT_DIR=/opt/tinypilot
+ TINYPILOT_README=/opt/tinypilot/README.md
+ '[' -f /opt/tinypilot/README.md ']'
++ head -n 1 /opt/tinypilot/README.md
+ '[' '# TinyPilot' = '# TinyPilot Pro' ']'
+ '[' 0 = 1 ']'
+ sudo apt-get update
Hit:1 http://archive.raspberrypi.org/debian buster InRelease
Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
Reading package lists... Done
+ sudo apt-get install -y git libffi-dev libssl-dev python3-dev python3-venv
Reading package lists... Done
Building dependency tree
Reading state information... Done
git is already the newest version (1:2.20.1-2+deb10u3).
libffi-dev is already the newest version (3.2.1-9).
python3-dev is already the newest version (3.7.3-1).
python3-venv is already the newest version (3.7.3-1).
libssl-dev is already the newest version (1.1.1d-0+deb10u6+rpt1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
+ INSTALLER_DIR=/opt/tinypilot-updater
+ sudo mkdir -p /opt/tinypilot-updater
++ whoami
++ whoami
+ sudo chown root:root --recursive /opt/tinypilot-updater
+ pushd /opt/tinypilot-updater
/opt/tinypilot-updater ~
+ python3 -m venv venv
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt-get install python3-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/opt/tinypilot-updater/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

python3-venv is already installed:
root@raspberrypi:~# apt policy python3-venv
python3-venv:
  Installed: 3.7.3-1
  Candidate: 3.7.3-1
  Version table:
 *** 3.7.3-1 500
        500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
        100 /var/lib/dpkg/status

and LANG is set:
root@raspberrypi:~# echo $LANG
en_GB.UTF-8

Thanks for reporting this! Does this reproduce consistently on a clean Raspbian install?

Hi,
From a fresh clean install, it appears to work. I executed the upgrade script twice, with a reboot inbetween.
I will close the ticket.
-=david=-


⚠ GitLab Server has degraded performance

In 2e3866e, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 11164 ms

Resolved: GitLab Server performance has improved in c0bcd9f.


fix: Version update user analytics

Collects more info about how users are updating their app to help us understand the effectiveness of current update prompts

/ok-to-test


Bump docker base images

Amazon linux: 2017.12 => 2
Ubuntu: 18.04 => 20.04

@brikis98 FYI I just realized I don't have access to the docker hub repos to push these images. Is it possible to grant me access, or grant the gruntworkci user access?

Closing as stale


test: Fix awk syntax error.

Description
Example:
INFO: Wed May 8 10:01:20 UTC 2024: Polling for images required for system tests
awk: cmd. line:1: print $1
awk: cmd. line:1: ^ syntax error
INFO: Wed May 8 10:01:20 UTC 2024: Will poll for:

Introduced in #10750 🤦🏻
Checklist

[x] Investigated and inspected CI test results
[x] ~Unit test and regression tests added~
[x] ~Evaluated and added CHANGELOG entry if required~
[x] ~Determined and documented upgrade steps~
[x] ~Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)~

If any of these don't apply, please comment below.
Testing Performed
Here I tell how I validated my change

[x] inspected logs from CI

Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

I'll merge it because it's already better, but I'm wondering if I should make this one long line 🤔
WDYT @gavin-stackrox ?
INFO: Wed May  8 10:31:22 UTC 2024: Polling for images required for system tests
INFO: Wed May  8 10:31:23 UTC 2024: Will poll for: central-db
collector
collector-slim
main
scanner
scanner-db
scanner-v4
scanner-v4-db
stackrox-operator
stackrox-operator-bundle
stackrox-operator-index
INFO: Wed May  8 10:31:23 UTC 2024: Checking for central-db using https://quay.io/api/v1/repository/rhacs-eng/central-db/tag?specificTag=4.4.x-637-g56f0fd8515


feat: migrate to API rewrite

Summary
Migrate PeterPortal Client to the API rewrite. After [too long] in development, hopefully it will have been worth the wait.
TODO before merging: remove the hardcoded API URLs and update the CI variables to point to the API rewrite.
Test Plan
Overall: Check that the site still works like it does before, and doesn't crash. I tested most cases I could think of, but there are probably some edge and corner cases that I missed since I'm not that familiar with the codebase.
Specific issues:

#238: Check that the prerequisites are correct.
#301: It seems that Prof. Zhao no longer teaches at UCI (as of the last instructor scrape) so the case cannot be reproduced directly. However, UCI maintains a list of Chancellor's Professors here, and all of them seem to have the apostrophe rendered correctly.
#343: Check that Stats 67 and Math 3A share "AP CALCULUS BC" as a prerequisite, and that one entry in Peter's Roadmap satisfies the prerequisite for both courses.
#350: Check that Bio Sci 48 shows up in the courses tab.

Known Issues and Next Steps

#352
#305
#353

Issues
Closes #224; closes #238; closes #301; closes #343; closes #344; closes #350.

The term indicator issue is tracked on the API end, and I can write a solution for it before this gets merged if needed, but most of the substantial bugs and blockers should be resolved.

The term indicator issue is tracked on the API end, and I can write a solution for it before this gets merged if needed, but most of the substantial bugs and blockers should be resolved. Thanks for the help and sorry about the delay.

All good, it can wait since we still have an outstanding issue on our end to make the term indicator relevant.

The latest staging build now points to an API staging instance which has the GraphQL change. It still seems quite slow to me which makes me think this is a symptom of a larger issue. Could you confirm whether this is the case?

Yeah, still seems slow.

Also, the review form is still broken. Seems to just be that the GraphQL response for professors is not mapped properly.

ucinetid should be the keys

Some really rough performance measurements from throwing print statements in a few places. In general, API rewrite does seem to be a lot slower at responding for course queries, instructors queries seem fine from what I can tell (this is with the batch REST requests for courses and single GraphQL query for instructors). Some improvement could probably be done on our end as well so I might draft up a PR for that.
API time is time measured from right before the request(s) are sent by the express server to after they're all received. Total was measured around the frontend's request to our express server (and maybe some other stuff in there, again really rough measurement).
api rewrite
search page

query
courses total search time (ms)
courses API time (ms)
professors total search time (ms)
professors api time (ms)

Empty
1011
475
1997
5

compsci
769
320

mike

364
1.29

compsci 121
668
241

course page

course
total time (ms)
API time (aggregated) (ms)

compsci 121
1252
232 + 242

professor page

professor
total time (ms)
API time (aggregated) (ms)

mikes
1655
3 + 826

original
search page

query
courses total search time (ms)
courses API time (ms)
professors total search time (ms)
professors api time (ms)

Empty
540
3
547
3

compsci
332
3

mike

211
2.5

compsci 121
233
2.5

course page

course
total time (ms)
API time (aggregated) (ms)

compsci 121
664ms
3ms

professor page

professor
total time (ms)
API time (aggregated) (ms)

mikes
1031
2.6

Thanks for running the numbers. They're definitely worse than I thought; while I suspect it is mostly cold start and/or DB overhead, it's still a pretty hefty penalty even if it doesn't happen that often. I'll look into this further after Wednesday since I should be done with everything except finals by then.

Ran a couple more tests on just the search page with GraphQL being used for the course queries. My conclusion is that we should definitely avoid any batch requests. I'm not too sure what went wrong on the course page though, I wouldn't think 3 requests would be an issue, might need to test again with nullable course query.

query
courses total search time (ms)
courses API time (ms)

Empty
855
1.8

compsci
962
2

Something's up with the prisma set up on the staging instances. Following endpoints aren't working:

Thanks for pointing this out, I think I forgot to upgrade the query engine version to the one that the Node 20 lambda runtime supports. Pretty sure things should be working now though.

Is there a reason why you didn't include the nested GraphQL queries for the api rewrite? I feel like it's been causing a bit of a headache.

Yeah now that I'm actually integrating the new API with PPC it seems like there was a reason that things were done a certain way. (Go figure.)
I think a way forward that we could consider is to have a separate sub-endpoint that returns nested entries for courses/instructors. I have a rough idea of how to implement this but will still need to work through it, and I'm unsure as to what the performance will look like compared to fetching everything, but it stands to reason that we could reduce the overhead of making multiple HTTP requests.

I think the performance issues should be resolved for good this time but can't tell for sure due to regional latency. Could you check on your end?


[Question] Recommended Discord Server/Channels

Do you have any recommended server/channel combinations? I am in TriviaCentral, but I don't think I'll be able to put my Discord bot in there.

At the moment, many Discord servers do not allow you to run your own bot just for yourself. Thus, HQTrivia is designed for selfbot usage. This means you can use your own profile.
Technically, Discord does not recommend this but this bot will never reply to anyone, it simply logs incoming messages and checks to see if they're votes. So you can use your own profile on a browser and have the bot running the same profile and everything will work fine (it's actually what I do).

Thanks. That's kind of what I figured you were doing.
Is there a way to test my config and make sure it is connecting and reading messages properly before the game is live?

Yes. I will push a major commit later today that will add Cash Show and Joyride support. You'll be able to see the connect status in the log, but I'll add a label soon to show this in the UI.

You're awesome. I was actually about to start looking into adding Cash Show support. I'm not near the coder you are though and it would have taken me weeks if I was ever able to get it.

It will only support Discord votes for CS/JR at the moment, but I will look into adding OCR support for these shows in the future (no public websocket, unfortunately). Either way, it should help a lot for playing these games provided you're in 2-3 good servers.


Translations

Vielen Dank,
There is an error in your code missing a semicolon on line 260.

Thanks


emoji picker: fix closing immediately on mobile

This stops the emoji picker from closing immediately on mobile by removing the search input.
Disabling autoFocus for the input isn't enough, if the user selects the input the picker will still close.
This stops the bleeding while we figure out what is causing the picker to close when the search within the picker is focused (or we switch to another picker or another method for displaying the picker).
Also pops the picker up closer to the message itself since we don't have to worry about the keyboard anymore.
Fixes #2253

I have a feeling people will be pretty upset if we take the search input away.
Is this turning into something where we evaluate another approach altogether? This has given us nothing but trouble.

kinda feels like it, yeah. I could go back to looking into other emoji picker options?

Note that this only removes the search input on mobile, though. Also note that this issue only seems to happen with latest Android. Not sure how we could detect if the user is on the latest version of Android, but we could detect if they're on Android. Would be a regression for everyone on Android that isn't on the latest version, though.


Update yml and developer guide use cases

Fixes #51. For v1.1 documentations.

Codecov Report

Merging #56 (7bac065) into master (201e2e1) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #56   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 201e2e1...7bac065. Read the comment docs.


Prevent calling SSL_set_session in the middle of handshake

ATS crashes if origin session cache is used with BoringSSL, because BoringSSL doesn't allow you to call SSL_set_session after starting handshake.
The crash is not 100% reproducible and I was only able to reproduce it on 9.2.x, but the change should make sense on master as well.
https://www.openssl.org/docs/man1.1.1/man3/SSL_set_session.html

If there is already a session set inside ssl (because it was set with SSL_set_session() before or because the same ssl was already used for a connection), SSL_SESSION_free() will be called for that session. If that old session is still open, it is considered bad and will be removed from the session cache (if used). A session is considered open, if SSL_shutdown(3) was not called for the connection (or at least SSL_set_shutdown(3) was used to set the SSL_SENT_SHUTDOWN state).

https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_set_session

It is an error to call this function after the handshake has begun.

The change is very small if you hide whitespace (indent) changes.

Cherry-picked to v9.2.x


Quest plugin issue with Tracking

I have latest Build 3.9.5 and seem to have a issue where every time a player moves on the server it sends permissions info on my verbose  saying Quests.* , Quests.Quest.
If 10 people run around it can cause tps to drop using 8gb of ram running on a KVM 2693 Mhz 8 Core Logical Processer
With A pci super express ssd drive that goes over 6k  mbps and still get alittle lag from this issue ive changed the strick player movement from -1 to 200 and none of these seem to change the checker for this if u have a verbose on your server like luck perms or spartan u could see these as well

Thanks for the report. Would you mind checking if the issue still occurs on the latest development build? https://ci.codemc.io/job/PikaMug/job/Quests/
Also, you've enabled advanced tracking by setting strict-player-movement to any value other than -1, so you may wish to revert that as your new setting will cause a (small) spike every 200 seconds.

yes this problem still Happens every time a player moves also changing the strict-player-movement dosnt affect this issue for it happens no matter what u set that two... another issue we seem to not be able to fix is the prefix for Money being Dollars cant seem to find why it cant be changed to something else but dont see it on your jar file as an item is it using vault or another plugin to read the dollar prefix when u get payed for a quest

It doesn't affect this issue because the setting isn't related - but setting it to anything other than -1 will negatively impact the server in exchange for more precise tracking when riding Armor Stands. So unless you're running a vehicles plugin, set it back to -1.
The money prefix is determined by Vault in most cases, but you may wish to adjust the money: setting in your Quests language file as well.
Anyway, I'll look into the issue shortly.

@DeadlyBladez Please try the latest development build and let me know whether or not it improves performance.
https://ci.codemc.io/job/PikaMug/job/Quests/141/
Note that the build may not be fully compatible with QuestsGUI if you have that installed.


fix: renames fleetscope and appfactory folders

This PR renames 3-appfactory to 4-appfactory and 4-fleetscope to 3-fleetscope

I don't know if that is intentional, but you will be losing the terraform.tfvars symlinks under fleetscope
maybe thats because tfvars is under .gitignore
you can force add the tfvars file by using git add -f filename


DataLogger: cant add non-ascend x value data point into DataLogger

Question: (Ask your question here)
when i try to add non-ascend x value data point into data logger, i get a exception say its not allowed.
i checked  all issues, found  #3743 which was published at apr 27 says its fixed, so i went to download the 5.0.30 which was released at apr 27, and it works just fine. then i checked all release's update logs after 5.0.30, it seems no one metioned datalogger update on this feature, but i cant take advantage of this feature in the latest version.
ScottPlot Version: (What NuGet package are you using?)
5.0.40
Code Sample: See http://scottplot.net/faq/repro/ for tips about creating reproducible code samples
ScottPlot.Plot myPlot = new();
/
/* a code sample may improve communication */
/
![屏幕截图 2024-10-24 155026](https://github.com/user-attachments/assets/da006d5d-2831-481d-aa14-b5f6d942ad66)

Hi @ZooMEISTER,
The changes you highlighted relate to PR #4114. In this PR, the dataLogger is based on a logic for drawing similar to SignalXY. This allows millions of points to be drawn without performance loss, but imposes a limitation on the ordering of points along the X axis.
A typical use case for DataLogger I think is data from time, which is always ordered, so this change was easily accepted.
It's hard to say how to meet your request, the original solution is redesigned in place rather than left alone, so there is no easy solution to add an additional DataLogger(Scatter).

Thank you for your time ! @StendProg
I wonder if you can give me some tip on plot selection
My requests are, the plot should be able to:

display dynamic data and don't need to refresh whole plot every time new data comes in ( like data steamer, but data streamer need to "Plots data as a fixed width line plot" )
the non-ascend data value is acceptable for x-axis ( like scatter, but scatter need to redraw whole plot every time new data comes in)
its in WPF.

actually the DataLogger before 5.0.38 perfectly meet my request, but after that, the new DataLogger doesn't allow non-ascending x-axis value

the non-ascend data value is acceptable for x-axis

You have no choice but to use the Scatter rendering.
Here are 4 ways you can go about it, from simple to complex:

stay on version 5.0.37, but you will lose the fixes/improvements of the new versions.
Use Scatter and implement your own X-axis scale control logic similar to DataLogger, but only in the part you need.
Implement a custom DataLoggerScatter Plottable based on the 5.0.37 source code and use it in conjunction with the latest version of ScottPlot.
Make a Pull Request with DataLoggerScatter from step 3 to the main ScottPlot library branch.

got it, thank you


Columns in Function not suppposed to be in group by

See the following query
select
	sum(caaggregat0_.CAAS_ACTUAL_SPEND) as col_0_0_,
	sum(caaggregat0_.CAAS_FIRST_PRICE_SPEND) as col_1_0_,
	caaggregat0_.CAAS_AGREED_CUR as col_2_0_,
	year(caaggregat0_.CAAS_PERIOD) as col_3_0_
from
	CA_AGGREGATED_SPEND caaggregat0_
inner join
	CA_CATEGORY cacategory1_ on caaggregat0_.CAAS_CAC_ID=cacategory1_.CAC_ID
inner join
	CA_FACTORY_GROUP cafactoryg2_ on caaggregat0_.CAAS_CAFG_ID=cafactoryg2_.CAFG_ID
inner join
	CA_SUPPLIER casupplier3_ on caaggregat0_.CAAS_CAS_ID=casupplier3_.CAS_ID
inner join
	CA_ERP_SYSTEM caerpsyste4_ on casupplier3_.CAS_CAE_ID=caerpsyste4_.CAE_ID
where
	cafactoryg2_.CAFG_CCC_ID=1
and
	caerpsyste4_.CAE_CCC_ID=1
and
	cacategory1_.CAC_CCC_ID=1
and
	(year(caaggregat0_.CAAS_PERIOD) in (2013,2014,2015))
group by
	caaggregat0_.CAAS_AGREED_CUR , caaggregat0_.CAAS_PERIOD , year(caaggregat0_.CAAS_PERIOD)
order by
	col_3_0_ ASC

As you can se the CAAS_PERIOD AND the YEAR(CAAS_PERIOD) are in the group by clause but only the YEAR(CAAS_PERIOD) should occurre there.
It breaks the grouping by the year and results in much larger result set as expected.

The following criteria has been used to get this query.
        final CriteriaBuilder<Object> cb = cbf.create(getEm(), Object.class).from(CAAggregatedSpend.class, "ca");
        cb.select("SUM(ca.actualSpend)");
        cb.select("SUM(ca.firstPriceSpend)");
        cb.select("ca.agreedCurrency");
        cb.select("FUNCTION('YEAR', ca.period)", "years");
        cb.where("ca.supplier.erpSystem.customer.id").eq(customer.getId());
        cb.where("ca.category.customer.id").eq(customer.getId());
        cb.where("ca.factoryGroup.customer.id").eq(customer.getId());
        cb.where("FUNCTION('YEAR', ca.period)").in(2013,2014,2015);
        // Was not able to define FUNCTION('YEAR',ca.period) and alias not allowed here too, so builder added it along with ca.period
        cb.groupBy("ca.agreedCurrency");
        cb.orderBy("years", Boolean.TRUE, Boolean.TRUE);

I took a look at it and I think the only real solution would be to allow full expressions in the group by. I will implement the support for that in the next few days.

Thanks


Bug OCPBUGS-3296: PTP Event (AMQ) : Failed to get GetCurrentState for Events Clock Class Change

get current-state was returning timeout for clock class .
This PR fixes that call
Signed-off-by: Aneesh Puttur<EMAIL_ADDRESS>

/lgtm

/cherry-pick release-4.12