[NVIDIA XLA] Fix Fp32 biasAdd fusion for FP8 cublasLt matmul

Cast Fp32 vector bias to Bf16 to be compatible with  cuBLASLT Fp8 support surface. @reedwm @philipphack

@reedwm  Sorry for informing late. We decided not to cast F32->BF16 directly by XLA since it breaks what we previously did. Instead, our proposal is to only fuse the following pattern:
y_fp32 = matmul(a, b) + broadcast(cast(cast(bias, bf16), fp32))  ---> y_fp32 = cublas-custom-call(a, b, cast(bias, bf16))

Should I not merge this PR then? Why does casting F32->BF16 break what you previously did? It seems weird to require the user to cast the bias to bf16 then back to f32.

Yes, please don't merge this PR. User will not normally do that.  Casting the bias to bf16 then back to f32 is only for FP8 purpose.


Heroku button should set up auto-deploy

Will be fixed by https://github.com/theodi/bothan-deploy/pull/3


Explorer to show more "overall metrics"

What's the problem? (required)
I am unable to easily access information about the overall network in terms of tokens, bonding, reward amounts, meaning that as a token holder, I don't have much context about the bigger picture and how I fit into it.
What's the current behavior? (required)
Currently, the explorer shows a whole heap of information at the level of an individual address. But limited information which puts that address in context in the bigger picture.
Do you have any personal experiences dealing with this problem? (optional)
When explaining to people about the token mechanics, it isn't easy for them to understand the bigger picture of the overall network, and how they fit in.
Describe a possible solution you've considered (optional)
Show some of the following metrics in the explorer

total LPT bonded to all nodes
total LPT "in circulation" (this would need to be defined)
total LPT minted
how many new LPT are minted per round

Furthermore, this could lead to showing things like:

Node's total stake as % of overall network stake
My LPT holding as a % of total LPT

Additional context (optional)
Perhaps this can tie in with Yondon's work reporting protocol level metrics?

This is pretty similar to what's being discussed in #12 . There are a lot of stats mentioned in that issue. So I'm going to close this issue. But for clarity, here is what the process for getting these metrics into the explorer looks like:

Step 1 is to add them into the SDK.
Step 2 is to model in GraphQL (probably in the Protocol type, but there are a lot of requested data points that may deserve separate type definitions).
Step 3 is to put them into the explorer UI

Step 3 is easier said than done. Plopping even more unfamiliar metrics into a dApp already filled with them can have a really bad effect on UX. We'll at least need some tooltip text to help guide users. But more importantly, it probably doesn't make sense to put everything on a single page. Information requires context to be understood, and this means we need to consider which of these numbers belong in existing views and which may require some new page entirely.
That being said, let's keep the conversation going on #12. I'd like to make some choices about which metrics we need, where they live in the explorer, and potentially break out that issue into several smaller ones so the updates can be smaller and released incrementally 👍


chore: Adds g6e instances

Fixes #6789
Description
g6e instances are now GA. This PR regenerates code and documentation to enable scheduling of these new instances.
How was this change tested?
Does this change impact docs?

[x] Yes, PR includes docs updates
[ ] Yes, issue opened: #
[ ] No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>5
Details

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.01%) to 79.29%

Files with Coverage Reduction
New Missed Lines
%

pkg/providers/amifamily/ami.go
1
91.67%

Totals

Change from base Build<PHONE_NUMBER>7:
0.01%

Covered Lines:
5923

Relevant Lines:
7470

💛  - Coveralls


Fix pip install --pre s3fs

It is not possible to run pip install --pre s3fs right now because the two latest pre-releases of aiohttp are broken (v4.0.0a0 and v4.0.0a1).
This PR proposes excluding these two versions from the s3fs requirements.
More details: https://github.com/fsspec/filesystem_spec/pull/1018

Will gcsfs need this too?


CYP2D6 call_stars terminated with an error exit status

Describe the bug
The call_stars process for CYP2D6 exits with error, resulting in a truncated cyp2d6.alleles file.
Here the error log:
$ cat .command.err
Traceback (most recent call last):
  File "bin/stellarpgx.py", line 588, in <module>
    phased_dup = dup_test_cn_n(sv_dup, hap_dbs, snv_cand_alleles[0], snv_cand_alleles[1], snv_def_alleles[0], snv_def_alleles[1], cn, av_cov, in_list)
  File "/data/SG10K-CYP2D6/sandbox/stellarpgx/launchdir/StellarPGx/scripts/cyp2d6/hg38/bin/sv_modules.py", line 254, in dup_test_cn_n
    max_het = max(test_list2)
ValueError: max() arg is an empty sequence

$ cat WHB10006_cyp2d6.alleles
--------------------------------------------

CYP2D6 Star Allele Calling with StellarPGx

--------------------------------------------

Initially computed CN = 8

Sample core variants:
42126611~C>G~1/1;42130692~G>A~1/1

Candidate alleles:
['10.v1_10.v1']

Result:

The trigger seems to be the fact that there are no het variants genotyped for this sample, which makes test_list2 an empty list.
To Reproduce
Here the command used to launch StellarPGx for the conflicting sample:
nextflow run main.nf -profile standard -c /data/SG10K-CYP2D6/sandbox/stellarpgx/WHB10006/nextflow.config --format compressed --build hg38 --gene cyp2d6 -resume

Expected behavior
Our suggestion would be to issue a no call, as opposed to exiting with an error code.

@mgonzalezporta Thanks for the useful suggestion. I will add the 'no call' assignment for such cases.
I see that the computed copy number for WHB10006 sample is 8 and the background alleles are CYP2D6*10/*10. This individual is likely to have one of the more complex *36+*10xN or *36xN+*10xN alleles, especially since they are part of SG10K. We are still working on enhancements for StellarPGx to be able to call these more complex hybrid tandem rearrangements where CN>4. Given the complexity of CYP2D6 allele combinations in SG10K, I would suggest you use Aldy and Cyrius as well for your CYP2D6 allele calling and generate a consensus call set.

Thanks for the quick response David. We are indeed inspecting the consensus across StellarPGx, Cyrius and Aldy. FYI, for the sample here Cyrius outputs a no call, and aldy reports *36 and *10 with multiple copies (*10+*10+*10+*10+*10+*10+*36.ALDY+*10/*36.ALDY+*10). Looking forward to the enhancement.

I have similar problems for CYP2D6 on multiple samples.
I hope it is related to the initial post.
python3 bin/stellarpgx.py b37/diplo_db_debugged2.dbs V350029497_L02_MANsbdpX078095-736-1_vars/V350029497_L02_MANsbdpX078095-736-1_core_snvs.dip V350029497_L02_MANsbdpX078095-736-1_vars/V350029497_L02_MANsbdpX078095-736-1_full.dip V350029497_L02_MANsbdpX078095-736-1_vars/V350029497_L02_MANsbdpX078095-736-1_gt.dip b37/genotypes4.dbs V350029497_L02_MANsbdpX078095-736-1_gene_del/V350029497_L02_MANsbdpX078095-736-1_gene_del_summary.txt V350029497_L02_MANsbdpX078095-736-1_gene_dup/V350029497_L02_MANsbdpX078095-736-1_gene_dup_summary.txt V350029497_L02_MANsbdpX078095-736-1_cyp2d6_dp b37/haps_var_new.dbs b37/a_scores.dbs
--------------------------------------------

CYP2D6 Star Allele Calling with StellarPGx

--------------------------------------------

Initially computed CN = 15

Sample core variants:
42522613~G>C~0/1;42523943~A>G~0/1;42526763~C>T~0/1

Candidate alleles:
['1.v1_35.v1']

Result:
check

Activity score:
Traceback (most recent call last):
  File "bin/stellarpgx.py", line 670, in <module>
    ac_score = get_ac_score(act_score, gene_alleles)
  File "bin/stellarpgx.py", line 645, in get_ac_score
    p_allele = allele_dict[m_allele] + "_" + n_allele
KeyError: 'check'

The crash happens because function dup_test_cn_n runs to the end of the if-chain and returns 'check' which cannot be handled by get_ac_score().
A quick fix was to replace in line stellarogx.py:664
`if gene_alleles == "":
    ac_score = "Indeterminate"
    print(ac_score)

with
if gene_alleles in ["", "check"]:
    ac_score = "Indeterminate"
    print(ac_score)

Is this appropriate, or does gene_alleles == 'check' demand a more informative output?

@mgonzalezporta – Thanks again for opening this issue. I have addressed the error and *36xN+*10xN calling with 3ab19a9

@pmginacio – Thanks for the suggestion at stellarpgx.py:664. We had included the 'check' output to flag any samples that require visual/ in-depth inspection. For instance, the V350029497_L02_MANsbdpX078095-736-1 sample you are analysing seems to have CN=15 which is rare. May you try running it again with the new commit. Thanks.

@mgonzalezporta and @pmginacio – Thanks again for pointing out these bugs. I am closing this issue – feel free to reopen it in case of anything else related to this.


Possible food effect rework

Description
As pointed out by Mirsario on the TML Discord server, stat bonuses as effects are kinda bad and hard to balance to be worthwhile during the whole playthrough. Thus, a rework to the food effects might be implemented.
Message reference on the TML Discord
Food Category effects after rework (WIP)
Meat

Current effect: +2.5% DR per level
Reworked effect:

Max level: 3
Provides a regenerating shield that survive X hits and reduces damage taken by X/Y/Z%
Regenerates after X seconds

Seafood

Current effect: +5% damage per level
Reworked effect:

X% chance to shoot a random sea creature that latches onto the enemy and deals X times X damage
Alternative: Triggers every X attacks

Fruit

Current effect: +1life regen per level
Reworked effect:

Every X seconds spawn a fruit up to X fruits, which after being destroyed heal the player for X HP

Vegetable

Current effect: +10 max HP per level
Reworked effect:

TODO: Some survival based effect

Sweet

Current effect: +5% speed per level
Reworked effect:

Additional jump (TODO: check if you can alter the modded double jumps strength etc. during runtime) + dash on high enough level (?)

Alcohol

Current effect: +2% damage, +3% attack speed, -1 defense per level
Reworked effect:

Some screen effect making everything blurry and slightly wobbly, provides X% dodge chance

Other

Current effect: +1 level of Well Fed per level (max 3)
Reworked effect:

I don't think it needs changing, the effect is not that powerful and it preserves the Well Fed's existence in the mod

Disclaimer
If this is ever to happen, it will be in mod version 2.0.
All notes made here are a subject to change.

Here's my idea for vegetable rework:
Idea 1.

Gain +x% defense effectiveness (or just +x defense, but that's kinda shrimple) for every y% missing hp.

Idea 2.

Gain +x% defense effectiveness for every y max hp.

Idea 3.

Natural regeneration is doubled each x seconds.

Idea 4.

When at full hp, regeneration can still "recover" up to x% of your max hp. (Basically a "shield" when at max hp, ya know?)

Idea 5.

When at full hp, your regeneration builds up, up to x hp. When you take damage, the built up regeneration is released and heals you.

Ideas 1-3:

Not really fittin' the rework theme, we want to move away from stat boosts as much as possible
Idea 4:
Maybe, would require changin' the reworked Meat effect
Idea 5:
RoR2 Medkit is kinda similiar, but it would work better as a Fruit rework


Display installed version of a package in the UI

Is your feature request related to a problem? Please describe.
It is not possible to determine which package is currently installed via the UI.
Describe the solution you'd like

Similar to the list  command the installed version of a package should be displayed on the overview and detail page.
On the detail page also display the latestVersion.
If an update is available display the update path e.g. v0.0.1 -> v0.0.2

Describe alternatives you've considered
Also always display the latest version.

Nice to hear that :-) for these kind of issues it always helps to also include a screenshot in the PR once it is done :-)

Hey @pmig I am getting the UI as the below image as I am using my local development mode i.e
go run main.go serve

Please guide me through this issue

Do I need to run any additional command to install the packages.

@hanshal101 you need to run make web to download the theme into your local environment. We should document this somewhere.


New version: DavidRios.Remindr version 2.0.0

Pull request has been created with WinGet Automation :rocket:
Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:


Remove "Quarter" option for Archive Search

Quarter option too confining
When in Archiving and conducting a search, remove the Quarter option entirely.
Rationale
Kirk determined that it's too time-consuming if a user doesn't know the quarter to have to be selecting various options. It's better to have it be more like keyword.
Notes
Vivian will look more closely at the code to make sure this change does not affect other things.

@gamontoya  - The update has been deployed to staging. Could you take a look?  Thanks

@VivianChu Looks good!


Updating Spockbot?

Is this still under development, and will it ever actually become working on 1.9/1.10/1.11?

I might pick it up again and update it to 1.10 in the next few weeks because I have a use for it again.
In the mean time, you can also look into https://github.com/PrismarineJS/mineflayer/

@Gjum I have seen mineflayer, but it is just not as well thought out as Spockbot and the bots that people have made with it aren't as good as the one that people have made with Spockbot. Also, will you just update to 1.10, or will you go all the way to 1.11?

The difference between 1.10 and 1.11 is minimal in terms of the protocol, so if I get it to 1.10 it'll be easy to also make it work in 1.11. My personal use case is for 1.10 though, so I will focus on that primarily. If possible I will keep 1.11 in mind too of course.

Nope, I went a different path, so don't expect any updating from my side. Sorry!
If you still want to use this with 1.10/1.11/1.12 you'd need to put in the work of updating it yourself, but if during that you have any questions about the inner workings of SpockBot, feel free to contact me.

Submarines are not a conducive environment to coding projects. So I'm out. I'm a huge fan of giving push privileged to literally everyone though

why not use a seperated protocol layer? like pyCraft, i saw it has support to 1.12, but it only focus on protocol level

I just wanted to comment on a possible work around for using Spockbot with newer servers would be to use Bungeecord with viabackwards, viaversion and viarewind.
Probably not the best solution for the problem but it's a good solution if you don't want to use malmo or mineflayer.

@yunfan Our protocol was mostly separate from implementation stuff. There is a certain amount of coupling you can't get around (if a packet that you relied on for say, movement, is completely removed you have to use the new packet that replaced it).
Spock is gone for two reasons that aren't related to implementation. 1) Lack of developer time but mostly 2) Better alternatives are around now.

@nickelpro by alternatives, are you meaning those non-python solutions?

@yunfan Yes. Python has never been a popular choice for MC hacking since Java and now C# are so favored by the community. Projects in those languages as well as mineflayer are way ahead of anything Spockbot did.
But as a said last year, if you make a pull request and the code builds (we need some standards) I'm happy to give anyone push privileges.


configure config.toml to use with the systemd cgroup driver

https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd-systemd

checked with

kubeadm v1.20.6
containerd 1.4.4-1

containerd config
SystemdCgroup = true
default
default
SystemdCgroup = true

kubeadm-kubelet config
cgroupDriver: "cgroupfs"
cgroupDriver: "cgroupfs"
cgroupDriver: "systemd"
cgroupDriver: "systemd"

kubelet status
kubeadm init failed 1)
running with cgroupDriver: cgroupfs  2)
running with cgroupDriver: cgroupfs 2)
running with cgroupDriver: systemd 2)

"The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)"
checked in /var/lib/kubelet/config.yaml on the nodes . Note: With kubeadm 1.21, the cgroup-driver will be set to systemd by default. https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/#configuring-the-kubelet-cgroup-driver.

/no-stale-please

/reopen please

no-stale


optional telemetry

closes #490
Logs creation and deletion according to specs.
Accepts stablished ENV var DO_NOT_TRACK to disable it.
TPI_ANALYTICS_HOST and TPI_ANALYTICS_KEY to modify the Jitsu provider if desired.

Example in jitsu
{
  "_timestamp": "2022-04-25T20:06:22.710592Z",
  "action": "task_delete",
  "api_key": "s2s.....",
  "backend": "az",
  "error": "",
  "eventn_ctx_event_id": "9543f9bf-f9d8-4b6b-b48b-ab916ab4477e",
  "extra": {
    "ci": false,
    "cloud": "az",
    "cloud_disk_size": 30,
    "cloud_machine": "m",
    "cloud_region": "us-west",
    "cloud_spot": 0,
    "cloud_spot_auto": true,
    "task_duration": 56,
    "task_resumed": true,
    "task_status": {
      "running": 0,
      "succeeded": 1
    },
    "terraform_version": "v1.1.9"
  },
  "group_id": "",
  "interface": "cli",
  "os_name": null,
  "os_version": "20.04",
  "source_ip": "<IP_ADDRESS>",
  "src": "api",
  "tool_name": "tpi",
  "tool_source": "terraform",
  "tool_version": "v0.0.0",
  "user_id": "e02b9c45-4afa-40da-83e5-4228c2e1cf79"
}

It's an interesting discussion. To be honest Im not sure about this.
I understand that we might do everything visibly for the user to be able to see that we are not bluffing her. However I can see that we are duplicating efforts writing the same in different languages when we might write a pipeline within Jutsi like the ip geolocation.
Indeed in the case of the IP the user MUST trust that we are good fella and we are not storing the IP for real, so why not for the rest of the data...

Please review updated PR description 👆

action="task_delete": should be task/destroy, task/apply, etc.
OK.

extra.ci=false: would use blank string (for consistency with presumably "github/gitlab/bitbucket")
I will check again the specs. I did not understand that when I read it.

extra.cloud*: could any of this be sensitive?
I think none of them are sensitive. None of them can give us specific details of our anonymised users.

os_name=null: can we fill this in?
Weird! this is a bug definitely but i have seen it previously populated. Might be related to run it inside a container? I will check

source_ip="<IP_ADDRESS>": this should not be passed/saved at all?
Controlled by Jitsu. It's the most worrying aspect for me. Im not including the IP by any meaning, indeed Im calling the endpoint with the ip hidden flag, but its jitsu the one that is handling this. Probably in our real instance (im using my dev this is not happening)

src="api": remove this key
Controlled by Jitsu. Jitsu sends the api key.

tool_source="terraform": what are the alternatives/options?
I was not sure about this field. But we do not yet have other options. It's right now filled to conform the given multi tool specs

tool_version="v0.0.0": unlikely 0.0.0 surely?
Version will be fixed in every release via  -ldflags (please read the note I have discussed with @0x2b3bfa0)

@casperdcl

I was not sure about this field. But we do not yet have other options.

It's fine, to my mind. 💯

action="task_delete": should be task/destroy, task/apply, etc.
OK.

I followed code structure:

resourceTaskDelete
resourceTaskCreate

Sounded to me me better than terraformic

@casperdcl, I wonder if we should use the “standard” create/delete instead of apply/destroy so it stays the same for any task backend we add in the future.

review 🙏

you can drop your opinions here

DROP TABLE opinions; 😄

Is there any other change this PR might need?

Missing task/refresh is intentional, I believe. To avoid overloading our backend with too many records. 🙃

This is ready for review. The only doubts that I have right now is the Id.
As far as I understood the id is read from the file
For CI
anon hash from env vars
For non CI
read appdirs.UserConfigDir("/iterative/telemetry", "",  "", false)
if not exists appdirs.UserConfigDir("dvc/user_id", "iterative", "", false) where iterative is just another level in the path only for Windows from a json user_id
If not then plain 36 hash id
Am I correct? Did I understood the current id extraction?

@iterative/cml I think i'm covering the id generation as specified and the other issues 🤞


Which Verion Of Sentinl Plugin for ELK 7.7?

Which Verion Of Sentinl Plugin for ELK 7.7?

Nothing for now for ELK 7.x

any instead  ?

Several alternative solutions exist, but for me Sentinl is one of the Best

Thanks @lhertzog27 for your kind words. We invested plenty of time and engineering making SENTINL but unfortunately, it never received sponsorships or support outside of QXIP and SIREN. People keep asking about it, but still no offers to sponsor it.


Support BlobFuse

As part of the strategy to support all Azure Data systems, we should make sure that BlobFuse works out-of-the-box for customers who want to mount a file system backed by blobs.

Installation instructions here: https://github.com/Azure/azure-storage-fuse/wiki/Installation
Looks pretty straightforward


changing the role from member to application administrator

@avnerms : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Getting ready for cuts, with evaluate_gradients and new tests. A few bug fixes.

All tests passing. Merging.

All tests passing. Merging.


General Issue: Update PR template

Background Information
Update PR template in line with updates made here.
Definition of Done
PR template should be the same as the version in admiralpeds.

@ukey1025 are you interested in taking this on?


Migrate to number version of einsum

I've been reading the code lately, and I've noticed that you make heavy use of einsum with the string notation, e.g.
einsum('ij,jk->ik', A, B)

Einsum supports also numerical specs instead of letters (same example as above):
einsum(A, [0,1], B, [1,2], [0,2])

Given that you need a lot of index gymnastic, I think it could be beneficial to change the standard and adopt the numerical version of the indices. I used it in my contribution and it allowed my code to be quite compact (I don't even want to think what it would have been like to do the same with letters!)
This could also allow you to be more consistent in the code across the various ops (at the moment it seems that each op was coded by a different person 😅).

Darn it, I just remembered that only the bumpy version of einsum supports it...

Thanks anyway for the suggestion @ziofil!


VS Code DevContainer Config and TestData Download with Maven

This PR now significantly improves the experience with GCToolKit project by not relying on gctoolkit-testdata artifacts hosted on the GitHub Package Registry.
Instead, it will use Maven/Ant to download the tag (currently, 1.0.2) source that is in GitHub Releases for the gctoolkit-testdata project, extract, and construct the gclogs folder structure.
The pom.xml configuration remains the same, meaning contributor profile is not enabled by default, and the download of test data, plus tests, will only happen when contributor profile is enabled.
I've also kept the contributor.sh shell script with an enable/disable feature for contributor profile, and have made the DevContainer configuration (for VS Code on Codespaces), to enable contributor profile by default.

Could be unrelated, but you have a test failure.

I believe that LFS is the issue.
From the uber-zip:
133 Sep 13 09:46 gclogs/preunified/ps/details/tenuring/p_app_02_gc.log.zip
From https://maven.pkg.github.com/microsoft/*/com/microsoft/gctoolkit/gctoolkit-gclogs/1.0.2
55298582 Aug  6 16:59 gclogs/preunified/ps/details/tenuring/p_app_02_gc.log.zip


ValueError reports that object does not exist when the object does actually exist, it is just not assigned to the content_type of the related object

Environment

Nautobot version (Docker tag too if applicable): 2.0.0b2
Python version: 3.10
Database platform, version: N/A
Middleware(s): N/A

Steps to Reproduce

Create a Role, but do not assign any ContentTypes to it
Create a Device with a role assigned to Role from step 1
Attempt to perform validated_save() on Device from step 2

Expected Behavior
Error reporting that the Role assigned to the Device cannot be associated with dcim.device objects

Observed Behavior
Error reporting that the Role does not exist (although it clearly does)

Should just be as easy as adding a try-except block in the Device Model's clean() method to catch this with a custom and helpful message without letting it fall to Django's default error handling.
def clean(self):
        from nautobot.ipam import models as ipam_models  # circular import workaround

        super().clean()
        # Validate Assigned Role
        from nautobot.extras.models import Role
        from django.core.exceptions import ObjectDoesNotExist

        if self.role is not None:
            try:
                Role.objects.get(id=self.role.id, content_types=ContentType.objects.get_for_model(Device))
            except ObjectDoesNotExist:
                raise ValidationError({"role": f"Assigned role {self.role} does not have Device ContentType assigned to it"})

This is only one of many potential cases where an ObjectDoesNotExist could be raised due to content-type restrictions... IMHO it would be better to find a deeper level fix.

I agree with @glennmatthews my example was only meant to provide a means of reproducing the issue, however the issue title was written to be generic to show that it is an issue throughout the codebase


Add Axe/Playwright tests

Context
This is setting up Playwright for accessibility and end-to-end testing, using the pytest-playwright package. The accessibility tests use Axe.
The primary focus was accessibility testing, but this also serves as an MVP for additional end-to-end testing in future.
Setup
Follow the instructions in the README at django_app/tests_playwright/.
How to test

First check that all the tests pass

Then create some accessibility issues on a page. Here's a few if you're struggling for ideas:

Main page heading, instead of a h1

And check the errors are reported

Not quite working well enough to approve, I think. I've left a bunch of comments.

Thanks @wpfl-dbt, these comments are super helpful. I'm working through them now, and will reply to each one individually.


please support fetching other annotations than personal ones

I do research collaboratively, using hypothes.is.
We do annotations in public, and I would love to be able to use Gooseberry for rendering annotations for the whole group.
I guess Gooseberry somewhere in the code hardcodes the username as a constraint for which annotations to fetch, and I would like to optionally be able to specify additional acct: values.

Implemented in PR #109, still needs testing though. You should be able to add a list of usernames to your config as hypothesis_users = [] (no need to add your own, that's always used) and then get all public annotations from those users as well. Would you be up for some testing? I've changed the tagging and deleting functionality to make sure it doesn't even attempt to modify other people's annotations but still not sure for instance how it works for annotations which are private to a group which you belong to. Also, trying to think how to add a test for this since it necessarily needs another hypothesis account.


How can I make sure that the files finished uploading before the form is submitted?

Summary
I'm looking for some sort of validation that would prevent form submission if there's a file still being "processed" by FilePond.
I've looked at the instance options, but haven't found what I'm looking for.
Is there an option or plugin I can use or do I have to write custom code for one of the callbacks/events?

@Cohaven I want to implement the exact same thing! I found this function in the documentation: FilePond:processfileprogress, which I have been using to disable the submit button if the progress is not 1 (100%), however this is only effective for single file uploads. If I have multiple files, the disabled attribute tends to toggle for each file (i.e. 1 file is complete, so submit button enables, but then the processfileprogress function is triggered again by the next file, thus disabling the submit button again, etc.). If anyone has any advice for how to accomplish the same effect for multiple files, that would be greatly appreciated. If you after only allowing for a single file, however, this method works.

@Cohaven @pqina I'm implementing the exact same thing! I found this function in the documentation: onprocessfileprogress (https://pqina.nl/filepond/docs/patterns/api/filepond-instance/), which I have been using to disable the submit button if the progress is not 1 (100%), however this is only effective for single file uploads. If I have multiple files, the disabled state tends to toggle for each file (i.e. 1 file is complete, so submit button enables, but then the processfileprogress function is triggered again by the next file, thus disabling the submit button again, etc.). If anyone has any advice for how to wait for bulk file progress, that would be greatly appreciated. If you are only allowing for a single file, however, this method works.

You can avoid the toggeling by checking it like this:
this.pond.getFiles().filter(x=>x.status !== 5).length !== 0
I do this in onprocessfilestart, onprocessfile, onremovefile and onupdatefiles
and save it to a property: isLoading
/Neno

@neno91 Thanks for your reply. That looks like it could work. But this seems like it should be more straightforward, since I'd have to pass my file progress function into four additional function props just to accomplish this task.

On futher reading i think onprocessfile and onupdatefiles should be enough.
I currently working on a rather complex implementation myself in react.
I will post back if i find a better way to do it.

@neno91 Based on the docs, onprocessfile is only for when the upload has completed, but thanks for updating me on your progress!

I agree it would be nice to have a generic state indicator of what is going on inside FilePond, in projects where I use FilePond I currently check if it's busy by doing something similar to @neno91

Say I want to add the onprocessfilestart and onupdatefiles callbacks. How would I go about doing that? I'm not seeing callback implementation examples in the docs.

Are you using react?
Med vennlig hilsen
Jørgen Skar
Skar utvikling
+47 454 19 500

From: Artemy Matvienko<EMAIL_ADDRESS>Sent: Thursday, March 14, 2019 1:47 PM
To: pqina/filepond
Cc: Jørgen Skar; Mention
Subject: Re: [pqina/filepond] How can I make sure that the files finished uploading before the form is submitted? (#263)
Say I want to add the onprocessfilestart and onupdatefiles callbacks. How would I go about doing that? I'm not seeing callback implementation examples in the docs.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/pqina/filepond/issues/263#issuecomment-472837891, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AB8StZJDYoNQdrw8lBkAiIPS8lz0Qxh_ks5vWkTVgaJpZM4bv_09.

Are you using react? Med vennlig hilsen Jørgen Skar Skar utvikling +47 454 19 500
…
________________________________ From: Artemy Matvienko<EMAIL_ADDRESS>Sent: Thursday, March 14, 2019 1:47 PM To: pqina/filepond Cc: Jørgen Skar; Mention Subject: Re: [pqina/filepond] How can I make sure that the files finished uploading before the form is submitted? (#263) Say I want to add the onprocessfilestart and onupdatefiles callbacks. How would I go about doing that? I'm not seeing callback implementation examples in the docs. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#263 (comment)>, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AB8StZJDYoNQdrw8lBkAiIPS8lz0Qxh_ks5vWkTVgaJpZM4bv_09.

Just JavaScript.

Are you using react? Med vennlig hilsen Jørgen Skar Skar utvikling +47 454 19 500
…
________________________________ From: Artemy Matvienko<EMAIL_ADDRESS>Sent: Thursday, March 14, 2019 1:47 PM To: pqina/filepond Cc: Jørgen Skar; Mention Subject: Re: [pqina/filepond] How can I make sure that the files finished uploading before the form is submitted? (#263) Say I want to add the onprocessfilestart and onupdatefiles callbacks. How would I go about doing that? I'm not seeing callback implementation examples in the docs. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#263 (comment)>, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AB8StZJDYoNQdrw8lBkAiIPS8lz0Qxh_ks5vWkTVgaJpZM4bv_09.

Just JavaScript.

Something like this?
 $('.my-pond').on('FilePond:processfilestart', function(e) {         console.log('file added event', e);     });
Not completly sure this is the correct way but try this.

@Cohaven
const myPond = FilePond.create({
    onprocessfilestart: (file) => {

    },
    onupdatefiles: (files) => {

    }
});

The following seems to be working for disabling the submit button while one or more files are being loaded or processed.
const inputElement = document.querySelector('input[type="file"]');
        const pond = FilePond.create(inputElement, {
            onaddfilestart: (file) => { isLoadingCheck(); },
            onprocessfile: (files) => { isLoadingCheck(); }
        });

        function isLoadingCheck(){
            var isLoading = pond.getFiles().filter(x=>x.status !== 5).length !== 0;
            if(isLoading) {
                $('#createCaseForm [type="submit"]').attr("disabled", "disabled");
            } else {
                $('#createCaseForm [type="submit"]').removeAttr("disabled");
            }
        }

@Cohaven That works nicely, thanks! I am using React, so I solved it like this:
<FilePond           ...           onaddfilestart={(file) => this.onProcessFiles()}           onprocessfile={(error, file) => this.onProcessFiles()}> </FilePond>
Then I bound my onProcessFiles function to my wrapper component and implemented it like so:
  onProcessFiles() {           let { onProcessFiles } = this.props;           let pondFiles = this.pond.getFiles();           if (onProcessFiles) onProcessFiles(pondFiles);   }
Then when I used my FilePond wrapper component, I passed in my function implementation:
<FileUploader                  ....                 onProcessFiles={(pondFiles) => props.onProcessFiles(pondFiles)}  />
I would use this as a form field and pass in my function to handle the state of the submit button:
<SomeParentComponentName               index={props.index}               mode={props.mode}               label={props.label}               value={props.value}               onFilesChanged={props.handleFieldChange}               onProcessFiles={props.handleSubmitState}               isError={props.isError}             />
And finally, my button implementation:
  handleSubmitState(pondFiles) {     const STATUS_COMPLETE = 5;     const isUploadPending = pondFiles.filter(f => f.status !== STATUS_COMPLETE).length > 0;     this.setState({ isSubmitDisabled: isUploadPending });   }
I'd use isSubmitDisabled to set the disabled state of the submit button.

Version 4.3.0 adds onprocessfiles, it's called when all files have been processed.

Version 4.3.1 adds the status property.

Add status property to the FilePond instance, use this property to determine the current FilePond status (EMPTY, IDLE, ERROR, BUSY, or READY).

Will close this issue for now as this should be sufficient to make it easier to control when the user is allowed to submit a parent form.

I'm implementing the exact same thing! I found this function in the documentation: onprocessfileprogress (https://pqina.nl/filepond/docs/patterns/api/filepond-instance/), I have problem when I want to upload multiple files then I am not able to enable or disable submit button as exactly I want.
For example:-
If I have multiple files, the disabled state tends to toggle for each file (i.e. 1 file is complete, so submit button enables, but then the processfileprogress function is triggered again by the next file, thus disabling the submit button again, etc.).
If anyone has any advice for how to wait for bulk file progress, that would be greatly appreciated.

I'm implementing the exact same thing! I found this function in the documentation: onprocessfileprogress (https://pqina.nl/filepond/docs/patterns/api/filepond-instance/), I have problem when I want to upload multiple files then I am not able to enable or disable submit button as exactly I want.
For example:-
If I have multiple files, the disabled state tends to toggle for each file (i.e. 1 file is complete, so submit button enables, but then the processfileprogress function is triggered again by the next file, thus disabling the submit button again, etc.).
If anyone has any advice for how to wait for bulk file progress, that would be greatly appreciated.

@rikschennink Hi, how to make onprocessfiles work only for newly uploaded files? Now it does not work if there are already files.
``
FilePond.registerPlugin(
FilePondPluginImageExifOrientation,
FilePondPluginImagePreview,
FilePondPluginFileValidateSize,
FilePondPluginImageCrop,
FilePondPluginImageResize,
FilePondPluginFileValidateType
);
FilePond.setOptions({
  // allowDrop: false,
  // allowReplace: false,
  status: 4,
  instantUpload: true,
  maxFileSize: '5MB',
  allowImagePreview: true,
  imagePreviewMinHeight: 50,
  imagePreviewMaxHeight: 100,
  imagePreviewMaxFileSize: '5MB',
  imageCropAspectRatio: '1:1',
  // className: 'image-upload',
  server: {
    url: '/images',
    process: {
      url: '/upload',
    },
    revert: '/revert',
    restore: '/restore/',
    load: '/load/',
  },
  files: filepondInitialFiles,
  onremovefile: function(error, file) {
    if (file.serverId) {
      let input = document.createElement('input');
      input.type = 'hidden';
      input.name = 'DeletedFilepondImages';
      input.value = file.serverId;

      uploadForm.appendChild(input);
    }
  },
  onaddfilestart: function(file) {
    console.log(`onaddfilestart`);
    buttonForm.classList.add('filepondUpload');
    buttonForm.setAttribute('disabled', 'true');
  },
  onaddfileprogress(file, progress) {
    console.log(`onaddfileprogress`);
    buttonForm.classList.remove('filepondUpload');
    buttonForm.removeAttribute('disabled');
  },
  onprocessfile: function(file) {
    console.log(`onprocessfile`);
    buttonForm.classList.remove('filepondUpload');
    buttonForm.removeAttribute('disabled');
  },
  onprocessfiles: () => {
    console.log('onprocessfiles');
    buttonForm.classList.remove('filepondUpload');
    buttonForm.removeAttribute('disabled');
  },
});
// console.log(fpFiles);
// get a reference to the input element

const filepondInput = document.querySelector(
  '#filepondFileUploader input[type="file"]'
);

// create a FilePond instance at the input element location
const filepondObject = FilePond.create(filepondInput, {
  maxFiles: 5,
  allowBrowse: true,
  acceptedFileTypes: ['image/*'],
  files: filepondInitialFiles,
  labelIdle:
    '
Add images
',
});

``

You're setting options multiple times files: filepondInitialFiles, in both the setOptions as the creation method. Please create a clean test case.

Updated. The result is the same
``
console.log(filepondInitialFiles);
FilePond.setOptions({
instantUpload: true,
allowImagePreview: false,
server: {
url: '/images',
process: {
url: '/upload',
},
revert: '/revert',
restore: '/restore/',
load: '/load/',
},
onremovefile: function(error, file) {
if (file.serverId) {
let input = document.createElement('input');
input.type = 'hidden';
input.name = 'DeletedFilepondImages';
input.value = file.serverId;
      uploadForm.appendChild(input);
    }
  },
  onaddfilestart: function(file) {
    console.log(`onaddfilestart`);
    buttonForm.classList.add('filepondUpload');
    buttonForm.setAttribute('disabled', 'true');
  },
  onaddfileprogress(file, progress) {
    console.log(`onaddfileprogress`);
    buttonForm.classList.remove('filepondUpload');
    buttonForm.removeAttribute('disabled');
  },
});
// get a reference to the input element

const filepondInput = document.querySelector(
  '#filepondFileUploader input[type="file"]'
);

// create a FilePond instance at the input element location
const filepondObject = FilePond.create(filepondInput, {
  maxFiles: 5,
  acceptedFileTypes: ['image/*'],
  labelIdle:
    '
Add images
',
  files: filepondInitialFiles,
  onprocessfiles() {
    console.log('onprocessfiles');
    buttonForm.classList.remove('filepondUpload');
    buttonForm.removeAttribute('disabled');
  },
});

``

I have the same issue. Is there a solution?

This works for me.
	$('#fform').on('submit',function(e) {
		if (pond.status != 4) {
			return false;
		}
		$(this).find(':input[type=submit]').hide();
		return true;
	});

Version using jquery
        $('#mapCreateSubmitBtn').on('click',function(e) {
            e.preventDefault();

            if ($('.upload-filepond{{$class??''}}').filepond('status') == 4) {
                console.log('submit!');
                $('#mapCreateForm').submit();
            }else{
                const Toast = Swal.mixin({
                    toast: true,
                    position: 'top-end',
                    showConfirmButton: false,
                    timer: 3000,
                })

                Toast.fire({
                icon: 'warning',
                title: 'Upload in progress, please wait before trying again!'
                })

            }
        });

if ( imageFilePond.status === 3 )
{
//uploading...
}

Updated. The result is the same `
console.log(filepondInitialFiles);

FilePond.setOptions({
  instantUpload: true,
  allowImagePreview: false,
  server: {
    url: '/images',
    process: {
      url: '/upload',
    },
    revert: '/revert',
    restore: '/restore/',
    load: '/load/',
  },
  onremovefile: function(error, file) {
    if (file.serverId) {
      let input = document.createElement('input');
      input.type = 'hidden';
      input.name = 'DeletedFilepondImages';
      input.value = file.serverId;
      uploadForm.appendChild(input);
    }
  },
  onaddfilestart: function(file) {
    console.log(`onaddfilestart`);
    buttonForm.classList.add('filepondUpload');
    buttonForm.setAttribute('disabled', 'true');
  },
  onaddfileprogress(file, progress) {
    console.log(`onaddfileprogress`);
    buttonForm.classList.remove('filepondUpload');
    buttonForm.removeAttribute('disabled');
  },
});

// get a reference to the input element
const filepondInput = document.querySelector(
  '#filepondFileUploader input[type="file"]'
);

// create a FilePond instance at the input element location
const filepondObject = FilePond.create(filepondInput, {
  maxFiles: 5,
  acceptedFileTypes: ['image/*'],
  labelIdle:
    '
Add images
',
  files: filepondInitialFiles,
  onprocessfiles() {
    console.log('onprocessfiles');
    buttonForm.classList.remove('filepondUpload');
    buttonForm.removeAttribute('disabled');
  },
});

`

did you manage to resolve it? i'm with the same error, i've tried some things but not working at all

I just test the events (@processfiles and @addfilestart) on vue3 and they do the job effectively:

<template>
  <file-pond
  ...
 @processfiles="() => loadingFiles=false"
 @addfilestart="() => loadingFiles=true"
 ...
 />
 </template>

We are having the same problem with onprocessfiles not firing (we are using version 4.31):
onprocessfiles: function () {
    document.querySelectorAll('#controls button').forEach((el) => {
        el.disabled = false;
    });
    console.log("finished");
}

Sorry - Apologies. I was adding the even in the process as opposed to the filePond object itself.
Works PERFECT.


Create verification system for admins to verify other users

From BEE Notes (Google Drive):
"Verification: there could be an option for admins to update a mutual aid or volunteer user from unverified (default state) to verified after you review whatever information you need. Unverified accounts would limit users from seeing sensitive information (item gallery, etc). The donor verification could be automated just by checking the format of the information they entered."

Acceptance criteria:

Admins can set a user (e.g. mutual aid user, volunteer user) as verified.

A verify claim is introduced in pull request 22.
In pull request 43, users can be assigned to any supported claim via a dashboard that is visible to users with an admin claim.
In pull request 43, user registration is modified to conform to stakeholder requirements.
Additional work is needed to identify what needs to be verified and requirements of the verification system.


Need help with Rfuzz inputs

Hi,
I want to compare the output of rocket-chip when fuzzed with Rfuzz with SPIKE, the ISA simulator. For this, I will have to run the spike with the same inputs as the ones used on rocket-chip. In the repository where can I find the elf or binary files used as input to drive the rocket-chip? Are they output in a format that SPIKE can accept directly or does the input generated by Rfuzz have to be modified to use with SPIKE?
Any help regarding this task would be really helpful.
Thanking you in advance.

If you are interested in fuzzing RISC-V processors in particular and to compare their execution to a software model, I would advise you to look into Difuzz. It is really cool work that makes the fuzzing idea much more practical for the purpose of processor testing. The paper shows how they found a bunch of bugs in open source RISC-V processors and their implementation might help you with the fuzzing setup you are asking about.
I hope this helps!


Redesign Calendar Page to Match Design

Please provide a summary of the feature
Currently, the calendar only has the calendar and none of the design elements associated with the design docs. This task is build the front end of the calendar page.

Are there any potential issues you foresee with this feature
Is this feature related to any current or past issues? Please provide the
Additional context
Add any other context or screenshots about the feature request here.
Which Branch should this fix be Pulled into?
PR into feature/CalendarPageRedesign
If you're interested in working on this feature, please comment so you can be assigned

I'd like to get in on this B)

I'd like to get in on this B)

All yours. This is a decently big task. What type of ETA would you like to set?

I'd like to get in on this B)

All yours. This is a decently big task. What type of ETA would you like to set?

How does 12/21-12/23 sound? I am attending a wedding this weekend. Do you need it done sooner?

I'd like to get in on this B)

All yours. This is a decently big task. What type of ETA would you like to set?

How does 12/21-12/23 sound? I am attending a wedding this weekend. Do you need it done sooner?

Nope that works great. We look forward to seeing it on 12/23!

@luciousmc feature/CalendarPageRedesign is ready for PR. I have also linked @HondaChan14 and @Natashka333 to help assist with the needed alterations.

Hi guys! I appreciate your ideas about the calendar page. I will try to send the mock-ups by the end of the day 😊

Hi guys! I appreciate your ideas about the calendar page. I will try to send the mock-ups by the end of the day 😊

Thanks Natalie. Can you throw in a Feedback and Join the Team buttons into your next design?

Hi guys! I appreciate your ideas about the calendar page. I will try to send the mock-ups by the end of the day 😊

Thanks Natalie. Can you throw in a Feedback and Join the Team buttons into your next design?

Sure! Will do it.

Hi, team.
I did the sidebar:

It can be just icons. When you click on an icon, the wide sidebar pops up.

Or we can see the icons with their functions' names on the sidebar.

Also, I added some pictures on the left bottom corners. So, 'the people' can be different every month.
Next, I changed the grey gaps on the calendar (first left option)  into white. I think I like white more. For me, it looks more balanced.
Let me know what you think about redesigning, including functionality and colors.

@Natashka333 Looks great!
@luciousmc and @HondaChan14 here are my thoughts, but I'd love your input. Add the togglable nav bar to the left. Keep the original design that @luciousmc did, and setup a media query to only the togglable nav bar when it gets too small.
If you feel that there is even the largest screens cannot fit the original design then I would just include the nav bar, but I feel like we're missing some more blue as a contrast.
What are yall's thoughts?

Nice job @Natashka333 !
A few things I was thinking about...

Will the sidebar expand on hover like from Honda's project, or on click? Or another option is to have another icon that handles opening and closing it.
When the sidebar is expanded, I'm assuming the whole peachy area will expand and the calendar will shrink? Or would it just expand over the calendar?
Maybe the 'Add to Calendar' icon could be a plus sign or have one on it to be more obvious what it is while the sidebar is collapsed? Just a  thought, I know you can expand it to see the text.
Another thought, if the sidebar is toggleable we could have tooltips popup on hover showing the full text. This can also be an option for the events that get truncated.

What do ya'll think? @Natashka333 @HondaChan14 @Caleb-Cohen

#1 I would say maybe a button for accessibility so people can know where the nav menu is at all times.
#2 hmmm would could have it maybe overlap the screen so that it doesn't disrupt the calendar to shrink when opening and collapsing
#3 Yea thats sounds good to me to add the Plus Sign
#4 yea i like that idea

@HondaChan14, @Caleb-Cohen and @luciousmc , thanks for your feedbacks. I took your points and tried to play around in Figma.

I change the background color to light blue.
I put an expanded sidebar to check what it will look like. I also brought two ways of sections ( all icons in one section, or each icon is a separate section).
Another mockup includes a 'pancake' icon. Click on it and the expanded sidebar comes over the calendar.
(And if I understand your @luciousmc  forths point right), I tried to create the look of the icons when you guide the mouse to/click on one of them, and the expanded sidebar of that icon appears.

Let me know, what you think.

This one is my favorite thus far.  Maybe the popout starts a bit too far on the right?
The transparency is a nice touch.

I like the expanded sidebar with the full calendar next to it, but I'm also ok with it just expanding on top of it. I prefer the block of items over the separated one.

I'm thinking the toggle icon could be a small arrow or something similar that changes direction based on the collapsed state of the sidebar.
@Natashka333 what do you think about the tooltips just being a dark background with some white text that appear above the icon you're hovering?

Other than that I'll start working on this tonight. Honda has been under the weather and said I can start.

I like the expanded sidebar with the full calendar next to it, but I'm also ok with it just expanding on top of it. I prefer the block of items over the separated one.

Can you include pictures of which ones you're referencing so I can better follow them? I'm open to this! We do not need to go with just what I like.

Merry Christmas, team!
Here I am attaching new mockups:
This one with the arrow to expand the bar and the arrow to reduce the size  (it will be only one button, I am just showing how it will look in both conditions). (as @luciousmc idea the arrow changes its direction based )

And here is the other great point (thanks @luciousmc ). I added tooltips that appear above the icon.

Feedbacks are welcome!
Happy Holidays!


[WIP] Optimize quota usage calculation

For the moment I just want to execute tests, you busybody. :roll_eyes:

For the moment I just want to execute tests, you busybody. 🙄

xD. You cannot escape my scrutiny 🕵

Closing this one in favor of https://github.com/CartoDB/cartodb/pull/11794


cf dev start gets stuck at 48 services out of 50

So I'm working with a machine with 8 GB of RAM (which I think should be enough to run pcf dev) and I'm getting stucked at "48 out of 50 running" at the "Waiting for services to start..." stage. Is it supposed for the services to take longer than an hour to start?
I am, in theory, not behind any kind of proxy and I'm using the 5.0.20 version of Virtualbox and the lastest version of the CF CLI.
Could you give me a helping hand?
Thanks,
Marc
EDIT: If I take a look at the VM, it's showing a login screen. Also, sorry if I make any kind of gramatical mistakes, since english is not my first language.
EDIT 2: I'm using Windows 10 64bits

Hey @MarcSolde
It definitely is not supposed to take a hour to start, or even more than 20 minutes.
We have reason to believe that you might have ran into a race condition with the VM starting. Please cancel the operation if not already, please run cf dev destroy and then cf dev start and tell us if this issue resurfaces. If it does, we can use more advanced troubleshooting methods.

Hi @MarcSolde,
Can you provide the full output of running cf dev start as well?

@aemengo I already did that a couple of times before, but I tried once again and it worked. Since I've changed the network, maybe there was something wrong with the original one, I don't know.
@sclevine Now it works, so I can't :smile: . Anyway, thanks you both for the help!


[FEATURE] usinghelp

hello,
Excuse me
I'm a beginner in deep learning.Want to use your code, but don't know how to call it.I was wondering where the help documentation for using this code.
Thank you for your help](javascript.
Thank you very much.

@cvliucv there's documentation linked from the homepage but it's pretty minimal right now https://huggingface.co/docs/timm/index
This is extensive, https://towardsdatascience.com/getting-started-with-pytorch-image-models-timm-a-practitioners-guide-4e77b4bf9055 ... but if you're starting from scratch maybe fastai or some intro courses to familiarize yourself with the concepts? lots of stuff online
Regards,
Ross


[ZEPPELIN-2410]: Using UUID from filename in registry cache dir inste…

…ad of uri.
What is this PR for?
Fixing issue ZEPPELIN-2410:
The previous version of this file used the full uri as file name in the
cache. This included special characters like the : in http: which
caused problems.
Since this is just a cache file, a better approach is to use an UUID.
The UUID is constructed from the URI to avoid the cache dir to grow
over time (in case there is no clean up of the cache dir).
Output of ls -la on macOS before the fix:
drwxr-xr-x  4 fries  staff  136 17 Apr 22:28 .
drwxr-xr-x  8 fries  staff  272 21 Apr 12:43 ..
drwxr-xr-x  3 fries  staff  102 17 Apr 10:39 https:

Output of ls -la on macOS after the fix:
drwxr-xr-x  4 fries  staff    136 21 Apr 17:32 .
drwxr-xr-x  8 fries  staff    272 21 Apr 17:32 ..
-rw-r--r--  1 fries  staff  89666 21 Apr 17:32 a6618b3e-540a-340e-b624-07bf7f2b5e7d

Note: Windows 7 just fails to create the cash before the fix.
What type of PR is it?
[Bug Fix]
What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2410?filter=-2
How should this be tested?
Non-Regression Test.
Run on Windows, check cache dir.
Screenshots (if appropriate)
N/A
Questions:

Does the licenses files need update?
No
Is there breaking changes for older versions?
No
Does this needs documentation?
No

LGTM

Merge to master and branch-0.7 if no further comments.


sentry

FYI, sentry adopted a new term "fair source" and "rebranded" as such: https://blog.sentry.io/sentry-is-now-fair-source/
does this mean they should be removed from the list? i think what might be more useful to track the time windows in the readme, so we keep the historical perspective?

Thanks @Dieterbe,
Yeah, having been providing input to Sentry along their FSL and fair journey I've been watching their change to this, but their active qualification here has mainly been due to retaining references that could mislead, the main one being this page but it looks like that's now been updated.
That said, I'm not sure if users could still be mislead. The "Open Source" link in their footer leads to that page, with no actual reference to "Fair Source" so the "We're Fairly Open" may only make sense (as referring to not being open) for people already familiar with fair source.
If a user reads this page would they come away with the impression Sentry is open source or not?
Overall, Still at least sits in a grey area for me.


Various pandas deprecation warnings from test suite

We've started getting several types of deprecation warnings from the tests:
pvlib/tests/test_modelchain.py::test_aoi_model_interp
  /home/runner/work/pvlib-python/pvlib-python/pvlib/tests/test_modelchain.py:1476: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
    assert mc.results.ac[1] < 1

pvlib/tests/test_clearsky.py::test_detect_clearsky_window
  /home/runner/work/pvlib-python/pvlib-python/pvlib/tests/test_clearsky.py:634: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value 'True' has dtype incompatible with int64, please explicitly cast to a compatible dtype first.
    expected.iloc[-3:] = True

pvlib/tests/test_singlediode.py: 14 warnings
  /home/runner/work/pvlib-python/pvlib-python/pvlib/tests/test_singlediode.py:118: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.
    joined[is_number] = joined[is_number].applymap(np.float64)

  /home/runner/work/pvlib-python/pvlib-python/pvlib/scaling.py:279: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
    df = df.fillna(method='bfill').fillna(method='ffill')

Here's an example run: https://github.com/pvlib/pvlib-python/actions/runs/6550206298/job/17788805508?pr=1827#step:9:65
Addressing these would be good first PRs for anyone wanting to get started with contributing to pvlib's codebase!  If you plan to submit a PR for one of these warnings, please comment here so that someone else does not duplicate the work.

Is the intention to resolve these warnings by updating the deprecated methods with current ones, or rather to just hide the deprecation warnings?

@matsuobasho preferably update the methods to resolve the warnings. If updating breaks other code, perhaps hide them.

Any recommendations on putting a breakpoint in the tests where the warnings occur in order to help with diagnosing the issue and testing for a resolution?  I see one way mentioned in this SO post, but since I'm in an environment pdb doesn't reocognize pytest when I run python -mpdb $(which pytest)

Ok, I'm tackling this issue.  Since is this my first issue, would like to ask about a nuance that I've encountered.
In the test_bird function of test_clearsky.py, we get a warning on lines 857-859:
irrads3 = clearsky.bird(
        zenith[12], airmass[12], aod_380nm, aod_500nm, h2o_cm, dni_extra=etr[12]
    )

Here's the warning:
FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`

Now the interesting part is that that warning only pertains to etr[12].  The reason why is that out of the 3 objects here that are being referenced by an index, only etr is a Pandas series.  zenith is a Pandas index type, while airmass is a numpy array.  Neither of those have an iloc method.
So changing to etr.iloc[12] gets rid of the warning.  However, there is now an inconsistency in how we would be referencing these objects.  I think it's fine but looking for confirmation especially as I'll potentially run into the same scenario in other warnings that are part of this issue.  Thanks.

Wow, three different vector types going into a single function :P Don't worry about the inconsistency with not using iloc for all inputs in that case; changing to just etr.iloc[12] is fine.  Maybe that test could be cleaned up a bit more generally, but that would be a separate PR.  Thanks @matsuobasho!

Ah yeah, the SRML website has been down for a while and causing the associated tests to fail.  If you check the recent CI test results for main you'll see the same failures there.
I suggest ignoring those failing tests for now.  If any deprecation warnings come out of those tests whenever they eventually start passing again, we can just address them at that time.

@matsuobasho we add the labels to the PR.


Follow up: window.intercomSettings

It seems like window.intercomSettings isn't used in SPAs, according to https://github.com/devrnt/react-use-intercom/pull/481#issuecomment-1148940938. This is confirmed by someone from Intercom. We should follow this up and check for updated documentation, so we can remove the use window.intercomSettings  uses in this codebase

Also note that in both boot and update the "old" window.intercomSettings are spread into the new window.intercomSettings. So by removing the use of these global settings, we should introduce our own global settings reference


var declaration shorthand

There's been requests for a shorthand var to go alongside let and const.
Here are some ideas:
..=
,=
;=
Any others we should consider?

var is closer to let than const so I'd suggest nixing ;=.
,= seems superior to ..= because it's shorter...
I'm not sure that any of them communicate "function scope". Ah, maybe ..= does because it could mean "some levels up".  Maybe the comma in ,= is pointing to the function — though that should probably be up not down. In some ways ^= or `= or '= would make sense, pointing to where the declaration hoists, but these already have meaning.  \= seems too weird.
I think I'm currently leaning ,=.  Anyone else have thoughts?

My preference is ..= since it is the same as .= for let but with expanded scope. And typing two of the same character twice isn't much slower than typing it once.
I'd prefer not to use ;= since it has the upper dot which I like to think of as a "read only" bit. We could also theoretically add a .:= for a var scoped const (only enforced by compiler warning, still transpiles to var)

@edemaine @STRd6 Why not just =? Whereby if not previously declared, it defaults to var, like CoffeeScript…

@danielbayley We already have that as an option, but you have to enable it, via "civet autoVar" or "civet coffeeCompat".

@danielbayley We already have that as an option, but you have to enable it, via "civet autoVar" or "civet coffeeCompat". See the documentation.

A yeah I had seen that thanks, just wondered why it wasn’t the default behaviour… Then again I thought maybe let should be the default… which I didn’t realise was also an existing option.
Speaking of prologue directives, I would much prefer to put these settings in the config instead… Is that in the pipeline? @edemaine

@danielbayley Better to take that discussion elsewhere (new discussion or Discord), but there is already limited (but undocumented) config support.


updating for Try .NET -- final set of first round

Docs PR is https://github.com/dotnet/dotnet-api-docs/pull/3894

#hold-off

@mairaw, @BillWagner -- Maira or Bill, please review these changes -- this should be the last set of samples for this first round of Try .NET.
I'll have the doc PR ready tomorrow.

I'm going to close this PR for now. We should try to use the new syntax, which wouldn't require sample changes.


Chatterino 2 crashes when adding and removing a Notification channel on OSX

recorded it here:
https://www.youtube.com/watch?v=_2Y8pTu7GZ0

Does this issue still occur?

Does this bug still occur?

I can confirm this bug still occurs and also happens when removing highlight phrases

Cannot reproduce on 6d5ba0c442681fb30afd01f787a10cd3922c10aa and Catalina 10.15.4


Fix: Review and address Use parentheses within macros around parameter names

PR Description:
This pull request addresses two macro-related issues found in velox/tpch/gen/dbgen/build.cpp and velox/common/base/SpillStats.cpp to improve code correctness and maintainability:
1.  Add Parentheses Around Macro Parameters (velox/tpch/gen/dbgen/build.cpp)
Issue: The TEXT macro uses parameters like avg without parentheses, which can lead to incorrect results if the parameter is an expression. For example:
#define TEXT(avg, seed, tgt) \
  dbg_text(tgt, (int)(avg * V_STR_LOW), (int)(avg * V_STR_HGH), seed)

If avg is replaced with an expression (e.g., a + b), the result would be incorrect due to operator precedence:
TEXT(a + b, seed, tgt) // Expands to dbg_text(tgt, (int)(a + b * V_STR_LOW), ...)
Fix: Wrap macro parameters in parentheses to ensure correct precedence and behavior:
#define TEXT(avg, seed, tgt) \
  dbg_text(tgt, static_cast<int>((avg) * V_STR_LOW), static_cast<int>((avg) * V_STR_HGH), seed)

We are also replacing (int) C style casts with a static_cast as it ensures safer, clearer, and more maintainable type conversions by enforcing explicit and compile-time-checked behavior.
2. Remove Trailing Semicolon in Macro Definition (velox/common/base/SpillStats.cpp)
Issue: The UPDATE_COUNTER macro ends with a semicolon after the while (0) block:
#define UPDATE_COUNTER(counter)           \
  do {                                    \
    if (counter < other.counter) {        \
      ++ltCount;                          \
    } else if (counter > other.counter) { \
      ++gtCount;                          \
    }                                     \
  } while (0);

This results in double semicolons (;;) when the macro is used:
UPDATE_COUNTER(someCounter);  // Expands to: do { ... } while (0);;
Fix: Remove the trailing semicolon in the macro definition:
#define UPDATE_COUNTER(counter)           \
  do {                                    \
    if ((counter) < (other.counter)) {    \
      ++ltCount;                          \
    } else if ((counter) > (other.counter)) { \
      ++gtCount;                          \
    }                                     \
  } while (0)

These fixes enhance code safety and eliminate potential pitfalls in macro usage.

@anandamideShakyan Please provide a description (see guidelines in contributing).
For example, https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#es41-if-in-doubt-about-operator-precedence-parenthesize can e used as rationale when expanding macros.
In the description also mention cleanup of semicolons.
Also please squash commits and provide a better abstract than copying the name from the internal issue.

@jkhaliqi done.


DINO traning error when using internimage

The following error occurs.
Please check.
/usr/local/lib/python3.8/dist-packages/torch/autograd/init.py:173: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed.  This is not an error, but may impair performance.
grad.sizes() = [1280, 1, 3, 3], strides() = [9, 1, 3, 1]
bucket_view.sizes() = [1280, 1, 3, 3], strides() = [9, 9, 3, 1] (Triggered internally at  ../torch/csrc/distributed/c10d/reducer.cpp:326.)
Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass

The following error occurs. Please check.
/usr/local/lib/python3.8/dist-packages/torch/autograd/init.py:173: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance. grad.sizes() = [1280, 1, 3, 3], strides() = [9, 1, 3, 1] bucket_view.sizes() = [1280, 1, 3, 3], strides() = [9, 9, 3, 1] (Triggered internally at ../torch/csrc/distributed/c10d/reducer.cpp:326.) Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass

OK, we will check the bug recently~

The following error occurs. Please check.
/usr/local/lib/python3.8/dist-packages/torch/autograd/init.py:173: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance. grad.sizes() = [1280, 1, 3, 3], strides() = [9, 1, 3, 1] bucket_view.sizes() = [1280, 1, 3, 3], strides() = [9, 9, 3, 1] (Triggered internally at ../torch/csrc/distributed/c10d/reducer.cpp:326.) Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass

OK, we will check the bug recently~

Thank you sir!!


use defineProps "as PropType" volar-plugin tip error: ts(2693)

Name: Volar
Id: johnsoncodehk.volar
Description: Language support for Vue 3
Version: 0.25.12
Publisher: Johnson Chu
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=johnsoncodehk.volar
vue3 defineProps use T as PropType  like:
`

valor tip: ts(2693)...PropType...
“PropType”Represents only the type, but is used here as a value。ts(2693)`
My ability is limited and I don't know how to solve it. Turn off the valor plug-in and use vetur again. Here it doesn't report this error.

packge,json
"dependencies": {
"element-plus": "^1.0.2-beta.44",
"vue": "^3.0.5",
"vue-i18n": "^9.1.6",
"vue-router": "^4.0.8"
},
"devDependencies": {
"@types/node": "^15.6.0",
"@vitejs/plugin-vue": "^1.2.2",
"@vue/compiler-sfc": "^3.0.5",
"prettier": "^2.3.0",
"sass": "1.32.13",
"typescript": "^4.1.3",
"vite": "^2.3.3",
"vite-plugin-style-import": "^0.10.0",
"vue-tsc": "^0.0.24"
}
tsconfig
{
"compilerOptions": {
"target": "ESNext",
"module": "esnext",
"moduleResolution": "node",
"strict": true,
"jsx": "preserve",
"sourceMap": true,
"resolveJsonModule": true,
"esModuleInterop": true,
"lib": ["esnext", "dom"],
"types": ["vite/client"]
},
"include": ["src//*.ts", "src//.d.ts", "src/**/.tsx", "src/**/*.vue"]
}

You should use import type { PropType }, otherwise you will have a runtime error.
Please try to search the error message first.

https://github.com/johnsoncodehk/volar/issues/165
https://github.com/johnsoncodehk/volar/issues/132


false ffi_native_unexpected_number_of_parameters error on getters

In flutter we are getting the following analyzer error on a bunch of getters annotated with @FfiNative:
  error • text.dart:2714:3 • Unexpected number of FfiNative annotation parameters. Expected 1 but has 0. Make sure parameters match the function annotated. •
          ffi_native_unexpected_number_of_parameters

The code appears to be working correctly, so I think this is a bug in the analyzer.
Examples of where we are getting this error:
@pragma('vm:entry-point')
class Paragraph extends NativeFieldWrapperClass1 {
  // ...

  // see https://github.com/flutter/engine/blob/862773f0d9093b97271237b31e379dde890b0b69/lib/ui/text.dart#L2709-L2712
  @FfiNative<Double Function(Pointer<Void>)>('Paragraph::ideographicBaseline', isLeaf: true)
  external double get ideographicBaseline;

  // ...

  // see https://github.com/flutter/engine/blob/862773f0d9093b97271237b31e379dde890b0b69/lib/ui/text.dart#L2714-L2722
  @FfiNative<Bool Function(Pointer<Void>)>('Paragraph::didExceedMaxLines', isLeaf: true)
  external bool get didExceedMaxLines;

  // ...
}

@dcharkes

I tried to fix this in https://dart-review.googlesource.com/c/sdk/+/252941.

Thanks @osa1 !

Thanks for the quick fix! 🚀


what about the CSS-i18n in Valaxy to be an independent project

I think it's interesting . I want to try myself to make CSS-i18n better though I'm not a good FE development and not good at CSS.
你能帮帮我吗，求求你了，求求你了

I simply extracted a css-i18n package, let's move the discussion here.

https://github.com/valaxyjs/css-i18n


load css properly when scanning

The scanner is not linking CSS properly, so the files being scanned are not rendered with styles.

This is false, I made a mistake. We're spidering, not downloading the files, so the pages are being rendered properly.


Cannot insert variable into global scope from within let.

Attempting to use define to insert a variable into the global scope from within a let statement throws an error. A minimal example to demonstrate
extern crate ketos;

use ketos::{Interpreter, FromValue};

fn main() {
    let interp = Interpreter::new();

    match interp.run_code(r#"
        (let ((plus +) (mult *)) (
            (define ten (mult (plus 3 2) 2))
        ))
        "#, None) {
        Ok(k) => k,
        Err(e) => panic!(interp.display_error(&e)),
    };

    let m = i32::from_value(interp.get_value("ten").unwrap()).unwrap();

    println!("{}", m);
}

will throw
execution error: type error: expected function; found name: ten

In contrast, if we change the Ketos code executed to
...
match interp.run_code(r#"
    (define ten (* (+ 3 2) 2))
"#, None) {
...

this code executes without error, printing 10.

This one had me confused for a bit. I didn't see the issue right away.
The problem is that you're placing extra parentheses around the body of let. This becomes clearer if you use the traditional indentation style.
(let ((plus +) (mult *))
     ((define ten (mult (plus 3 2) 2))))

Double parentheses around (define ...) means that the define operator is processed, then an attempt is made to execute the result (which is the name of the definition). Remove the extra parentheses and it executes just fine:
ketos=> (let ((plus +) (mult *))
ketos(>      (define ten (mult (plus 3 2) 2)))
ten
ketos=> ten
10

So let can only take two arguments? I apologize for misinterpreting the syntax! Is there any way around this behavior? I'd like to feed in something akin to one of Rust's block expressions.

The do operator is what you're looking for. It evaluates a series of expressions and yields the value of the final expression.

Thanks!


Unable to Start td-agent

Environment

OS: RHEL7
td-agent-3.2.0-0.el7.x86_64
Successfully installed via yum repo specified in install-redhat-td-agent2.sh

Configuration
<source>
  @type syslog
  port 5140
  bind <IP_ADDRESS>
  protocol_type tcp
  tag system
</source>

<match system.**>
 @type stdout
</match>

Description
I am unable to start td-agent 9 times out of 10. On the occasions I am able to start it, it is unclear exactly what the cause and fix were as it seems inconsistent.
<EMAIL_ADDRESS>~]# systemctl start td-agent.service
Job for td-agent.service failed because the control process exited with error code. See "systemctl status td-agent.service" and "journalctl -xe" for details.

<EMAIL_ADDRESS>~]# systemctl status td-agent.service
● td-agent.service - td-agent: Fluentd based data collector for Treasure Data
   Loaded: loaded (/usr/lib/systemd/system/td-agent.service; disabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Fri 2018-06-22 19:42:18 BST; 40s ago
     Docs: https://docs.treasuredata.com/articles/td-agent
  Process: 11748 ExecStart=/opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=1/FAILURE)

Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: td-agent.service: control process exited, code=exited status=1
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: Unit td-agent.service entered failed state.
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: td-agent.service failed.
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: td-agent.service holdoff time over, scheduling restart.
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: start request repeated too quickly for td-agent.service
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: Unit td-agent.service entered failed state.
Jun 22 19:42:18 ip-[..REDACTED..] systemd[1]: td-agent.service failed.

Executing the command myself results in an error surrounding config file existence:
<EMAIL_ADDRESS>~]# /opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS
/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/supervisor.rb:760:in `initialize': No such file or directory @ rb_sysopen - /etc/fluent/fluent.conf (Errno::ENOENT)
	from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/supervisor.rb:760:in `open'
	from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/supervisor.rb:760:in `read_config'
	from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/supervisor.rb:477:in `run_supervisor'
	from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/command/fluentd.rb:310:in `<top (required)>'
	from /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/bin/fluentd:8:in `<top (required)>'
	from /opt/td-agent/embedded/bin/fluentd:23:in `load'
	from /opt/td-agent/embedded/bin/fluentd:23:in `<main>'

It's defaulting to /etc/fluentd/fluentd.conf which doesn't exist as I am using /etc/td-agent/td-agent.conf. So I try to use it and hit another issue:
<EMAIL_ADDRESS>~]# /opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid -c /etc/td-agent/td-agent.conf
Unexpected error No such file or directory @ rb_sysopen - /var/run/td-agent/td-agent.pid
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.0.6/lib/serverengine/daemon.rb:200:in `initialize'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.0.6/lib/serverengine/daemon.rb:200:in `open'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.0.6/lib/serverengine/daemon.rb:200:in `write_pid_file'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.0.6/lib/serverengine/daemon.rb:193:in `daemonize_with_double_fork'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.0.6/lib/serverengine/daemon.rb:107:in `main'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/serverengine-2.0.6/lib/serverengine/daemon.rb:68:in `run'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/supervisor.rb:632:in `supervise'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/supervisor.rb:502:in `run_supervisor'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/command/fluentd.rb:310:in `<top (required)>'
  /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
  /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
  /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/bin/fluentd:8:in `<top (required)>'
  /opt/td-agent/embedded/bin/fluentd:23:in `load'
  /opt/td-agent/embedded/bin/fluentd:23:in `<main>'

I can successfully run td-agent like this but undaemonised:
LD_PRELOAD=/opt/td-agent/embedded/lib/libjemalloc.so /usr/sbin/td-agent -c /etc/td-agent/td-agent.conf --user td-agent --group td-agent

At this point I'm thinking 'OK, I must have something inherently wrong here'.  Figured here would be the best place to get some assistance.

Unexpected error No such file or directory @ rb_sysopen - /var/run/td-agent/td-agent.pid

Error says directory doesn't exist and add --daemon option again.

This seems not fluentd bug. closed..

Facing same issue, here are my conf and issue details. Is there anyone can help
ulimit -n
65536
cat /etc/td-agent/td-agent.conf
type syslog tag graylog2
type gelf host <IP_ADDRESS> port 12201 flush_interval 5s
cat /etc/rsyslog.conf
. @<IP_ADDRESS>:5140
/usr/sbin/td-agent-gem list --local
fluent-config-regexp-type (1.0.0)
fluent-logger (0.8.0)
fluent-plugin-elasticsearch (3.5.1)
fluent-plugin-kafka (0.9.4)
fluent-plugin-prometheus (1.4.0)
fluent-plugin-record-modifier (2.0.1)
fluent-plugin-rewrite-tag-filter (2.2.0)
fluent-plugin-s3 (1.1.10)
fluent-plugin-td (1.0.0)
fluent-plugin-td-monitoring (0.2.4)
fluent-plugin-webhdfs (1.2.3)
fluentd (1.5.2, 1.4.2)
gelf (3.1.0)
systemctl status td-agent.service
● td-agent.service - td-agent: Fluentd based data collector for Treasure Data
Loaded: loaded (/etc/systemd/system/td-agent.service; disabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Fri 2019-06-21 11:37:24 GMT; 44min ago
Docs: https://docs.treasuredata.com/articles/td-agent
Process: 29745 ExecStart=/opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=1/FAILURE)
Jun 21 11:37:23 lit-rhel7-test3 systemd[1]: td-agent.service: control process exited, code=exited status=1
Jun 21 11:37:23 lit-rhel7-test3 systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Jun 21 11:37:23 lit-rhel7-test3 systemd[1]: Unit td-agent.service entered failed state.
Jun 21 11:37:23 lit-rhel7-test3 systemd[1]: td-agent.service failed.
Jun 21 11:37:24 lit-rhel7-test3 systemd[1]: td-agent.service holdoff time over, scheduling restart.
Jun 21 11:37:24 lit-rhel7-test3 systemd[1]: Stopped td-agent: Fluentd based data collector for Treasure Data.
Jun 21 11:37:24 lit-rhel7-test3 systemd[1]: start request repeated too quickly for td-agent.service
Jun 21 11:37:24 lit-rhel7-test3 systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Jun 21 11:37:24 lit-rhel7-test3 systemd[1]: Unit td-agent.service entered failed state.
Jun 21 11:37:24 lit-rhel7-test3 systemd[1]: td-agent.service failed.

@repeatedly
Same here... found this issue while searching for failure message after setup a new instance by saltstack unmodified after some months of not needed server changes (=> no modifications done on my side but td-agent updates on "your" side).
This may be "not a fluent bug" but it is somehow a combined td-agent/fluent packaging / using bug... offered in td-agent repository as origin package.
From release history the other repository:
https://github.com/treasure-data/omnibus-td-agent/releases
didn't match the offered packages:
# apt-cache policy td-agent
td-agent:
  Installed: 3.4.0-0
  Candidate: 3.4.0-0
  Version table:
 *** 3.4.0-0 900
        900 http://packages.treasuredata.com/3/debian/stretch stretch/contrib amd64 Packages
        100 /var/lib/dpkg/status
     3.3.0-1 900
        900 http://packages.treasuredata.com/3/debian/stretch stretch/contrib amd64 Packages
     3.3.0-0 900
        900 http://packages.treasuredata.com/3/debian/stretch stretch/contrib amd64 Packages
     3.2.1-0 900
        900 http://packages.treasuredata.com/3/debian/stretch stretch/contrib amd64 Packages
     3.2.0-0 900
        900 http://packages.treasuredata.com/3/debian/stretch stretch/contrib amd64 Packages
     3.1.1-0 900
        900 http://packages.treasuredata.com/3/debian/stretch stretch/contrib amd64 Packages
     3.1.0-0 900
        900 http://packages.treasuredata.com/3/debian/stretch stretch/contrib amd64 Packages

but maybe it's regardless the right repository for the issue.
But then please notify it and do not only close the issue here as "not our bug/problem".
From systemd control file the right config is given by ENV but not used:
# systemctl cat td-agent.service
# /lib/systemd/system/td-agent.service
[Unit]
Description=td-agent: Fluentd based data collector for Treasure Data
Documentation=https://docs.treasuredata.com/articles/td-agent
After=network-online.target
Wants=network-online.target

[Service]
User=td-agent
Group=td-agent
LimitNOFILE=65536
Environment=LD_PRELOAD=/opt/td-agent/embedded/lib/libjemalloc.so
Environment=GEM_HOME=/opt/td-agent/embedded/lib/ruby/gems/2.4.0/
Environment=GEM_PATH=/opt/td-agent/embedded/lib/ruby/gems/2.4.0/
Environment=FLUENT_CONF=/etc/td-agent/td-agent.conf
Environment=FLUENT_PLUGIN=/etc/td-agent/plugin
Environment=FLUENT_SOCKET=/var/run/td-agent/td-agent.sock
Environment=TD_AGENT_OPTIONS=
EnvironmentFile=-/etc/sysconfig/td-agent
PIDFile=/var/run/td-agent/td-agent.pid
RuntimeDirectory=td-agent
Type=forking
ExecStart=/opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS
ExecStop=/bin/kill -TERM ${MAINPID}
ExecReload=/bin/kill -HUP ${MAINPID}
Restart=always
TimeoutStopSec=120

[Install]
WantedBy=multi-user.target

where it

has mentioned bug https://github.com/treasure-data/omnibus-td-agent/issues/207 from other repository,
misses configuration commands to create it's run folder and rights like:ExecStartPre=/bin/mkdir -p /var/run/td-agent
ExecStartPre=/bin/chown td-agent:td-agent /var/run/td-agent
ExecStartPre=/bin/chmod 755 /var/run/td-agent

and is no "foreign bug" not as mentioned by you:

Error says directory doesn't exist and add --daemon option again.

where it's clear the somehow fluentd (=> this repository...) ignores the environment FLUENT_CONF.

Btw. it seems that the "pre-packaged" gems / ruby installation in td-agent package seems not sufficient/complete enough so newer packages can break this service in above mentioned behavior.
One workaround I found was to remove the debugging port setup by default like:
## live debugging agent
<source>
        @type debug_agent
        bind <IP_ADDRESS>
        port 24230
</source>

but it was not very useful because it helped only to drop the "port already bound" error and let further errors open.
I had setup before latest rkhunter from Debian testing with pin prio <100 so there should't be testing packages installed if not explicit requested.
But somehow there where all dependencies also upgraded to testing packages;
here the ones which aren't Perl based:
 libc6
 libc6-dev
 libc-bin
 libc-dev-bin
 libc-l10n
 libgnutls30
 libhogweed4
 libidn2-0
 libnettle6
 libp11-kit0
 libssl1.1
 libssl-dev
 libtasn1-6
 locales
 locales-all
 python3-boto
 python-boto

After downgrading them back to Debian Stretch versions td-agent/fluentd works again fine.
Hope this helps as hint also other people to fix their equal errors.

Why is it looking for config here /etc/fluent/fluent.conf when the installation guide

Because systemd unit file overwrites this value.
See td-agent systemd unit file: https://github.com/treasure-data/omnibus-td-agent/blob/d2f46d1d1f24f34b92b13fd26c3883b3be63ca94/templates/etc/systemd/td-agent.service.erb#L14

My comment is for /opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS error.
Your error with systemctl is other reason.

@jroberts07
To begin with, your reproduction step seems wrong.
As @repeatedly said, running td-agent via systemd completely diffs from running it by your command(/opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTION).  systemd sets enviroment variables for td-agent. No such file or directory @ rb_sysopen - /etc/fluent/fluent.conf has nothing to do with your systemd's error.
Also,  Please give us other logs, such as journalctl's ? systemd's log does not help as you know.

In the end I solved it by creating a symlink file /etc/fluent/fluent.conf that points to td-agent.conf. And also changing the permissions of the var/run folder so td-agent user can read it

I am still facing the same issue, pasting all my configs. Please help me with a solid fix
cat /etc/td-agent/td-agent.conf
# Fluentd GELF output
<source>
  @type syslog
  tag graylog2
</source>

<match graylog2.**>
  @type gelf
  host <graylog-server-fqdn>
  port 12201
  <buffer>
    flush_interval 5s
  </buffer>
</match>

cat /etc/rsyslog.conf
# Fluentd info
*.* @<IP_ADDRESS>:5140

/usr/sbin/td-agent-gem list --local
fluent-config-regexp-type (1.0.0)
fluent-logger (0.8.2)
fluent-plugin-elasticsearch (4.0.9)
fluent-plugin-kafka (0.13.0)
fluent-plugin-prometheus (1.8.0)
fluent-plugin-prometheus_pushgateway (0.0.2)
fluent-plugin-record-modifier (2.1.0)
fluent-plugin-rewrite-tag-filter (2.3.0)
fluent-plugin-s3 (1.3.2)
fluent-plugin-systemd (1.0.2)
fluent-plugin-td (1.1.0)
fluent-plugin-td-monitoring (0.2.4)
fluent-plugin-webhdfs (1.2.5)
fluentd (1.11.1)
gelf (3.1.0)

cat /lib/systemd/system/td-agent.service
[Unit]
Description=td-agent: Fluentd based data collector for Treasure Data
Documentation=https://docs.treasuredata.com/articles/td-agent
After=network-online.target
Wants=network-online.target

[Service]
User=td-agent
Group=td-agent
LimitNOFILE=65536
Environment=LD_PRELOAD=/opt/td-agent/embedded/lib/libjemalloc.so
Environment=GEM_HOME=/opt/td-agent/embedded/lib/ruby/gems/2.4.0/
Environment=GEM_PATH=/opt/td-agent/embedded/lib/ruby/gems/2.4.0/
Environment=FLUENT_CONF=/etc/td-agent/td-agent.conf
Environment=FLUENT_PLUGIN=/etc/td-agent/plugin
Environment=FLUENT_SOCKET=/var/run/td-agent/td-agent.sock
Environment=TD_AGENT_LOG_FILE=/var/log/td-agent/td-agent.log
Environment=TD_AGENT_OPTIONS=
EnvironmentFile=-/etc/sysconfig/td-agent
PIDFile=/var/run/td-agent/td-agent.pid
RuntimeDirectory=td-agent
Type=forking
ExecStart=/opt/td-agent/embedded/bin/fluentd --log $TD_AGENT_LOG_FILE --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS
ExecStop=/bin/kill -TERM ${MAINPID}
ExecReload=/bin/kill -HUP ${MAINPID}
Restart=always
TimeoutStopSec=120

[Install]
WantedBy=multi-user.target

stat /lib/systemd/system/td-agent.service
  File: ‘/lib/systemd/system/td-agent.service’
  Size: 1087            Blocks: 8          IO Block: 4096   regular file
Device: fd02h/64770d    Inode: 17060040    Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (  991/td-agent)   Gid: (  988/td-agent)
Access: 2020-09-08 06:47:36.856881024 +0000
Modify: 2020-09-02 11:18:57.800655295 +0000
Change: 2020-09-08 07:07:44.017793124 +0000
 Birth: -

cat td-agent.log
2020-09-08 03:12:01 +0000 [info]: #0 flushing all buffer forcedly
2020-09-08 06:40:27 +0000 [info]: Received graceful stop
2020-09-08 06:40:27 +0000 [info]: Received graceful stop
2020-09-08 06:40:27 +0000 [info]: #0 fluentd worker is now stopping worker=0
2020-09-08 06:40:27 +0000 [info]: #0 shutting down fluentd worker worker=0
2020-09-08 06:40:27 +0000 [info]: #0 shutting down input plugin type=:forward plugin_id="input_forward"
2020-09-08 06:40:27 +0000 [info]: #0 shutting down input plugin type=:http plugin_id="input_http"
2020-09-08 06:40:27 +0000 [info]: #0 shutting down input plugin type=:debug_agent plugin_id="input_debug_agent"
2020-09-08 06:40:27 +0000 [info]: #0 shutting down output plugin type=:tdlog plugin_id="output_td"
2020-09-08 06:40:27 +0000 [info]: #0 shutting down output plugin type=:stdout plugin_id="output_stdout"
2020-09-08 06:40:27 +0000 [info]: Worker 0 finished with status 0

systemctl restart td-agent
Job for td-agent.service failed because the control process exited with error code. See "systemctl status td-agent.service" and "journalctl -xe" for details.
systemctl status td-agent.service
● td-agent.service - td-agent: Fluentd based data collector for Treasure Data
   Loaded: loaded (/usr/lib/systemd/system/td-agent.service; disabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Tue 2020-09-08 07:15:35 GMT; 4s ago
     Docs: https://docs.treasuredata.com/articles/td-agent
  Process: 38074 ExecStart=/opt/td-agent/embedded/bin/fluentd --log $TD_AGENT_LOG_FILE --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=1/FAILURE)
 Main PID: 3438 (code=exited, status=0/SUCCESS)

Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service: control process exited, code=exited status=1
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Unit td-agent.service entered failed state.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service failed.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service holdoff time over, scheduling restart.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Stopped td-agent: Fluentd based data collector for Treasure Data.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: start request repeated too quickly for td-agent.service
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Unit td-agent.service entered failed state.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service failed.

journalctl -xe
Sep 08 07:15:35 lit-vatam-t009 fluentd[38074]: from /opt/td-agent/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:5
Sep 08 07:15:35 lit-vatam-t009 fluentd[38074]: from /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.11.1/bin/fluentd:
Sep 08 07:15:35 lit-vatam-t009 fluentd[38074]: from /opt/td-agent/embedded/bin/fluentd:23:in `load'
Sep 08 07:15:35 lit-vatam-t009 fluentd[38074]: from /opt/td-agent/embedded/bin/fluentd:23:in `<main>'
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service: control process exited, code=exited status=1
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
-- Subject: Unit td-agent.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit td-agent.service has failed.
--
-- The result is failed.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Unit td-agent.service entered failed state.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service failed.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service holdoff time over, scheduling restart.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Stopped td-agent: Fluentd based data collector for Treasure Data.
-- Subject: Unit td-agent.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit td-agent.service has finished shutting down.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: start request repeated too quickly for td-agent.service
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data.
-- Subject: Unit td-agent.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit td-agent.service has failed.
--
-- The result is failed.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: Unit td-agent.service entered failed state.
Sep 08 07:15:35 lit-vatam-t009 systemd[1]: td-agent.service failed.
Sep 08 07:15:40 lit-vatam-t009 systemd[1]: Configuration file /usr/lib/systemd/system/td-agent.service is marked executable. Pl

If any more info is needed please let me know


testing status updates on github (don't review)

dont review

Can someone verify this PR?

test123

Refer to this link for build results (access rights to CI server needed):
http://www.jenkins.clap.com/job/ClusterHQ-flocker/job/flocker-updateGHstatus_from_jenkins_FLOC-3336/job/run_trial_on_AWS_Ubuntu_Trusty_flocker.restapi/11/

Refer to this link for build results (access rights to CI server needed):
http://www.jenkins.clap.com/job/ClusterHQ-flocker/job/flocker-updateGHstatus_from_jenkins_FLOC-3336/job/run_trial_on_AWS_Ubuntu_Trusty_flocker.restapi/12/


Add support for HttpHeaders for Dart 1.0

I did make a functional change, but there are no tests to update, so I don't know if tests are needed for that case.

FYI, these changes are needed in order to bring web_server to Dart 2 and still have it support Dart 1


Doubled colon problem - Unknown command ::f

To print function-valued variable we should use ::f instead f the same as in the :type command. This cause error message:
kotlin> fun f() = 1
kotlin> :t ::f
kotlin.reflect.KFunction0<kotlin.Int>
kotlin> ::f
Unknown command ::f

If we will fix it we would have something like this
kotlin> fun f() = 1
kotlin> ::f
res1: kotlin.reflect.KFunction0<kotlin.Int> = fun Line_3.f(): kotlin.Int

Is it really okey or Line_3.f() should be replaced to f()?

Fixed in 0.1.2, but result will be in form fun Line_3.f(): kotlin.Int.


add t1ha (Fast Positive Hash)

Fast Positive Hash - just the fastest portable hash function.
Briefly, it is a portable 64-bit hash function:

Intended for 64-bit little-endian platforms, predominantly for Elbrus and x86_64,
but portable and without penalties it can run on any 64-bit CPU.
In most cases up to 15% faster than StadtX hash, xxHash, mum-hash, metro-hash, etc.
and all others portable hash-functions (which do not use specific hardware tricks).
Provides a set of terraced hash functions.
Currently not suitable for cryptography.
Licensed under Zlib License.

Have added this to my "to-do eventually" list.


Links to other Metagrid / search sites

https://github.com/aims-group/metagrid/pull/540
Needs better placement


call TADs

Hi Ruochi,
Nice work!
Now I impute a scHi-C data and generate corresponding cool files. I want to use Higashi to call TADs to generate the number and fraction of corresponding TADs. Can Higashi call TADs for a single cool file?

The Higashi can call scTADs but not on the cool file but the .h5 file it genereates. See wiki here:
https://github.com/ma-compbio/Higashi/wiki/Single-cell-TAD-calling
Thanks!


[5.1] Mark test dynamic_replacement_multifile as UNSUPPORTED in optim…

…ize mode
rdar://52073880

@swift-ci Please test and merge