watchtower: Add retry logic for fetching blocks

Change Description
Fixes #8205 , This change sets the total retry attempt count for fetching blocks to 4 (1 original attempt + 3 Retries).
Steps to Test
I'm uncertain about how reviewers might naturally replicate a scenario in which the backend fails to deliver blocks to lnd. For testing purposes, I introduced a probabilistic failure in the GetBlock function (by returning a non-nil error). Below is the code snippet I used to implement this probabilistic error injection. For the purposes of testing, I have used bitcoind as backend.
type MyCustomError struct {
	Msg string
}

func (e MyCustomError) Error() string {
	return fmt.Sprintf("Error: %s", e.Msg)
}

func (b *BtcWallet) GetBlock(blockHash *chainhash.Hash) (*wire.MsgBlock, error) {
	rand.Seed(time.Now().UnixNano())
	randomNumber := rand.Intn(100) + 1
	_, ok := b.chain.(*chain.NeutrinoClient)
	if !ok {
		k, v := b.blockCache.GetBlock(blockHash, b.chain.GetBlock)
		if randomNumber < 20 {
			return k, v
		}
		return k, MyCustomError{Msg: "Animesh: Pseudo Error"}
	}

	// For the neutrino implementation of lnwallet.BlockChainIO the neutrino
	// GetBlock function can be called directly since it uses the same block
	// cache. However, it does not lock the block cache mutex for the given
	// block hash and so that is done here.
	b.blockCache.HashMutex.Lock(lntypes.Hash(*blockHash))
	defer b.blockCache.HashMutex.Unlock(lntypes.Hash(*blockHash))

	return b.chain.GetBlock(blockHash)
}

Testing

This represents a scenario when block is finally fetched after all the attempts have been exhausted.

This represents a scenario when block is fetched after 2 attempts (1 original + 1 retry) have been exhausted.

This represents a scenario when block fetching fails.
I've been contemplating the practicality of implementing a unit test for this function. Given that we'll be mocking the GetBlock function to return either a nil or a non-nil error, I'm curious about the overall usefulness of such a test. Could someone help clarify if there's a significant benefit to this approach? I'm open to insights or suggestions on how we might enhance the testing strategy for this part of the code.

@ellemouton  I have implemented the change you suggested, I have further added a bool in the list of returned values, the purpose of the that is early exit so that we don't run into something weird (we might not, but I've placed it there for safety).

Thanks @anibilthare :) Here is my suggested patch to your diff (i havent tested it yet so please do so): towerRetry.patch
I've re-added the max-retries as well since we probably dont want a failure in 1 block to prevent us from fetching more blocks forever (my bad - I should have thought of this before 🙈 )
So this improves what we have today (which tries once and then moves on) but eventually (in a different PR) we should be able to handle the case where one block fetch continues to fail: we should not let that prevent us from moving on but should also not just never get back to that block. So we should either handle retries of past blocks explicitly or we should just do a hard error out (ie, LND will be forced to restart) if we cannot fetch the block after x number of retries since then clearly something is wrong.

Thanks for the patch @ellemouton  I have pushed the changes according to your suggestions. PTAL.

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

closing as it seems the author has disappeared

@ellemouton  can I continue on this ? will it be possible for you to reopen this ?

@anibilthare  - re-opened 👍

@ellemouton  I have rebased my changes and updates release notes as well.
Can you please let me know if I need to move my release notes to some other file ?
Apart from this can how please also tell me how do we decide where to put these release notes

@ellemouton: review reminder

@ellemouton: review reminder

@ellemouton  are these test locally runnable ?

@ellemouton  I have updated the docs. Thanks for the review

@saubyk pls take a look

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

@anibilthare, remember to re-request review from reviewers when ready

cc @guggero for override merge 🙏  (failing required check is unrelated)


Derive macro

How to use this macro correctly?
Where use this macro?
When use this macro?
Friday, on Globo Reporter.

I don't want waiting to friday:
https://doc.rust-lang.org/book/first-edition/procedural-macros.html


remove INDEPENDENT from compatibility_visitor checks (issue #303)

Independent should not be supported by NMODL as stated in:
https://www.neuron.yale.edu/neuron/static/py_doc/modelspec/programmatic/mechanisms/nmodl.html
Thus, the error can (and should be) dropped.

INDEPENDENT if it exists must declare t and the units must be milliseconds. I believe there are on the order of 240 models (and many more mod files) in ModelDB that use this construction. Often in the form
INDEPENDENT {t FROM 0 TO 1 WITH 1 (ms)}

INDEPENDENT if it exists must declare t and the units must be milliseconds. I believe there are on the order of 240 models (and many more mod files) in ModelDB that use this construction.

@nrnhines : what are semantics of this INDEPENDENT? how does that affect generated C  code?
I quickly checked generated C file from NEURON with and without INDEPENDENT but couldn't see any difference.
→ ~/nrn/build/bin/nocmodl tmp/nafx.mod
→ ~/nrn/build/bin/nocmodl tmp/nafx_without.mod
→ diff tmp/nafx.c tmp/nafx_without.c
185c185
<  static double delta_t = 1;
---
>  static double delta_t = 0.01;
270c270
<  void _nafx_reg() {
---
>  void _nafx_without_reg() {
291c291
<  	ivoc_help("help ?1 Nafx /Users/kumbhar/workarena/repos/bbp/netpyne/tmp/nafx.mod\n");
---
>  	ivoc_help("help ?1 Nafx /Users/kumbhar/workarena/repos/bbp/netpyne/tmp/nafx_without.mod\n");
720c720
< static const char* nmodl_filename = "/Users/kumbhar/workarena/repos/bbp/netpyne/tmp/nafx.mod";
---
> static const char* nmodl_filename = "/Users/kumbhar/workarena/repos/bbp/netpyne/tmp/nafx_without.mod";
739d738
<   "INDEPENDENT {t FROM 0 TO 1 WITH 1 (ms)}\n"

The delta_t is not used anywhere in the code. (I haven't looked inside nocmodl or mod2c implementation though).

Although INDEPENDENT is not required, if it exists it must be a valid NMODL statement syntactically
and semantically it must declare t with units of (ms). Thus
INDEPENDENT { t (millisec) }

gives the error message
$ nocmodl temp.mod
Translating temp.mod into temp.c
syntax error:
 name FROM number TO number WITH integer
 at line 2 in file temp.mod
INDEPENDENT { t (millisec) }
                ^

INDEPENDENT { x FROM 0 TO 1 WITH 1 (millisec) }

is accepted by nmodl  (to my surprise) but appears nowhere in the c file. I believe this should be
an error.
INDEPENDENT { t FROM 0 TO 1 WITH 1 (meter) }

is accepted by nocmodl but modlunit gives
$ modlunit temp.mod
model   <IP_ADDRESS>   1994/10/12 17:22:51
Checking units of temp.mod
t must have the units, ms, instead of meter.
 at line 1 in file temp.mod
NEURON<<ERROR>> {SUFFIX test}

In summary, it appears that it is acceptable for NMODL to only check the syntax and otherwise
ignore the INDEPENDENT block.

@cattabiani : was anything pending here? If not, could you rebase/merge with master and make this up to date?

please retest

everything passed \o/


Update libs

jquery to 3.2.1
jquery.growl to 1.3.5
jquery.timeago to 1.6.7
bootstrap to 3.4.1

This also fixes #92

Thanks :)


Adding Redacted Keys

Goal

Filtering config option is case sensitive and is no longer in line with our notifier spec; as such it should be deprecated in favour of redactedKeys.
Design

Added redactedKeys configuration option, which is case insensitive, as a replacement for the filters configuration option. Additionally due to conflictions the default filters parameters were changed.
Changeset

Added redactedKeys config option.

Made redactedKeys case insensitive.

Changed filters config default parameters from
 {"password", "secret", "Authorization", "Cookie"};
to
{"ipAddress", "logLevel"};

Set refactedKeys config default parameters to
 {"password", "secret", "Authorization", "Cookie"};

Testing

Added and updated tests for filtering mechanism and redactedKeys mechanism.

May have to discuss this change with reviewer. It was causing conflicts in tests as these two methods are so similar.
Changed filters config default parameters from
 {"password", "secret", "Authorization", "Cookie"};
to
{"ipAddress", "logLevel"};


Is Account Ordering Possible?

Looking through Actual's API it looks like you are returning every property of an Account  they expose, but I just wanted to ask if you knew of a way to pull any additional information or properties to determine the order the accounts should be listed in, specifically the same order they are in on the web. Thanks!

Hey there, the api already returns the accounts sorted as they appear in the ui, see the nodejs api logic: https://github.com/actualbudget/actual/blob/master/packages/loot-core/src/server/db/index.ts#L570
Unfortunately the nodejs api doesn’t return anymore fields that can be used for sorting, only these 4 fields: https://github.com/actualbudget/actual/blob/master/packages/loot-core/src/server/api-models.ts#L18-L25
If you want to sort based on a field that is not there, or based in the order from the UI but say in reverse, you would  have to modify the nodejs actual api (pull request) to expose the fields you want to use.
We could also change this http api to use the ActualQL language to query the accounts and sort them somehow, but seems more like a hack, so I like the former approach more.

That makes sense thank you! The issue must be in how I'm processing and storing the accounts.
I have another question related to categories and budget months. Should I ask here or open another issue?

You can ask here too

Great thanks!
I am trying to better understand the relationship, if any, between BudgetMonthCategoryGroup / BudgetMonthCategory and CategoryGroup/Category. I know they have the same ids, though don't share every single property. How each impacts the overall budget is still confusing me a bit.
For example, if I want to rename a category, do I update both objects, or just one? Same with deleting a category. I created a test category and then went to delete it. Since there is only a delete for a Category, that worked fine. However, with no method to delete a BudgetMonthCategory, that remained and it appears deleting just the Category had no impact at all. Does that mean CategoryGroup and Category don't need to be used at all?
I know that was long, but I appreciate any help!

BudgetMonthCategory represents the category properties that are tied to a specific month, I.e budgeted amount, and carry over. Category represents the category configuration, name, grouping, hidden, etc. To manage categories use the Category endpoints, to change budgeted amounts for a particular month and category, use BudgetMonthCategory.
If you want to delete a category, the Category delete endpoint should do it. If it didn’t work it’s probably cuz you have transactions using that category, so you want to update those transactions to point to a different category, and then delete the category.

That is incredibly helpful, thank you so much!

Going back to my accounts ordering question. I ran some test queries in Postman on accounts, and it does return in the same order each time, but it is not in the order that the UI has. I cannot tell why it is returning in the specific order it is, to be honest heh.

Mmmm strange. You can try exporting your data from the Ui, that gives you a zip file. You unzipped and you get a db.sqlite file. You can open it with any SQLite viewer, and explore the records in the table accounts. There there is a column called sorting or something like that. That tells you what is the sorting of your accounts. AFAIK that’s the sorting used in the Ui, and that’s the order used by the api too. It appears in that order for me.
Can you try the request using curl command ? Maybe Postman is formatting the output ?

I tried it using curl and its the same order as Postman and how I implemented it. I will look at the sqlite file and see what is going on. I figured it was something wonky with my budget haha.
I appreciate your help!

You are very welcome!


Exchanging Gadget not able to select Productive Bees Comb Blocks

Working with BlueFireFox on the Direcord on an issue is discovered and wanted to submit a tkt on it for investigation.
Productive Bees Comb Blocks, which appear to be a normal block entity, produce an error when you attempt to select them, (See IMGUR gallery for screenshots.  I am able to select normal Vanilla Comb blocks with no issues.  I am also able to replace the PB Comb blocks with other blocks with no issue but not able to undo back.  No error is given when attempting to undo them back.  I am assuming it may be something do do with the raft of NBT and meta data attached to the PB Combs but though I'd submit it anyway incase there is some other tom foolery occurring.
https://imgur.com/a/mqVT7wf

This was observed in ATM8, buildinggadgets-3.16.2-build.22+mc1.19.2

Cap you ever give dire a break xD I think between you and howie yall found most of the bugs :3

Can you let me know if this still happens in 1.21?

Ok, in 1.21 it now does select all blocks, Productive Bees Comb blocks included, But when pasting in of the 6 blocks I placed down and copied, 2 each of 3 types, The 2st 2 pasted in properly but the other 4 did not.  The other blocks are Black, so their texture did not load properly which may be a Productive Bees interaction problem as much as a BG2 issue.  But they also, when block picked, show as the 1st type of comb not the type they were in the initial copy.  So I would say better but still not playing nice.

Its likely just a cross mod compat i won't be easily able to fix at this time, sorry!


InvalidSchema: No connection adapters were found for

Hi, I get this error, "InvalidSchema: No connection adapters were found for"
Do you have solution for this issue?

Removing the curly braces from your URL should fix the problem.

Hi, i remove the curly braces, but still get another issue,
import pyktok as pyk
pyk.specify_browser("firefox")
pyk.save_tiktok("https://www.tiktok.com/@vantoan___/video/7294298719665622305?is_from_webapp=1&sender_device=pc",
True,
'video_data.csv',
browser_name = "firefox")


Feature added to Find all Donors

Added a feature to find all donors
Fixes #140

hey @Saurav-Pant check it out

Looks Good


[5.4] Random optional strict mode

In this PR , collection random() method would have optionally strict mode so if I don't know the count of the array but I just need to get 4 items from it, I don't have to check the count first.
summary :
$array = collect([1,2,3])->random(4,false); // [1,2,3]

instead of
$array = collect([1,2,3]);
if($array->count() > 4){
    $array = $array->random(4);
}

Is this even necessary?  The function is already throwing an InvalidArgumentException when the $amount is greater than the count.
Having strict and nonstrict modes and boolean as a parameter just pollutes the interface.
We should keep the interface the same as php's array_rand.
http://php.net/array_rand

That's why I set the strict mode true by default , the point here is to write less code for picking limited random items from collection.
PS in my case the scenario was like :

Get all campaigns
GroupBy asset
Map and Select random 1 from asset
Select random 2

Before this PR you need to check array count in 3,4. But after PR you don't need count checks at all.

I dont want to add a "strict" flag to a random method.


Error while installing

i got these errors find a solution on the internet but non of them work i use ubuntu 16.04
ERROR: Error [Errno 2] No such file or directory: 'git' while executing command git clone -q https://github.com/szagoruyko/pyinn.git /tmp/pip-req-build-qhmmsbku
ERROR: Cannot find command 'git' - do you have 'git' installed and in your PATH

Hello, would you like to ask you whether this problem has been solved?

u can try (pip install git)，and install again


Generic JSX props don't work properly when props contain generic functions

In 2.5.3, I was using JSX components with generic props containing functions successfully by wrapping stateful ones with stateless components, as also described in #6395 and in this comment
However, in 2.6.x and typescript@next, it breaks:
  Type '{ a: T; b: T[]; c: (t: T) => number; }' is not assignable to type 'Readonly<Props<{}>>'.
    Types of property 'c' are incompatible.
      Type '(t: T) => number' is not assignable to type '(t: {}) => number'.
        Types of parameters 't' and 't' are incompatible.
          Type '{}' is not assignable to type 'T'.
src/bug.tsx(32,18): error TS2322: Type '{ a: number; b: number[]; c: (x: number) => number; }' is not assignable to type 'IntrinsicAttributes & IntrinsicClassAttributes<Comp2<{}>> & Readonly<{ children?: ReactNode; }> &...'.
  Type '{ a: number; b: number[]; c: (x: number) => number; }' is not assignable to type 'Readonly<Props<{}>>'.
    Types of property 'c' are incompatible.
      Type '(x: number) => number' is not assignable to type '(t: {}) => number'.
        Types of parameters 'x' and 't' are incompatible.
          Type '{}' is not assignable to type 'number'.

TypeScript Version:  2.7.0-dev.20171224, 2.6.2
Code
import * as React from 'react';

interface Props<T> {
  a: T
  b: T[]
  c: (t: T) => number // This addition breaks it
}

class Comp2<T> extends React.Component<Props<T>> {
  render(){
    return null
  }
}

function Comp1<T>(props: Props<T>) {
    return <Comp2 {...props} /> // Error
}

const props = {
  a: 1,
  b: [2],
  c: (x: number) => x
}

const a = <Comp1 {...props} />

const b = <Comp2 {...props} /> // Same error

Expected behavior:
I expect it to work. It worked on 2.5.3
Actual behavior:
It breaks as soon as I add generic functions to the props
Config used:
{
    "compilerOptions": {
      "outDir": "build/dist",
      "module": "commonjs",
      "target": "es5",
      "lib": ["es6", "dom"],
      "sourceMap": true,
      "allowJs": true,
      "jsx": "react",
      "moduleResolution": "node",
      "rootDir": "src",
      "forceConsistentCasingInFileNames": true,
      "noImplicitReturns": true,
      "noImplicitThis": true,
      "noImplicitAny": true,
      "strictNullChecks": true,
      "suppressImplicitAnyIndexErrors": true,
      "experimentalDecorators": true,
      "emitDecoratorMetadata": true,
      "strict": true
      //"noUnusedLocals": true
    },
    "exclude": [
      "node_modules",
      "build",
      "scripts",
      "acceptance-tests",
      "webpack",
      "jest",
      "src/setupTests.ts"
    ],
    "types": [
      "typePatches"
    ]
  }

The example works as expected on 2.5.3.

For posterity, that this changed is due to your config containing strict: true, and 2.6 and onward introducing the strictFunctionTypes flag. If you set strictFunctionTypes to false you should no longer have the new errors.
As explaination for why what you've got is unsafe:
interface Props<T> {
  a: T
  b: T[]
}

is fine. You'll note there are no function types in your props, so we don't do any strict variance checking any anything. Once you add
  c: (t: T) => number // This addition breaks it

we start doing strict function type checking.

@weswigham Thank you for the reply.
From what I understand, variance doesn't come into play in my example, so this is not "working as intended", right?
I just want to make sure I understand correctly whether you're explaining why the example doesn't and shouldn't work or why the bug? appears in 2.6.


⚠ Bofrost.at has degraded performance

In c3dcc71, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 3963 ms

Resolved: Bofrost.at performance has improved in 1c58151 after 6 minutes.


pls address the issues created

i request developers to address the issues created. I am very new and my issues may be very naïve, but if answers are available, then it will be of great help.

Sorry for the delay in answering your questions. We will answer all of them in a couple of hours!


Let action work with dbt cloud jobs which don't produce artifacts

Fixes #23
Previously this action would always fail on dbt cloud jobs which don't produce artifacts, now the action can be configured to skip fetching artifacts. I considered catching & silencing errors when fetching artifacts rather than exposing another config option but decided against it since it could lead to hard-to-debug scenarios for this github action's primary use-case (running fal). Requiring a configuration opt-in to cover this niche instead feels quite harmless.
Questions

Do you have any tips for testing this? I've never worked on a Github Action before
Does the approach generally look ok to you?

I'm very open to any style or naming suggestions, and any other advice you have to give. Thanks

In regards to testing, we dont' really do it in this repo but in https://github.com/fal-ai/fal_bike_example.  Maybe you can test this internally by pointing to your branch?
      - uses: daniel-ferguson/dbt-cloud-action@allow-running-with-no-artifacts
        id: dbt_cloud_run
        with:
          dbt_cloud_token: ${{ secrets.DBT_CLOUD_API_TOKEN }}
          dbt_cloud_account_id: ${{ secrets.DBT_CLOUD_ACCOUNT_ID }}
          dbt_cloud_job_id: ${{ secrets.DBT_CLOUD_JOB_ID }}
          interval: 5
          failure_on_error: false

          cause: "Some cause"
          git_sha: ${{ github.sha }}
          # git_branch:
          # schema_override:
          # dbt_version_override:
          threads_override: 1
          # target_name_override:
          generate_docs_override: false
          # timeout_seconds_override:
          steps_override: |
            - dbt seed
            - dbt run

We should definetily add integration tests ourselves, though.

Realised I forgot to update the README with this new option, fixed

Ok, tested internally - caught a syntax error in the initial commit, I forgot that javascript needs parentheses in if statements.

Great, I did not realize that detail either! Thanks for the tests and adding it to the README. Merging!

And published in v4.2!
This a great quality PR, thanks for the help!


Add #[track_caller] to with_* and set_* functions.

I hit one of the debug assertions in my project recently, but the panic message was unhelpful because it pointed at the generated code instead of my own code.
Ideally, the panic message would also include the value that's out of bounds, but the feature that lets us format panics in const contexts is unstable.

Hi @rj00a
This is a good idea and definitely helps tracking error.


MAINT: use numpy-config and fix support for numpy 2.0

Note that we do have a job that installs numpy with --pre, but 2.0.0b1 gets uninstalled again because matplotlib (also used/needed in CI jobs) doesn't support it yet.

Okay, this is happy in CI and works locally with numpy 2.0.0b1, so let's give it a go.


Remove-RSUserProfile -Delete "User1", "User2 not working

We would like to use the command "Remove-RSUserProfile -Delete "User1", "User2"".  However, we always get an error
"It is not possible to call a method for an expression that has the NULL.In C:\ProgramFiles\WindowsPowerShell\Modules\DeleteUserProfile\0.1.2\DeleteUserProfile.psm1:179 characters:29

...                 if ($Profile.LocalPath.split('')[-1] -in $Exclude) {"

Can you fix this please?

@ui67 I'll take a look at it during this weekend

Thank you very much.
Von: Robin Stolpe @.>
Gesendet: Mittwoch, 5. April 2023 22:48
An: rstolpe/DeleteUserProfile @.>
Cc: Ilgin, Ugur @.>; Mention @.>
Betreff: Re: [rstolpe/DeleteUserProfile] Remove-RSUserProfile -Delete "User1", "User2 not working (Issue #24)
@ui67https://github.com/ui67 I'll take a look at it during this weekend
—
Reply to this email directly, view it on GitHubhttps://github.com/rstolpe/DeleteUserProfile/issues/24#issuecomment-1498132748, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A656XUE3SXLJ3UXIDSUSB3TW7XLA3ANCNFSM6AAAAAAWRKBC4Q.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>[mailsig01] https://www.heimhaus.de
HEIM & HAUS Produktion und Vertrieb GmbH
Hochstraße 7-9, 47169 Duisburg
USt-IdNr.: DE 272301801 - Sitz Duisburg - HRB 22438
Geschäftsführer: Michael Demming, Dipl.-Ing. Alexander Höpken, Dipl.-Kfm. Christoph Kersken

@ui67 I have been home with sick kids, but I'll take a look at this tomorrow evening.

@ui67 sorry for late replay, I'm working with a big update and it will run even faster.
I have also fixed your issue in that update.
I'm planning to release it tomorrow.
I have also added logging.

@ui67 this should be fixed in latest version 2.0.1


ehancement Add HTTP interface for q-error

Proposed changes

Add Http interface for query q-error
Fix the selectivity calculation of inner join, it would always be 0 if there is only one join condition before

Problem summary
Describe your changes.
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall


Add command-line switch -S/--display-style-guide

When enabled style guide URLs are added to offence messages. Setting
DisplayStyleGuide to true in the config has the same effect.

Coverage decreased (-0.2%) to 99.44% when pulling 81f8e3344a8a57f9da66181de55d7bd9761acbc7 on marxarelli:feature/style-guide-url-in-output into 76e8f88122b3277a0ae237982bc16df59aa1bb57 on bbatsov:master.

Coverage decreased (-0.2%) to 99.44% when pulling 81f8e3344a8a57f9da66181de55d7bd9761acbc7 on marxarelli:feature/style-guide-url-in-output into 76e8f88122b3277a0ae237982bc16df59aa1bb57 on bbatsov:master.

Coverage decreased (-0.2%) to 99.44% when pulling 81f8e3344a8a57f9da66181de55d7bd9761acbc7 on marxarelli:feature/style-guide-url-in-output into 76e8f88122b3277a0ae237982bc16df59aa1bb57 on bbatsov:master.

Isn't this an overkill? After all the style guide rule is part of a cop's description and can easily been retrieved from there.
Any opinions?

We are running rubocop via Jenkins whenever someone propose a pull request. We sometime have new developers slightly confused by the information message and the extended description on the web page is really helpful.

You forgot to update the changelog.

:+1:


Instanced mesh

This pull request adds support for instance colors to InstancedMesh.
It introduces a new attribute called 'instanceColor'.
I wonder now though if it isn't possible to simply use the existing 'color' attribute instead. Hmm!

Sorry this took a minute. This is not the right way of doing this i think. You tapped into the vertex colors which shouldn't be overridden by the instance colors. I think that instance colors should replace the color uniform of the material. The vColor should get filled with the vertex color attribute. This, along with a possible texture should be modulated/tinted by the instance color, the same way a material per instance would.

Ahh I had an inkling I wasn't doing it in the right way. So we would have to change the color uniform into an attribute? Or add an attribute and override the color uniform instead?

I think combine the two, the same way we use the mesh transformation along with the per instance transformation we could multiply the "global" color uniform with the instance color.

Sounds good - I will have another look at this next week.

I now pass the instanceColor to the fragment shader through varying vec3 vInstanceColor, where the global color uniform (diffuse) is multiplied by it. Let me know what you think.

Hey, i ended up doing this in InstancedMesh-dev before i figured out what's going on. The shader stuff is exactly what it should be, the only thing i did different is only create the color buffer if the flag is set and i made a normalized Uint8Array for the color instead of a Float32.

Oh i also tried to optimize the quat matrix stuff:
https://stackoverflow.com/questions/43825986/glsl-vector-addition-vs-multiple-floats
I grouped the rows so that they can be multiplied three at a time, and grouped (but unfortunately obfuscated) some of the quat stuff ( quat x 2 and various component multiplications ). If i understand this correctly it takes out 16 multiplications.


DI-26838 - add support of Empty / NotEmpty on the query builder

DI-26838 - add support of Empty / NotEmpty on the query builder

DI-26838 - add support of Empty / NotEmpty on the query builder


Dedup elementwise in LLVM IR based on constancy

Summary
When Triton GPU IR is lowered into LLVM IR, we can make use of the constancy information about the result of the elementwise ops to deduplicate otherwise redundant computation. That is the contribution of this PR: the constancy is checked and, if possible, some of the values in LLVM IR are reused multiple times instead of computing equal values separately.
The change is beneficial for the PyTorch 2 / TorchInductor-generated Triton code, as the leftmost sub-indices extracted from the flat index by div / mod operations can be equal, given sufficiently large 2^n factor in the rightmost rightmost dimension(s). This makes the computation resulting in those sub-indices redundant. Consequently, under the necessary constancy conditions, the redundant indexing arithmetics can be deduplicated. We observe up to 25% decrease in the latency of some of our jagged tensor kernels.
Example
For this simple Triton kernel:
@triton.jit
def triton_kernel(in_ptr, out_ptr, xnumel, XBLOCK: tl.constexpr):
    xoffset = tl.program_id(0) * XBLOCK
    xindex = xoffset + tl.arange(0, XBLOCK)[:]
    xmask = xindex < xnumel

    x1 = xindex // 256

    tmp = tl.load(in_ptr + x1, xmask)
    tl.store(out_ptr + xindex, tmp, xmask)

BLOCK_SIZE = 1024
input_t = torch.randn((1234,)).half().cuda()
output_t = torch.randn((1234, 256)).half().cuda()
grid = (triton.cdiv(output_t.numel(), BLOCK_SIZE),)
res = triton_kernel[grid](
    input_t,
    output_t,
    output_t.numel(),
    XBLOCK=BLOCK_SIZE,
    num_warps=4,
)

This is the Triton GPU IR annotated with the axis analysis info (including the constancy):
@triton_kernel_0d1d2de
%cst = arith.constant dense<256> : tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [256], constancy = [1024], constant_value = 256
%c1024_i32 = arith.constant 1024 : i32 => contiguity = [1], divisibility = [1024], constancy = [1], constant_value = 1024
%0 = tt.get_program_id x : i32 => contiguity = [1], divisibility = [1], constancy = [1], constant_value = <none>
%1 = arith.muli %0, %c1024_i32 : i32 => contiguity = [1], divisibility = [1024], constancy = [1], constant_value = <none>
%2 = tt.make_range {end = 1024 : i32, start = 0 : i32} : tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1024], divisibility = [1073741824], constancy = [1], constant_value = <none>
%3 = tt.splat %1 : (i32) -> tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [1024], constancy = [1024], constant_value = <none>
%4 = arith.addi %3, %2 : tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1024], divisibility = [1024], constancy = [1], constant_value = <none>
%5 = tt.splat %arg2 : (i32) -> tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [16], constancy = [1024], constant_value = <none>
%6 = "triton_gpu.cmpi"(%4, %5) <{predicate = 2 : i64}> : (tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>>, tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>>) -> tensor<1024xi1, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [1], constancy = [16], constant_value = <none>
%7 = arith.divsi %4, %cst : tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [1], constancy = [256], constant_value = <none>
%8 = tt.splat %arg0 : (!tt.ptr<f16, 1>) -> tensor<1024x!tt.ptr<f16, 1>, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [16], constancy = [1024], constant_value = <none>
%9 = tt.addptr %8, %7 : tensor<1024x!tt.ptr<f16, 1>, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>>, tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [2], constancy = [256], constant_value = <none>
%10 = tt.load %9, %6 {cache = 1 : i32, evict = 1 : i32, isVolatile = false} : tensor<1024xf16, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [1], constancy = [16], constant_value = <none>
%11 = tt.splat %arg1 : (!tt.ptr<f16, 1>) -> tensor<1024x!tt.ptr<f16, 1>, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1], divisibility = [16], constancy = [1024], constant_value = <none>
%12 = tt.addptr %11, %4 : tensor<1024x!tt.ptr<f16, 1>, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>>, tensor<1024xi32, #triton_gpu.blocked<{sizePerThread = [8], threadsPerWarp = [32], warpsPerCTA = [4], order = [0], CTAsPerCGA = [1], CTASplitNum = [1], CTAOrder = [0]}>> => contiguity = [1024], divisibility = [16], constancy = [1], constant_value = <none>

Before this PR, the LLVM IR (shown below) and the PTX (not shown here) both contain 8 redundant computations of the same x1 value for each element in the thread; the 8 loads are performed from the 8 independent pointers containing the same address:
define void @triton_kernel_0d1d2de(ptr addrspace(1) %0, ptr addrspace(1) %1, i32 %2) local_unnamed_addr !dbg !5 {
  %4 = tail call i32 @llvm.nvvm.read.ptx.sreg.tid.x(), !dbg !8
  %5 = shl i32 %4, 3, !dbg !8
  %6 = and i32 %5, 1016, !dbg !8
  %7 = tail call i32 asm "mov.u32 $0, %ctaid.x;", "=r"() #1, !dbg !9
  %8 = shl i32 %7, 10, !dbg !10
  %9 = or i32 %8, %6, !dbg !11
  %10 = or i32 %9, 1, !dbg !11
  %11 = or i32 %9, 2, !dbg !11
  %12 = or i32 %9, 3, !dbg !11
  %13 = or i32 %9, 4, !dbg !11
  %14 = or i32 %9, 5, !dbg !11
  %15 = or i32 %9, 6, !dbg !11
  %16 = or i32 %9, 7, !dbg !11
  %17 = icmp slt i32 %9, %2, !dbg !12
  %18 = icmp slt i32 %10, %2, !dbg !12
  %19 = icmp slt i32 %11, %2, !dbg !12
  %20 = icmp slt i32 %12, %2, !dbg !12
  %21 = icmp slt i32 %13, %2, !dbg !12
  %22 = icmp slt i32 %14, %2, !dbg !12
  %23 = icmp slt i32 %15, %2, !dbg !12
  %24 = icmp slt i32 %16, %2, !dbg !12
  %25 = sdiv i32 %9, 256, !dbg !13
  %26 = sdiv i32 %10, 256, !dbg !13
  %27 = sdiv i32 %11, 256, !dbg !13
  %28 = sdiv i32 %12, 256, !dbg !13
  %29 = sdiv i32 %13, 256, !dbg !13
  %30 = sdiv i32 %14, 256, !dbg !13
  %31 = sdiv i32 %15, 256, !dbg !13
  %32 = sdiv i32 %16, 256, !dbg !13
  %33 = sext i32 %25 to i64, !dbg !14
  %34 = getelementptr half, ptr addrspace(1) %0, i64 %33, !dbg !14
  %35 = sext i32 %26 to i64, !dbg !14
  %36 = getelementptr half, ptr addrspace(1) %0, i64 %35, !dbg !14
  %37 = sext i32 %27 to i64, !dbg !14
  %38 = getelementptr half, ptr addrspace(1) %0, i64 %37, !dbg !14
  %39 = sext i32 %28 to i64, !dbg !14
  %40 = getelementptr half, ptr addrspace(1) %0, i64 %39, !dbg !14
  %41 = sext i32 %29 to i64, !dbg !14
  %42 = getelementptr half, ptr addrspace(1) %0, i64 %41, !dbg !14
  %43 = sext i32 %30 to i64, !dbg !14
  %44 = getelementptr half, ptr addrspace(1) %0, i64 %43, !dbg !14
  %45 = sext i32 %31 to i64, !dbg !14
  %46 = getelementptr half, ptr addrspace(1) %0, i64 %45, !dbg !14
  %47 = sext i32 %32 to i64, !dbg !14
  %48 = getelementptr half, ptr addrspace(1) %0, i64 %47, !dbg !14
  %49 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %34, i1 %17) #1, !dbg !15
  %50 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %36, i1 %18) #1, !dbg !15
  %51 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %38, i1 %19) #1, !dbg !15
  %52 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %40, i1 %20) #1, !dbg !15
  %53 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %42, i1 %21) #1, !dbg !15
  %54 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %44, i1 %22) #1, !dbg !15
  %55 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %46, i1 %23) #1, !dbg !15
  %56 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %48, i1 %24) #1, !dbg !15
  %57 = sext i32 %9 to i64, !dbg !16
  %58 = getelementptr half, ptr addrspace(1) %1, i64 %57, !dbg !16
  %59 = insertelement <2 x i16> undef, i16 %49, i64 0, !dbg !17
  %60 = insertelement <2 x i16> %59, i16 %50, i64 1, !dbg !17
  %61 = bitcast <2 x i16> %60 to i32, !dbg !17
  %62 = insertelement <2 x i16> undef, i16 %51, i64 0, !dbg !17
  %63 = insertelement <2 x i16> %62, i16 %52, i64 1, !dbg !17
  %64 = bitcast <2 x i16> %63 to i32, !dbg !17
  %65 = insertelement <2 x i16> undef, i16 %53, i64 0, !dbg !17
  %66 = insertelement <2 x i16> %65, i16 %54, i64 1, !dbg !17
  %67 = bitcast <2 x i16> %66 to i32, !dbg !17
  %68 = insertelement <2 x i16> undef, i16 %55, i64 0, !dbg !17
  %69 = insertelement <2 x i16> %68, i16 %56, i64 1, !dbg !17
  %70 = bitcast <2 x i16> %69 to i32, !dbg !17
  tail call void asm sideeffect "@$5 st.global.v4.b32 [ $4 + 0 ], { $0, $1, $2, $3 };", "r,r,r,r,l,b"(i32 %61, i32 %64, i32 %67, i32 %70, ptr addrspace(1) %58, i1 %17) #1, !dbg !17
  ret void, !dbg !18
}

After this PR, the redundant computations in the LLVM IR shown above are deduplicated; as a result, the 8 loads are performed from the same pointer:
define void @triton_kernel_0d1d2de(ptr addrspace(1) %0, ptr addrspace(1) %1, i32 %2) local_unnamed_addr !dbg !5 {
  %4 = tail call i32 @llvm.nvvm.read.ptx.sreg.tid.x(), !dbg !8
  %5 = shl i32 %4, 3, !dbg !8
  %6 = and i32 %5, 1016, !dbg !8
  %7 = tail call i32 asm "mov.u32 $0, %ctaid.x;", "=r"() #1, !dbg !9
  %8 = shl i32 %7, 10, !dbg !10
  %9 = or i32 %8, %6, !dbg !11
  %10 = icmp slt i32 %9, %2, !dbg !12
  %11 = sdiv i32 %9, 256, !dbg !13
  %12 = sext i32 %11 to i64, !dbg !14
  %13 = getelementptr half, ptr addrspace(1) %0, i64 %12, !dbg !14
  %14 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %15 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %16 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %17 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %18 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %19 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %20 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %21 = tail call i16 asm sideeffect "mov.u16 $0, 0x0;\0A\09@$2 ld.global.b16 { $0 }, [ $1 + 0 ];", "=c,l,b"(ptr addrspace(1) %13, i1 %10) #1, !dbg !15
  %22 = sext i32 %9 to i64, !dbg !16
  %23 = getelementptr half, ptr addrspace(1) %1, i64 %22, !dbg !16
  %24 = insertelement <2 x i16> undef, i16 %14, i64 0, !dbg !17
  %25 = insertelement <2 x i16> %24, i16 %15, i64 1, !dbg !17
  %26 = bitcast <2 x i16> %25 to i32, !dbg !17
  %27 = insertelement <2 x i16> undef, i16 %16, i64 0, !dbg !17
  %28 = insertelement <2 x i16> %27, i16 %17, i64 1, !dbg !17
  %29 = bitcast <2 x i16> %28 to i32, !dbg !17
  %30 = insertelement <2 x i16> undef, i16 %18, i64 0, !dbg !17
  %31 = insertelement <2 x i16> %30, i16 %19, i64 1, !dbg !17
  %32 = bitcast <2 x i16> %31 to i32, !dbg !17
  %33 = insertelement <2 x i16> undef, i16 %20, i64 0, !dbg !17
  %34 = insertelement <2 x i16> %33, i16 %21, i64 1, !dbg !17
  %35 = bitcast <2 x i16> %34 to i32, !dbg !17
  tail call void asm sideeffect "@$5 st.global.v4.b32 [ $4 + 0 ], { $0, $1, $2, $3 };", "r,r,r,r,l,b"(i32 %26, i32 %29, i32 %32, i32 %35, ptr addrspace(1) %23, i1 %10) #1, !dbg !17
  ret void, !dbg !18
}

Test Plan
New lit-based unit tests have been added to test/Conversion/dedup-by-constancy.mlir:
$ lit test
-- Testing: 39 tests, 39 workers --
...

Testing Time: 0.70s
  Passed           : 38
  Expectedly Failed:  1

@aakhundov This would benefit my kernels a lot as well. Related to this, would it be possible to achieve the same deduplication if you had non-contiguous repeating elements? For example, withx1 = xindex % 256 instead of x1 = xindex // 256.

I think the heuristic would be general since he checks the constancy of the result

Do you have any benchmark that demonstrate the speedup? That would be great

@francescocarzaniga

For example, withx1 = xindex % 256 instead of x1 = xindex // 256.

In the case of x1 = xindex % 256, the resulting x1 would have contiguity but not constancy. As the heuristic relies on the constancy for deduplication across the multiple (contiguous) elements in a single thread, I believe, dedup won't happen here.

@aakhundov I see, that's unfortunate. It would be great to have both kinds of deduplication working as they are complementary, but I'm no compiler expert so I have no idea how hard it would be to implement it.

@aakhundov python3 -m pre_commit run --all-files --verbose

@aakhundov I see, that's unfortunate. It would be great to have both kinds of deduplication working as they are complementary, but I'm no compiler expert so I have no idea how hard it would be to implement it.

Contiguity wouldn't cause duplication I think, but x % 256 does cause duplication if x is a vector and has constancy > 1 (e.g., [4, 4, 4, 4] % 256)?

Contiguity wouldn't cause duplication I think, but x % 256 does cause duplication if x is a vector and has constancy > 1 (e.g., [4, 4, 4, 4] % 256)?

Indeed in this case it would. What I am talking about is something like [0, 1, 2, 3] % 2 -> [0, 1, 0, 1] (this is complementary to [0, 1, 2, 3] // 2 -> [0, 0, 1, 1]). There is clearly some duplication going on here, but I assume it cannot be deduplicated in the same way.

Indeed in this case it would. What I am talking about is something like [0, 1, 2, 3] % 2 -> [0, 1, 0, 1] (this is complementary to [0, 1, 2, 3] // 2 -> [0, 0, 1, 1]). There is clearly some duplication going on here, but I assume it cannot be deduplicated in the same way.

There's no guarantee that [2,3] exist in the same thread as [0,1]. So in your case, maybe it is more of [0, 1, ... 512] % 256? So probably there's no duplication within the same thread?
There has been a long line of architecture research that automatically discovers duplication within and across threads and design function units to address the issues. The uniform register on nvidia GPUs is one way that addresses the duplicate values across threads within the same warp.
From the compiler perspective, I think we could gather more use cases and dive into the problems.

@aakhundov python3 -m pre_commit run --all-files --verbose

@Jokeren Fixed the linter. Could you approve the CI runs again? Thanks!

Do you have any benchmark that demonstrate the speedup? That would be great

Added the "Benchmark" section to the PR description.

Do you have any benchmark that demonstrate the speedup? That would be great

Added the "Benchmark" section to the PR description.

@Jokeren apparently, it's generally unsafe to reuse LLVM Values across different sizePerThread-sized blocks when elemsPerThread is larger. A few unit tests were failing due to this (e.g., test_masked_load), as some essential instructions got wrongly optimized out. I've added an additional condition for that and fixed the failing unit tests that I could verify on A100. Could you approve the CI again? Thanks!

There's no guarantee that [2,3] exist in the same thread as [0,1]. So in your case, maybe it is more of [0, 1, ... 512] % 256? So probably there's no duplication within the same thread?
There has been a long line of architecture research that automatically discovers duplication within and across threads and design function units to address the issues. The uniform register on nvidia GPUs is one way that addresses the duplicate values across threads within the same warp. From the compiler perspective, I think we could gather more use cases and dive into the problems.

My case is actually something like [0, ..., 2^N] // M (where M is not necessarily a power of 2), so there might be duplication in the same thread. I see what you mean however, that's why I was worried that optimising this case would not be as straightforward, but it would still be very useful.

so if I understand the optimization correctly the idea is that:
%10 = or i32 %9, 1, !dbg !11
%25 = sdiv i32 %10, 256, !dbg !13

can be changed to:
%25 = sdiv i32 %9, 256, !dbg !13

In general this is not correct for negative integers however this would be the case for unsigned integers (https://godbolt.org/z/7EqzzKEbM). Maybe the solution is to just cast this to an unsigned integer so that llvm will optimize things out. Technically llvm could also figure it out from the range but this would be more complicated.

so if I understand the optimization correctly the idea is that:
%10 = or i32 %9, 1, !dbg !11
%25 = sdiv i32 %10, 256, !dbg !13

can be changed to:
%25 = sdiv i32 %9, 256, !dbg !13

In general this is not correct for negative integers however this would be the case for unsigned integers (https://godbolt.org/z/7EqzzKEbM). Maybe the solution is to just cast this to an unsigned integer so that llvm will optimize things out. Technically llvm could also figure it out from the range but this would be more complicated.

@ThomasRaoux I don't think that or i32 is completely eliminated. From what I see in the example in the PR description, the following fragment with the 8 instances of each instruction in the original PTX:
  %9 = or i32 %8, %6, !dbg !11
  %10 = or i32 %9, 1, !dbg !11
  %11 = or i32 %9, 2, !dbg !11
  %12 = or i32 %9, 3, !dbg !11
  %13 = or i32 %9, 4, !dbg !11
  %14 = or i32 %9, 5, !dbg !11
  %15 = or i32 %9, 6, !dbg !11
  %16 = or i32 %9, 7, !dbg !11
  %17 = icmp slt i32 %9, %2, !dbg !12
  %18 = icmp slt i32 %10, %2, !dbg !12
  %19 = icmp slt i32 %11, %2, !dbg !12
  %20 = icmp slt i32 %12, %2, !dbg !12
  %21 = icmp slt i32 %13, %2, !dbg !12
  %22 = icmp slt i32 %14, %2, !dbg !12
  %23 = icmp slt i32 %15, %2, !dbg !12
  %24 = icmp slt i32 %16, %2, !dbg !12
  %25 = sdiv i32 %9, 256, !dbg !13
  %26 = sdiv i32 %10, 256, !dbg !13
  %27 = sdiv i32 %11, 256, !dbg !13
  %28 = sdiv i32 %12, 256, !dbg !13
  %29 = sdiv i32 %13, 256, !dbg !13
  %30 = sdiv i32 %14, 256, !dbg !13
  %31 = sdiv i32 %15, 256, !dbg !13
  %32 = sdiv i32 %16, 256, !dbg !13
  %33 = sext i32 %25 to i64, !dbg !14
  %34 = getelementptr half, ptr addrspace(1) %0, i64 %33, !dbg !14
  %35 = sext i32 %26 to i64, !dbg !14
  %36 = getelementptr half, ptr addrspace(1) %0, i64 %35, !dbg !14
  %37 = sext i32 %27 to i64, !dbg !14
  %38 = getelementptr half, ptr addrspace(1) %0, i64 %37, !dbg !14
  %39 = sext i32 %28 to i64, !dbg !14
  %40 = getelementptr half, ptr addrspace(1) %0, i64 %39, !dbg !14
  %41 = sext i32 %29 to i64, !dbg !14
  %42 = getelementptr half, ptr addrspace(1) %0, i64 %41, !dbg !14
  %43 = sext i32 %30 to i64, !dbg !14
  %44 = getelementptr half, ptr addrspace(1) %0, i64 %43, !dbg !14
  %45 = sext i32 %31 to i64, !dbg !14
  %46 = getelementptr half, ptr addrspace(1) %0, i64 %45, !dbg !14
  %47 = sext i32 %32 to i64, !dbg !14
  %48 = getelementptr half, ptr addrspace(1) %0, i64 %47, !dbg !14

gets deduplicated into a single instance of each instruction:
  %9 = or i32 %8, %6, !dbg !11
  %10 = icmp slt i32 %9, %2, !dbg !12
  %11 = sdiv i32 %9, 256, !dbg !13
  %12 = sext i32 %11 to i64, !dbg !14
  %13 = getelementptr half, ptr addrspace(1) %0, i64 %12, !dbg !14

as the result of each of those operations (or the downstream ones) has constancy covering the whole 8 values in the original PTX. Does this make sense?

so if I understand the optimization correctly the idea is that:
%10 = or i32 %9, 1, !dbg !11
%25 = sdiv i32 %10, 256, !dbg !13

can be changed to:
%25 = sdiv i32 %9, 256, !dbg !13

In general this is not correct for negative integers however this would be the case for unsigned integers (https://godbolt.org/z/7EqzzKEbM). Maybe the solution is to just cast this to an unsigned integer so that llvm will optimize things out. Technically llvm could also figure it out from the range but this would be more complicated.

@ThomasRaoux I don't think that or i32 is completely eliminated. From what I see in the example in the PR description, the following fragment with the 8 instances of each instruction in the original PTX:
  %9 = or i32 %8, %6, !dbg !11
  %10 = or i32 %9, 1, !dbg !11
  %11 = or i32 %9, 2, !dbg !11
  %12 = or i32 %9, 3, !dbg !11
  %13 = or i32 %9, 4, !dbg !11
  %14 = or i32 %9, 5, !dbg !11
  %15 = or i32 %9, 6, !dbg !11
  %16 = or i32 %9, 7, !dbg !11
  %17 = icmp slt i32 %9, %2, !dbg !12
  %18 = icmp slt i32 %10, %2, !dbg !12
  %19 = icmp slt i32 %11, %2, !dbg !12
  %20 = icmp slt i32 %12, %2, !dbg !12
  %21 = icmp slt i32 %13, %2, !dbg !12
  %22 = icmp slt i32 %14, %2, !dbg !12
  %23 = icmp slt i32 %15, %2, !dbg !12
  %24 = icmp slt i32 %16, %2, !dbg !12
  %25 = sdiv i32 %9, 256, !dbg !13
  %26 = sdiv i32 %10, 256, !dbg !13
  %27 = sdiv i32 %11, 256, !dbg !13
  %28 = sdiv i32 %12, 256, !dbg !13
  %29 = sdiv i32 %13, 256, !dbg !13
  %30 = sdiv i32 %14, 256, !dbg !13
  %31 = sdiv i32 %15, 256, !dbg !13
  %32 = sdiv i32 %16, 256, !dbg !13
  %33 = sext i32 %25 to i64, !dbg !14
  %34 = getelementptr half, ptr addrspace(1) %0, i64 %33, !dbg !14
  %35 = sext i32 %26 to i64, !dbg !14
  %36 = getelementptr half, ptr addrspace(1) %0, i64 %35, !dbg !14
  %37 = sext i32 %27 to i64, !dbg !14
  %38 = getelementptr half, ptr addrspace(1) %0, i64 %37, !dbg !14
  %39 = sext i32 %28 to i64, !dbg !14
  %40 = getelementptr half, ptr addrspace(1) %0, i64 %39, !dbg !14
  %41 = sext i32 %29 to i64, !dbg !14
  %42 = getelementptr half, ptr addrspace(1) %0, i64 %41, !dbg !14
  %43 = sext i32 %30 to i64, !dbg !14
  %44 = getelementptr half, ptr addrspace(1) %0, i64 %43, !dbg !14
  %45 = sext i32 %31 to i64, !dbg !14
  %46 = getelementptr half, ptr addrspace(1) %0, i64 %45, !dbg !14
  %47 = sext i32 %32 to i64, !dbg !14
  %48 = getelementptr half, ptr addrspace(1) %0, i64 %47, !dbg !14

gets deduplicated into a single instance of each instruction:
  %9 = or i32 %8, %6, !dbg !11
  %10 = icmp slt i32 %9, %2, !dbg !12
  %11 = sdiv i32 %9, 256, !dbg !13
  %12 = sext i32 %11 to i64, !dbg !14
  %13 = getelementptr half, ptr addrspace(1) %0, i64 %12, !dbg !14

as the result of each of those operations (or the downstream ones) has constancy covering the whole 8 values in the original PTX. Does this make sense?

right, this is because you use a sdiv, replacing that with a udiv should make the optimization happen.

I think you could just do xindex = xindex.to(tl.uint32) to force the index to be unsigned

I think that wouldn't deduplicate icmp and or

I think you could just do xindex = xindex.to(tl.uint32) to force the index to be unsigned

I think that wouldn't deduplicate icmp and or

right I missed the icmp part. How do we know that all the cmp will return the same value?

right I missed the icmp part. How do we know that all the cmp will return the same value?

It probably falls into this case and gets optimized by the PR?
        // Case 3: lhs all contiguous, rhs all constant
        // NOTE
        // lhs: 4 5 6 7
        // rhs: 4 4 4 4

I think you could just do xindex = xindex.to(tl.uint32) to force the index to be unsigned

I think that wouldn't deduplicate icmp and or

right I missed the icmp part. How do we know that all the cmp will return the same value?

Generally speaking, the code added by the PR checks the constancy property of the result of the lowered operator. And depending on what the constancy covers does the deduplication. In particular, the result of the cmp is deduplicated if the constancy associated with it (coming from the axis info analysis) allows it. Does this answer your question?

@Jokeren @ThomasRaoux could you please approve the CI? Thanks!

I'm not sure what's going on with the H100 node

I'm not sure what's going on with the H100 node

Perhaps we should try restarting the job?

I'm not sure what's going on with the H100 node

Perhaps we should try restarting the job? Doesn't seem that the current one is going to commence.

It's showing waiting for resources. I've asked nvidia

In case of rank-2+ results, with a non-default axis order (i.e., when the order is not reversed(range(rank))), the flat index mapping during deduplication was incorrect. I've added an axis reordering to fix the behavior in this case.

I'm not sure what's going on with the H100 node

Perhaps we should try restarting the job? Doesn't seem that the current one is going to commence.

It's showing waiting for resources. I've asked nvidia

Seems that the H100 CI job is still pending. There may be something wrong with the runner.

Both our H100 CI an our backup are down for maintenance unfortunately. Bad timing... H100 CI will be back in a couple of days.

@aakhundov I merged main but I think the lit tests need to be adjusted based on https://github.com/openai/triton/pull/2526

@aakhundov I merged main but I think the lit tests need to be adjusted based on #2526

@ptillet Done. the lit tests are passing for me locally now. Could you please approve the CI? Thanks!

Should be good to merge now

@ptillet Do you think we can merge this? Thanks!


命令示例错误

此示例：

提示报错：

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 12a3813d-668f-f3f0-2a10-97eef5cda914
Version Independent ID: 79f91447-90e1-716c-8ae0-ba4b09d356bb
Content: Organization and project-scoped queries - Azure DevOps
Content Source: docs/report/extend-analytics/account-scoped-queries.md
Product: devops
Technology: devops-analytics
GitHub Login: @KathrynEE
Microsoft Alias: kaelli

@carrieyang1212 - I'm closing this issue as there is insufficient information to address the feedback. I just tried the query, and it returned expected results:
https://analytics.dev.azure.com/{OrganizationName}/_odata/v4.0-preview/WorkItems?$filter=Project/ProjectName eq 'Fabrikam Fiber'&$expand=Parent
I've recently refreshed and added several articles under the getting started section available here: Data available from Analytics. Please reference that. If you run into problems, you might get help from our Developer community page.


Updates to JDK11 end of availability

Is your feature request related to a problem? Please describe.
Sorry. More of a question related to version support than a feature request to solve a problem. If there's a better channel for my question, please redirect me.
I'm a product manager whose product use Adoptium JDK11. We're having to plan for a JDK17 upgrade in the next major version of our product based on the current guidance on this page  that says JDK11 may become unavailable in October 2024. However, the "at least" implies that date may be extended, and we're seeing indications that other OpenJDK distros may already be extended until September 2026.
Are there specific plans to extend the JDK11 end of availability date at this time?
How does the linked OpenJDK distro relate to Adoptium, and effect its support dates if at all?

Closing based on @sxa 's response in Slack, https://adoptium.slack.com/archives/C09NLQQAV/p1688399906144419.


R4R cleanup error reporting

closes #242
without debug:
$ rly tx transfer ibc0 ibc0 10000n0token true $(rly keys show ibc1 testkey)
I[2020-05-14|12:08:27.262] ✘ [ibc0]@{43483} - msg(0:transfer) err(sdk:5:failed to execute message; message index: 0: 0transfer/ibconexfer/n0token is smaller than 10000transfer/ibconexfer/n0token: insufficient funds)
Error: failed to send first transaction

with debug:
$ rly tx transfer ibc0 ibc0 10000n0token true $(rly keys show ibc1 testkey) --debug
I[2020-05-14|12:08:24.024] - [ibc0] -> sending transaction:
[{"type":"ibc/transfer/MsgTransfer","value":{"source_port":"transfer","source_channel":"ibconexfer","destination_height":"43478","amount":[{"denom":"transfer/ibconexfer/n0token","amount":"10000"}],"sender":"cosmos1hr348hu9s824ukp7h4v4dcuy28jwy92aa60q8n","receiver":"cosmos1dl9sftjsujla8hcfqj7ppj4hfdmhqx3jjpjvh9"}}]
I[2020-05-14|12:08:24.025] ✘ [ibc0]@{43480} - msg(0:transfer) err(sdk:5:failed to execute message; message index: 0: 0transfer/ibconexfer/n0token is smaller than 10000transfer/ibconexfer/n0token: insufficient funds)
I[2020-05-14|12:08:24.025] - transaction response:
{"height":"43480","txhash":"65B8DA71E05702D28BB6D686FE2201AD5375E36A4306914C732D40EA47981193","codespace":"sdk","code":5,"raw_log":"failed to execute message; message index: 0: 0transfer/ibconexfer/n0token is smaller than 10000transfer/ibconexfer/n0token: insufficient funds","gas_wanted":"200000","gas_used":"51034"}
Error: failed to send first transaction

this updates all regularly logged errors to report in full. While not as clean on the log, it is much more useful in identifying quickly what caused the error. The debug flag releases a ton of information and is not likely to be used in common cases. The current error reporting doesn't provide any contextual information on how the error occurred.
cc: @jackzampolin

@colin-axner See details on the integration testing framework here: https://github.com/iqlusioninc/relayer/blob/master/docs/testing.md Happy to jump on a call an run throught it.


HTTP/1.1 404 Not Found - http://errorprone.info/bugpattern/SimpleDateFormatConstant

It looks like there is HTML page for SimpleDateFormatConstant

We renamed that check in<PHONE_NUMBER>cdb99b90599f61a3b006e5ebfd3769, so the doc tooling removed the markdown.
As a workaround, please refer to http://errorprone.info/bugpattern/DateFormatConstant.
I'll also investigate modifying the doc updater to keep docs for deleted checks.


Delist Firewall setup on fedora as it conflicts with Firewalld.

Type of Change

[x] Refactoring

Description
As Fedora comes with Firewalld preinstalled & preconfigured with rules set for fedora desktop out of the box. Installing ufw on it might conflict with it & is really not needed.
Testing
I have tested it
Impact
Hides firewall setup on fedora systems
Checklist

[x] My code adheres to the coding and style guidelines of the project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation.
[x] My changes generate no errors/warnings/merge conflicts.

This is not a refactor, it's a bugfix. The title also should be modified since my proposed changes make this PR delist firewall setup on any system with firewalld installed, with no relevance to the distro.


06/08/2018 03:10 : Create Pull Request for branch master -> prTesting1_2018060803

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestRef1.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Odd behavior with inclusions & exclusions in pivotui

Hi,
This may be my ineptitude w/i js,  but i' m trying to prepopulate a pivotui.
For the second argument, i'm sending
{"inclusions": {                       "month": ["2017-6", "2017-7", "2017-8", "2017-9", "2017-10", "2017-11", "2017-12", "2018-1", "2018-2", "2018-3", "2018-4", "2018-5"],                     "duration": [                         "Under 1 hour",                         "Over 1 hour but under 24 hours",                         "More than 24 hours"                         ]                     } }
but it comes back in the onRefresh as
  "exclusions": {     "month": [       "2016-01",       "2016-02",       "2016-03",       "2016-04",       "2016-05",       "2016-06",       "2016-07",       "2016-08",       "2016-09",       "2016-10",       "2016-11",       "2016-12",       "2017-01",       "2017-02",       "2017-03",       "2017-04",       "2017-05",       "2017-06",       "2017-07",       "2017-08",       "2017-09",       "2018-01",       "2018-02",       "2018-03",       "2018-04",       "2018-05"     ],     "duration": [       "unknown"     ]   },   "inclusions": {     "month": [       "2017-10",       "2017-11",       "2017-12"     ],     "duration": [       "Under 1 hour",       "Over 1 hour but under 24 hours",       "More than 24 hours"     ]   },
any ideas

One thing to note is that this library is not type-aware: it treats everything as strings. So if you're including 2017-6 it's normal that 2017-06 shows up in exclusions :)

doh,  of  course!
Sorry aboutthat!

No need to apologize, it’s an understandable thing to do and an easy one to miss!


Concurrency configuration

Implement
https://github.com/ublue-os/image-template/pull/38/commits/7864540b70cc6ca0dfb48431aea15bb384b7ca2c

Great idea. I use that for the BlueBuild CLI builds.


Use OpenSSL 3.3

[x] Forked repo is synced with upstream -> github shows no code delta outside of the desired.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork
[x] Tests are passing? https://github.com/microsoft/DeepGNN/blob/main/CONTRIBUTING.md#run-tests
[x] Changelog and documentation updated.
[x] PR is labeled using the label menu on the right side.

Previous Behavior
Relied on BoringSSL that comes with grpc.
New Behavior
Use OpenSSL for encryption.

Use openssl for all os?

We were already using it on windows: https://github.com/microsoft/DeepGNN/blob/a017ebed479c26a563bff89231c527ac418e635c/.bazelrc#L34
This is for other OS’s.


chore(dev): add devcontainer configuration

This makes it easier to develop and run tests against an environment that has the right python virtualenv + a local speckle server + postgres + redis instance/
You can get started in VSCode by using CTRL + SHIFT + P (or something else on Max I guess :upside_down_face: ). You can find more info about VSCode devcontainers and how to use them here.

Bump @izzylys for feedback :smile:

hello! this took forever for me to review and I'm sorry pls don't hate me!!
I actually couldn't get docker working on my machine until v recently (stupid cpu settings that I couldn't figure out how to change) but now I can and I tried this and it's very cool and I like it a lot!!! very nice to be able to run tests without having to spin up a local server
happy to merge - just have a couple quick qs:

is there a reason for py 3.6? if not, I will bump to 3.9
are you on windows? if so, how are you deving from in the container without git EOL problems? lol adding a .gitattributes to force fix this worked fine for me tho so I will prob just pop that in as well 😅

Erm... I use Linux :penguin:  because I'm not a savage :stuck_out_tongue:
No reason for python 3.6, feel free to change it to whatever :smile:
What do you mean by git EOL?

lololol sorry I'm 2 savage
ok sweet, I will update that then ^^
EOL = end of line -- git from within the container was freaking out and said all files have been modified bc of the win vs linux line endings nonsense. but you're not a savage so you don't have the pleasure of experiencing this 😉


Document the invocation format for debug()

New to the package. I've read the README and it's not specified how exactly to call debug. The lead paragraph mentions "console format string goodies" so one might be inclined to call debug('The value of foo is ', foo, ' and that of bar is ', bar), but examples use + to concatenate strings.
Would be nice to document the exact syntax for debug(). Thanks :)

@dandv can you put in a pull request?

Does PR #373 address this or would you like more clarification?


Add configurable wetchedFiles, configurable fmpurl and force push support

This PR adds to @lavere 's feature and assumes that will be tagged 1.1.0


MRU order messed up

The default matcher gives me the most recently used file as first suggestion, then the second most recentely used etc... (given, I didn't type anything yet).
That makes it possible to cycle through most recently used files with ctrl-j/ctrl-k.
I've gotten pretty used to that behaviour as it allows me to quickly change between the few most recently used files.
Any chance to see cpsm support this as well? (Alternatively: Is there a way to tell CtrlP to use a different matcher for the MRU list?)

I've added a new setting g:cpsm_match_empty_query, which you can set to 0 to prevent cpsm from rearranging items for empty queries; however, this will affect non-MRU mode as well, because CtrlP doesn't inform the matcher whether or not it's in MRU mode. If you want to keep cpsm's behavior for empty queries in non-MRU mode, this can be done with a binding that toggles the value of this setting (see README for an example).

Works nicely for MRU, thanks for the blazingly fast reaction :+1: :)
However for non-MRU mode I now get " ERROR: function takes at most 10 arguments (11 given)" as soon as I type the first character.

Another recent commit changed the Python extension, so you have to rebuild it by redoing step 2 in the installation instructions. Sorry for the inconvenience.

Oh no problem, sorry for not noticing that myself :)


Change domain for plausible analytics

This change modifies the plausible analytics domain to toonin.grey.software

@ArsalaBangash would this also resolves #233 ?


Support Parametric Circuits

Summary

Resolve #86

Details and comments

Thank you for this PR @king-p3nguin! Could you please make updates to:

Ensure that the failing checks succeed
Add unit tests for the changes made in the source code

Thank you for the review! After fixing the bug, I ran tests locally, and everything appears to be functioning correctly.


App tries to fetch Meta.json when user is offline

I am using the package with an app that can be accessed when the user is offline and I noticed that when the user is offline, the app tries to load the meta.json file.
I don't want this behaviour when users are offline. I only want to load the file only when the user is online.

hi @Akinmyde, sorry for the delay and thank you for pointing that out. Please install the latest version.

Please re-open if it doesn't work.

It works with version 1.0.11. Thanks, @noahjohn9259


Replace pudl_data script with datastore

There are several places in the documentation and testing that expect a pudl_data script rather than the new datastore.  Get that stuff up to date.

references to pudl_data have been removed from the code & documentation
the documentation correctly describes how to perform etl with the current codebase

Todo:

[ ] Run the etl process via README.rst

Bug ...
From README.rst
$ ferc1_to_sqlite pudl-work/settings/ferc1_to_sqlite_example.yml produces -
`
=================================== FAILURES ===================================
__________________________________ test_mcoe ___________________________________
cat_id = 41696
def grab_fuel_state_monthly(cat_id):
    """
    Grab an API response for monthly fuel costs for one fuel category.

    The data we want from EIA is in monthly, state-level series for each fuel
    type. For each fuel category, there are at least 51 embeded child series.
    This function compiles one fuel type's child categories into one request.
    The resulting api response should contain a list of series responses from
    each state which we can convert into a pandas.DataFrame using
    convert_cost_json_to_df.

    Args:
        cat_id (int): category id for one fuel type. Known to be
    """
    # we are going to compile a string of series ids to put into one request
    series_all = ""
    fuel_level_cat = get_response(make_url_cat_eiaapi(cat_id))
    try:

      for child in fuel_level_cat.json()['category']['childseries']:

E           KeyError: 'category'
.env_pudl/lib/python3.8/site-packages/pudl/output/eia923.py:614: KeyError
During handling of the above exception, another exception occurred:
fast_out = <pudl.output.pudltabl.PudlTabl object at 0x7f8a58852d30>
def test_mcoe(fast_out):
    """Calculate MCOE."""
    logger.info("Calculating MCOE.")

  mcoe_df = fast_out.mcoe()

test/fast_output_test.py:62:

.env_pudl/lib/python3.8/site-packages/pudl/output/pudltabl.py:613: in mcoe
self._dfs['mcoe'] = pudl.analysis.mcoe.mcoe(
.env_pudl/lib/python3.8/site-packages/pudl/analysis/mcoe.py:351: in mcoe
drop_cols = [x for x in pudl_out.gens_eia860().columns
.env_pudl/lib/python3.8/site-packages/pudl/analysis/mcoe.py:352: in
if x in pudl_out.fuel_cost().columns and x not in merge_cols]
.env_pudl/lib/python3.8/site-packages/pudl/output/pudltabl.py:556: in fuel_cost
self._dfs['fuel_cost'] = pudl.analysis.mcoe.fuel_cost(self)
.env_pudl/lib/python3.8/site-packages/pudl/analysis/mcoe.py:174: in fuel_cost
pudl_out.frc_eia923()[['plant_id_eia',
.env_pudl/lib/python3.8/site-packages/pudl/output/pudltabl.py:299: in frc_eia923
pudl.output.eia923.fuel_receipts_costs_eia923(
.env_pudl/lib/python3.8/site-packages/pudl/output/eia923.py:238: in fuel_receipts_costs_eia923
fuel_costs_avg_eiaapi = get_fuel_cost_avg_eiaapi(
.env_pudl/lib/python3.8/site-packages/pudl/output/eia923.py:686: in get_fuel_cost_avg_eiaapi
grab_fuel_state_monthly(fuel_cat_id)))

cat_id = 41696
def grab_fuel_state_monthly(cat_id):
    """
    Grab an API response for monthly fuel costs for one fuel category.

    The data we want from EIA is in monthly, state-level series for each fuel
    type. For each fuel category, there are at least 51 embeded child series.
    This function compiles one fuel type's child categories into one request.
    The resulting api response should contain a list of series responses from
    each state which we can convert into a pandas.DataFrame using
    convert_cost_json_to_df.

    Args:
        cat_id (int): category id for one fuel type. Known to be
    """
    # we are going to compile a string of series ids to put into one request
    series_all = ""
    fuel_level_cat = get_response(make_url_cat_eiaapi(cat_id))
    try:
        for child in fuel_level_cat.json()['category']['childseries']:
            # get only the monthly... the f in the childseries seems to refer
            # the recporting to frequency
            if child['f'] == 'M':
                logger.debug(f"    {child['series_id']}")
                series_all = series_all + ";" + str(child['series_id'])

    except KeyError:

      raise AssertionError(

            f"Error in Response: {fuel_level_cat.json()['data']['error']}")

E           AssertionError: Error in Response: invalid or missing api_key. For key registration, documentation, and examples see https://www.eia.gov/developer/
.env_pudl/lib/python3.8/site-packages/pudl/output/eia923.py:622: AssertionError
`
I think the default settings file is not in line with the required key.

$ pudl_etl /vagrant/pudl_work/settings/etl_example.yml
2020-07-06 21:09:51 [    INFO] pudl:84 verifying that the data we need exists in the data store
Traceback (most recent call last):
  File "/home/vagrant/miniconda3/envs/pudl-dev/bin/pudl_etl", line 33, in <module>
    sys.exit(load_entry_point('catalystcoop.pudl', 'console_scripts', 'pudl_etl')())
  File "/home/vagrant/git/pudl/src/pudl/cli.py", line 87, in main
    pudl.helpers.verify_input_files(ferc1_years=flattened_params_dict['ferc1_years'],
AttributeError: module 'pudl.helpers' has no attribute 'verify_input_files'

Hmm, do you have a valid EIA API key stored in the  API_KEY_EIA environment variable? We should add a check with some useful error message in there.


Cosmos SDK v0.47

Continuation of https://github.com/strangelove-ventures/ibctest/pull/337
Working branch for update to SDK v0.47

regarding the failure for the TestCosmosChain_BroadcastTx, I just checked out main, and changed the broadcast mode here to broadcast sync, and it failed. This implies that the current broadcasting code fails with BroadcastSync and these changes may not be breaking something that wasn't already failing.
cc @agouin

I opened a pr on the go relayer to bump to v0.47, unsure if it is related to issues faced here, but it seems the tests cannot get past ConnectionOpenInit. Hopefully those tests can help with debugging the remaining issues

I think we found the issue @agouin, in our genesis modification function,  we create a genesis doc object. The fields have changed in this type and so we are overwriting the genesis file that was created by ibctest with a new unknown format for chains using an older sdk.

Hi @agouin, I tested this branch with out newly updated 0.47 chain and I think there is an issue with some of the genesis commands that have been moved if you use the new genutil setup (which we do) from this: https://github.com/cosmos/cosmos-sdk/pull/14149
Meaning, things like add-genesis-account fails because it is now under genesis add-genesis-account.

I hacked a solution for myself to get past this blocker here: https://github.com/gjermundgaraba/ibctest/commit/28412748e293eec207f43c31d06248c7e266b905
Probably not a good way to solve this problem, but if anyone finds it useful 🤷

This seems like a good first step in making this possible. We can revisit to try and add a detection mechanism in a follow up issue. Would you like to open a PR with your change?

Another issue which I think is a blocker for me is the relayer itself which seems to be very unsupportive of either the SDK version or tendermint version I think. Just a heads up if anyone else finds this issue :)

https://github.com/cosmos/relayer/pull/1068 this branch should unblock you with the relayer. It has a detection mechanism for the tendermint version so it can relay before/after v0.37

@agouin I opened a PR against the v0.47 for this change. If you prefer I can try to move the changes to main and open the PR against it instead.

There was an issue with the broadcast messages functionality, I created a fix for it here #388

Hi, just curious what the status of this pr is? Is there more work to be done to get it merged or is it waiting on a final SDK version? Thanks :)

Hi, just curious what the status of this pr is? Is there more work to be done to get it merged or is it waiting on a final SDK version? Thanks :)

We are preparing to merge this to main asap. Remaining items are to rebase this on top of main's latest changes, branch v6, then this can be merged.

Just wanted to say thank you to everyone who worked on this! Being able to run e2e's has been a big improvement for us, so we really appreciate all maintence that goes into maintaining interchaintest :pray:


Sporadically stops ignoring own aircraft

Stratux version: 1.6r1-eu027

Stratux config:
SDR

[ ] single
[x] dual

GPS

[x] yes
[ ] no
type:  VK-162 G-Mouse

AHRS

[x] yes
[ ] no

power source: 25,800mAh power brick
usb cable: Stratux Serial Adapter CP2102

EFB app and version: AvMap EKP-V 1.13.101R
EFB platform: EKP-V
EFB hardware: EKP-V connected via RS-232 @ 38,400 baud using the FLARM protocol strings

Description of your issue:  All is normal until about 30 - 45 minutes after power on. Then Stratux STOPS ignoring my own aircraft and show it as traffic. I have my HEX code properly inputted in the settings. Sometimes, after a few minutes, it will START ignoring my own aircraft again. Other times it continues to display my plane as traffic right on top of me until I power cycle the Stratux.

I'll gather the logs the next time I go to the hangar.
If possible, enable "Replay Logs", reproduce the problem, and provide a copy of the logs in http://<IP_ADDRESS>/logs/stratux/ and http://<IP_ADDRESS>/logs/stratux.log.

Yes, please provide the log files. It will be logged if the ownship config ignoring is intentional and why

I didn't realize that those logs are disabled by default. I've enabled them and I'll upload them next time I fly as it has happened every time I fly. Might not be until the end of the week, but stay tuned .....

 I've turned them on in Settings, but they don't appear anywhere. I've tried using Persistent Logs, but that didn't seem to do anything either. stratux.log exists and is viewable but http://<IP_ADDRESS>/logs/stratux/ returns a 404 error.

Persistent logging needs to be on, yes. Then they are under /logs iirc. The file I need is indeed stratux.log.

I'm in the US, Arizona specifically. The EKP-V only understands FLARM when it comes to traffic. (I'm a beta tester for them) I've got 868 disabled, 978 & 1090 enabled. So far seems to be working pretty well. The only real issue I've seen is my "shadow plane" issue.
As for the logs, I've got persistent turned on, replay turned on. But when I go into logs, all I see are:
auth.log
btmp
daemon.log
fancontrol.err
fancontrol.log
kern.log
lastlog
messages
private/
stratux.log
syslog
user.log
wtmp
Where's the replay log?

Yes, I have ADS-B In/Out: uAvionix Echo.

I've got a bunch of logs, and it did do it again. How do I upload them? I don't see a way to upload.


Clean up hicfft

This PR is removing dead code from hicfft we have identified last week.
These are various functions from the old APIs.

Nice tidy-up !


build: use syntax that works fine on bash or dash

On make doc rules
Signed-off-by: Bruno Dilly<EMAIL_ADDRESS>

and our build on semaphoreci is good again :smile:

+1 :D


Record each user stories as an issue

Format:

Need to add type.Story and priority.* labels to these issues.

Added 10 user stories which are relevant for v1.2


Fix 'recheck' regexes to be more strict

Currently, pr-comment can be triggered by any comment containing our
recheck term.
Fixes BonnyCI/projman#101
Signed-off-by: Bradon Kanyid<EMAIL_ADDRESS>

Fixes hoist/projman#101

gate-recheck

Starting gate_github jobs.


Remove forced angle from teleporting

https://github.com/tf2cheater2013/Fedoraware/commit/40b94aed2c5b2cb164715c91201285439bc20c5d#commitcomment-74043578
When used in combination with Camera: Teleporter and RemoveScreenEffects, you could use this in order to kill anybody camping a teleporter. Or at least instantly start walking the direction you want instead of running into a wall.
https://user-images.githubusercontent.com/17802843/169581484-91c48648-3eec-4695-9755-ea26fd0e2c71.mp4

This is actually turning out to be something very interesting.

Uh not really... There is angles and placement of the teleport

Uh not really... There is angles and placement of the teleport

I don't understand what you are saying. Someone implementing a feature that would close this issue would solve angles. Placement is out of our hands.
The placement really isn't a big deal. If the placement is bad, the exit teleporter will just get destroyed. So an exit teleporter that still exists therefore has good placement.
The problem is the angles. Sometimes the angle of the teleporter points you straight into a wall- Or, in this case, literally anywhere other than the enemy. This suggestion fixes this problem by allowing you to force your own angle.

I'll have a look how the game is forcing the viewangle.

Is it not possible to paste it from ncc
On Thu, May 26, 2022, 11:15 AM lnx00 <
@.***> wrote:

https://github.com/xxmyn/TeamFortress2/blob/1b81dded673d49adebf4d0958e52236ecc28a956/tf2_src/game/server/tf/tf_obj_teleporter.cpp#L1143
Saw that too. No clue how we could prevent it though. Teleport w
DuckDuckGo removed 1 tracker.
More →
https://duckduckgo.com/email/report#RFVDSzI.G0gBoCwK7Abn_oxEmCbKr4ilyNYeiNtSXyjH0DRagkHhXMH032ix6ETf4DnyAy6584t1J3ow_f7v-36qRW3NA03SCLMSDi29YSmOW0Gz9UtDtalT1xDVTu63BF0Y2QhaBQIyReCQntWD68RENwC7PL_08-lUuMryEH18J-0SABt8QlCtBnub1-EQ_COpnTMssh4J7ya35eBeWPO_d4BzRQzsQFL_XR2D4J9-vgmMOf3H-jBlxnuDF0vpdY9niquVCHi5fkFOo57MByloVoKlJU5HZfD9Dw
https://github.com/xxmyn/TeamFortress2/blob/1b81dded673d49adebf4d0958e52236ecc28a956/tf2_src/game/server/tf/tf_obj_teleporter.cpp#L1143
Saw that too. No clue how we could prevent it though. Teleport will set
the angle through the m_angRotation prop. So we would need to ignore it
on the client side...
—
Reply to this email directly, view it on GitHub
https://github.com/tf2cheater2013/Fedoraware/issues/302#issuecomment-1138745711,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AY623Q5IUJXSDRIDU32GZRDVL6PRBANCNFSM5WQDWHBQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

does nc even have it?

https://github.com/xxmyn/TeamFortress2/blob/1b81dded673d49adebf4d0958e52236ecc28a956/tf2_src/game/server/tf/tf_obj_teleporter.cpp#L1143

Saw that too. No clue how we could prevent it though. Teleport will set the angle through the m_angRotation prop. So we would need to ignore/block it on the client side...

If u look at the https://github.com/xxmyn/TeamFortress2/blob/1b81dded673d49adebf4d0958e52236ecc28a956/tf2_src/game/server/baseentity.cpp#L4434, you see it only does SnapEyeAngles it if it's it's a player,
void CBasePlayer::SnapEyeAngles( const QAngle &viewAngles )
{
	pl.v_angle = viewAngles;
	pl.fixangle = FIXANGLE_ABSOLUTE;
}

sets CPlayerState::fixangle and I think it is only used to set the player's viewangles in https://github.com/xxmyn/TeamFortress2/blob/1b81dded673d49adebf4d0958e52236ecc28a956/tf2_src/engine/servermsghandler.cpp#L669, so you can probably do something to stop that from running

game event "base_player_teleported" occurs when you teleport no matter what ping, listen for the event and when it happens, hold your angles at whatever they were before?

Added