earnings_date failing (KeyError: 'Earnings Date')

Describe bug
The earnings_dates record is not available
Simple code that reproduces your problem
import yfinance as yf
msft = yf.Ticker('MSFT')
msft.earnings_dates
Debug log

Bad data proof
No response
yfinance version
yfinance 0.2.38
Python version
python3.10
Operating system
ubuntu

Yahoo has changed table structure
                                  Event Name  Earnings Call Time  EPS Estimate  Reported EPS  Surprise(%)
0                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
1                      Q1 2024 Earnings Call                 TAS           NaN           NaN          NaN
2                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
3   at Bank of America Healthcare Conference                 TAS           NaN           NaN          NaN
4                Annual Shareholders Meeting                 TAS          1.95          1.59      -0.1846
5                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
6                   Q3 2023 Earnings Release  After Market Close           NaN           NaN          NaN
7                   Q4 2023 Earnings Release  After Market Close           NaN           NaN          NaN
8   at Bank of America Healthcare Conference                 TAS         -0.74           NaN          NaN
9                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
10               Annual Shareholders Meeting   Time Not Supplied           NaN           NaN          NaN
11                  Q4 2023 Earnings Release  After Market Close           NaN           NaN          NaN

Event name is nice, but losing dates a problem. Hoping for a way to get old structure.

Looks like yahoo fixed their problem. yfinance works fine now. I suggest you close this issue.

I suspect it will return, I've seen this alternative table randomly rarely over last few weeks. Yahoo is up to something...

For some tickers, it works and some others it doesn't. At this time, ENB.TO works, TSLA still gives the issue.

It returned to normal, interesting!

Looks like Yahoo finance's earnings tab blew up again.

This works fine now, but it's also pulling up non-quarterly earnings.
Depending on how Yahoo Finance plans to move forward with the earnings table, can we please add an option in the future to filter out the non-quarterly earnings?
Right now, we can tell which ones are quarterly by the Company column:
https://finance.yahoo.com/calendar/earnings?symbol=MCD
However, this column is omitted in the get_earnings function.

Keeping column Company is fine, but filtering rows should be optional in case algorithm has bug.

Having the issue starting from this week.

when I tried to get the earnings of AI. https://finance.yahoo.com/calendar/earnings/?symbol=AI, yahoo may need some time to run js. I also tried selenium. It works but much slower.

I encountered this recently. I think what happens is, when Yahoo is overloaded they use a second data source which returns a completely different table structure.

Did the table structure blow up again. I'm getting an error

Even I'm facing the same issue.
Enter the company's ticker symbol: AMD
Traceback (most recent call last):
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexes\base.py", line 3805, in get_loc
return self._engine.get_loc(casted_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc
File "index.pyx", line 196, in pandas._libs.index.IndexEngine.get_loc
File "pandas\_libs\hashtable_class_helper.pxi", line 7081, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas\_libs\hashtable_class_helper.pxi", line 7089, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Earnings Date'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Revanth\Desktop\Final Project\Earnigns_date\earnings.py", line 57, in
earnings_dates = get_earnings_dates(ticker)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\Desktop\Final Project\Earnigns_date\earnings.py", line 28, in get_earnings_dates
earnings_dates = stock.get_earnings_dates(limit=10)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\utils.py", line 104, in wrapper
result = func(args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\base.py", line 634, in get_earnings_dates
tzinfo = dates[cn].str.extract('([AP]M[a-zA-Z])$')
~~~~~^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\frame.py", line 4090, in getitem
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexes\base.py", line 3812, in get_loc
raise KeyError(key) from err
KeyError: 'Earnings Date'

Same, does anyone know what's going on this time?

The data return by python request doesn't execute the filter that you would have on a browser:
https://finance.yahoo.com/calendar/earnings?symbol=GOOGL&offset=0&size=12
[   Symbol                       Company                                         Event Name  Earnings Call Time EPS Estimate Reported EPS Surprise(%)
0   FORTY      Formula Systems 1985 Ltd                           Q3 2024 Earnings Release  Before Market Open            -            -           -
1   WLMSQ                    NOS4-1 Inc  Q4 2023 Williams Industrial Services Group Inc...

This is why "Earnings Date" is not there. The javascript needs to be executed.

The data return by python request doesn't execute the filter that you would have on a browser: https://finance.yahoo.com/calendar/earnings?symbol=GOOGL&offset=0&size=12
[   Symbol                       Company                                         Event Name  Earnings Call Time EPS Estimate Reported EPS Surprise(%)
0   FORTY      Formula Systems 1985 Ltd                           Q3 2024 Earnings Release  Before Market Open            -            -           -
1   WLMSQ                    NOS4-1 Inc  Q4 2023 Williams Industrial Services Group Inc...

This is why "Earnings Date" is not there. The javascript needs to be executed.

Ah I'm also having this issue, tried using selenium/pyppeteer and it just recently broke like two days ago and it also gives that generic response

This is why "Earnings Date" is not there. The javascript needs to be executed.

@michaelmoopenn The table you quoted is the new-style, it is missing "Earnings Date" column. The link has the new style. Can you post a screenshot of successfully fetching the old-style?

This is why "Earnings Date" is not there. The javascript needs to be executed.

@michaelmoopenn The table you quoted is the new format, it is missing "Earnings Date" column. Your link shows new format. Can you post a screenshot of successfully fetching the old format?

If you go on the link with your browser you should see this:

Which is the correct old format. However if you click on "Clear" which clear the filter you get what python request get:

The issue is that python request don't get the same view as with your browser even though there is the filter attributes in the get parameters.

So, is this gonna stay permanently returning error for EU users ?

I'm in the US. cTicker.earnings_dates()  isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

If you go on the link with your browser you should see this:

I don't see that. It's not adblock etc, I turned that off. Maybe it's geo-specific - I'm Europe.

The filter works only when you are logged in.

The filter works only when you are logged in.

You're right. So probably yfinance needs to support logging in.

I just tried logging into yahoo and it showed me the normal earnings dates, so that might be a good solution

I just tried logging into yahoo and it showed me the normal earnings dates, so that might be a good solution

True, when logged in to Yahoo is works fine.
I also found that replacing:
dates = stock.earnings_date
with:
date = stock.calendar['Earnings Date'][0]
worked for me.

I'm in the US. cTicker.earnings_dates() isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

Hello Jim, When I try it, I don't get all the previous earnings (I just get the last 4 ones and tried for many different tickers ...)
Did I miss something ?

Hi David,
for a stock like 'F'  when I do:
self.cTicker.earnings_dates()
I get: builtins.KeyError: 'Earnings Date'
(this had worked a few weeks ago without issue; so it seemed yahoo changed something)
whereas if I used earnings_history() I get:
self.cTicker.get_earnings_history()
epsEstimate  epsActual  epsDifference  surprisePercent
2023-12-31      0.55761       0.54          -0.02          -0.0316
2024-03-31      0.53056       0.55           0.02           0.0366
2024-06-30      0.57366       0.57          -0.00          -0.0064
2024-09-30      0.57103       0.60           0.03           0.0507
I do see (as shown by others above) that if am not logged into yahoo then when I go to the earnings page
https://finance.yahoo.com/calendar/earnings?symbol=F&offset=0&size=12
I do not get just the earnings for F but a list of all the earnings for "today"..
when I am logged in to yahoo that url shows me the upcoming earnings as well as several of the past ones..
Is it that I need to somehow login to yahoo thru yfin?
tks
-jim

I'm in the US. cTicker.earnings_dates() isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

Hello Jim, When I try it, I don't get all the previous earnings (I just get the last 4 ones and tried for many different tickers ...) Did I miss something ?

For anyone struggling to extract the earnings from yfinance right now, I have this little snippet that got me out of trouble. It's not a drop-in replacement for yfinance, but those, like me, that needed to get the historical earnings to another platform, this might save you some trouble while this gets fixed.

Navigate to this link: https://finance.yahoo.com/calendar/earnings?symbol=GOOG&offset=0&size=100
Sign in to Yahoo with your own account (top right corner), you now should see the last 100 earnings for Google. Either change the filter manually to select another symbol or change the symbol parameter in the URL
Open your browser's console and run this snippet, which will download the visible table as a CSV:

const earnings_json = $$('#fin-cal-table tbody tr')
	.map(tr => ({
		symbol: $('td:nth-child(1)', tr).innerText,
		date: new Date($('td:nth-child(3) span:nth-child(1)', tr).innerText.substring(0, 12)),
	}))
	.filter(e => new Date() > e.date)
	.map(e => ({
		...e,
		date: e.date.toISOString().split('T')[0],
	}))

const rows = [
	["symbol", "date"],
	...earnings_json.map(e => [e.symbol, e.date])
]

let csvContent = "data:text/csv;charset=utf-8," + rows.map(r => r.join(",")).join("\n")

var encodedUri = encodeURI(csvContent)
var link = document.createElement("a")
link.setAttribute("href", encodedUri)
link.setAttribute("download", `earnings_${earnings_json[0].symbol}.csv`)
link.click()

You can remove the filter() line if you don't want to filter future earning dates. Or, change the direction of that comparison if you only want future earnings (for me, I only want past ones).
Hope this helps.

Thank you for your answers, login in does change the data displayed as well for me (I am in EU). I guess that indeed a login method in the class YfData along with the cookies handling would be helpful if possible ?

I just tried logging into yahoo and it showed me the normal earnings dates, so that might be a good solution

True, when logged in to Yahoo is works fine. I also found that replacing: dates = stock.earnings_date with: date = stock.calendar['Earnings Date'][0] worked for me.

@szapiro - your sample somehow doesn't work for example on AAPL.
I get the following dates:
'Earnings Date': [
datetime.date(2025, 1, 30), datetime.date(2025, 2, 3)
]
There are 4 dates in 2025 listed and only 1 matches the above.
https://finance.yahoo.com/calendar/earnings?symbol=AAPL&offset=0&size=12

Bad news: Yahoo is changing their layout again.
Good news: there's a new URL that displays all earnings dates, hopefully a quick change for a fix.
Old URL: https://finance.yahoo.com/calendar/earnings?symbol=GOOG&offset=0&size=100
New URL: https://finance.yahoo.com/calendar/earnings?day=2024-12-04&symbol=GOOG&offset=0&size=100
(Adding the day parameter gets a different layout, that displays the whole list of earning dates.)

Like @AlphaGit said, Yahoo changed their API route and their table formatting.
To fix this bug, make these two changes in base.py :

Load the datetime package in base.py :
from datetime import date

Replace these lines (576 to 661) in base.py:
 logger = utils.get_yf_logger()

 page_size = min(limit, 100)  # YF caps at 100, don't go higher
 page_offset = 0
 dates = None
 while True:
     url = f"{_ROOT_URL_}/calendar/earnings?day={date.today()}&symbol={self.ticker}&offset={page_offset}&size={page_size}"
     data = self._data.cache_get(url=url, proxy=proxy).text

     if "Will be right back" in data:
         raise RuntimeError("*** YAHOO! FINANCE IS CURRENTLY DOWN! ***\n"
                            "Our engineers are working quickly to resolve "
                            "the issue. Thank you for your patience.")

     try:
         data = pd.read_html(StringIO(data))[0]
     except ValueError:
         if page_offset == 0:
             # Should not fail on first page
             if "Showing Earnings for:" in data:
                 # Actually YF was successful, problem is company doesn't have earnings history
                 dates = utils.empty_earnings_dates_df()
         break
     if dates is None:
         dates = data
     else:
         dates = pd.concat([dates, data], axis=0)

     page_offset += page_size
     # got less data then we asked for or already fetched all we requested, no need to fetch more pages
     if len(data) < page_size or len(dates) >= limit:
         dates = dates.iloc[:limit]
         break
     else:
         # do not fetch more than needed next time
         page_size = min(limit - len(dates), page_size)

 if dates is None or dates.shape[0] == 0:
     _exception = YFEarningsDateMissing(self.ticker)
     err_msg = str(_exception)
     logger.error(f'{self.ticker}: {err_msg}')
     return None
 dates = dates.reset_index(drop=True)

 # Drop redundant columns
 dates = dates.drop(["Symbol", "Company"], axis=1)

 # Convert types
 for cn in ["EPS Estimate", "Reported EPS", "Surprise (%)"]:
     dates.loc[dates[cn] == '-', cn] = float("nan")
     dates[cn] = dates[cn].astype(float)

 # Convert % to range 0->1:
 dates["Surprise (%)"] *= 0.01

 # Parse earnings date string
 cn = "Earnings Date"
 dates[cn] = dates[cn].astype(str)

 # - remove "at"
 dates[cn] = dates[cn].replace(' at', ',', regex=True)

 # - remove EDT/EST
 dates[cn] = dates[cn].replace(' E[DS]T', '', regex=True)

 # - remove AM/PM and timezone from date string
 tzinfo = dates[cn].str.extract('([AP]M[a-zA-Z]*)$')
 dates[cn] = dates[cn].replace(' [AP]M[a-zA-Z]*$', '', regex=True)

 # - split AM/PM from timezone
 tzinfo = tzinfo[0].str.extract('([AP]M)([a-zA-Z]*)', expand=True)
 tzinfo.columns = ["AM/PM", "TZ"]

 # - combine and parse
 dates[cn] = dates[cn] + ' ' + tzinfo["AM/PM"]
 dates[cn] = pd.to_datetime(dates[cn], format="%B %d, %Y, %I %p")
 # - instead of attempting decoding of ambiguous timezone abbreviation, just use 'info':
 self._quote.proxy = proxy or self.proxy
 tz = self._get_ticker_tz(proxy=proxy, timeout=30)
 dates[cn] = dates[cn].dt.tz_localize(tz)

 dates = dates.set_index("Earnings Date")

 self._earnings_dates[limit] = dates

 return dates

I'm not a contributor, so if you are one and you're reading this, feel free to commit this change and link this issue with your commit. As a side note, this fix only supports Eastern U.S. Time (EDT/EST) earnings dates for now.

Great for the new URL!
Quick notice, after looking closer to the earnings dates, the dates are either 1 day behind or 1 day ahead. Data is inacurate.
I guess that they are still working on it...

Great for the new URL! Quick notice, after looking closer to the earnings dates, the dates are either 1 day behind or 1 day ahead. Data is inacurate. I guess that they are still working on it...

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few example across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

There are systematic errors across many different tickers on the Yahoo Finance website. Yahoo Finance devs must have changed something somewhere and didn't realize they broke anything. They should have done regression tests and they didn't.
In the meantime, until they fix the error, I would suggest finding a source of data outside of Yahoo Finance (if you really need the data).

On released earnings they looks to display: accurate earnings date - 1 day.

Do you think it is because all date is previously in UTC format? And then that's the reason why it is -1 day and also that's why  they have difference for EST and EDT?

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

What's your reference for dates, I can't find a good second source to refence that includes the times.

this link is broken ?
https://finance.yahoo.com/calendar/earnings?day=2024-12-04&symbol=GOOG&offset=0&size=100

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

What's your reference for dates, I can't find a good second source to refence that includes the times.

I get them from Zack's. Here's a link to  Agilent's earnings calendar for example. If you cross reference with the company's investor relations page on their website, you'll see that when it comes to past earnings dates, Yahoo is wrong whereas Zacks is correct. On the other hand, when it comes to future earnings data, Yahoo is correct whereas Zacks is wrong.
So I get my past earnings data from Zacks but my future earnings dates from Yahoo, as a workaround until Yahoo correct their bug.

I'm in the US. cTicker.earnings_dates() isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

I am not in the US, and this worked for me. I did:
ticker = yf.Ticker(ticker_selected)
earnings_dates = ticker.earnings_history
I can see a table with the information regarding the last 4 quarters

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

What's your reference for dates, I can't find a good second source to refence that includes the times.

I get them from Zack's. Here's a link to Agilent's earnings calendar from Zacks for example. If you cross reference with the company's investor relations page on their website, you'll see that when it comes to past earnings dates, Yahoo is wrong whereas Zacks is correct. On the other hand, when it comes to future earnings data, Yahoo is correct whereas Zacks is wrong.
So I get my past earnings data from Zacks but my future earnings dates from Yahoo, as a workaround until Yahoo correct their bug.

Oh geez what a mess yahoo have made smh, thank you for replying!

Yahoo's data was fine, yfinance was messing up timezone.


Improve test coverage

Add some tests to improve the global coverage

Coverage increased (+3.9%) to 69.196% when pulling 62d1dd0087c6254d0aaecff8db9d51e3dea0f4cb on gabrielseco:improveTestCoverage into 811a3803744236455b561af99b80faaec5aa07b9 on CachopoTeam:master.


Added Copy Feature

Added Copy option for elements instead of move, as per issue #265
sortable('.o-sortable', {
copy: true
}

Coverage increased (+0.4%) to 84.971% when pulling d1c92eab3593162c5bbb34491df56fb6d0259b39 on mfriesen:copy into 4ca889f56c733ca505fc2e40ab0b49274fc86571 on lukasoppermann:master.

Hey @mfriesen, thanks a lot, this looks pretty good. I will have a closer look later today or tomorrow (traveling today). Could you do me a favor and rebase to master? (I did some minor tweaks to the readme after I merged your PR).

I've rebased from master.

@lukasoppermann I should have made all the fixes.


fix: Add unit tests for async tag

Description
Fixes #12826
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Checked that this test case fails before #12684
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=6d4d0b1


Remove numbers in headings of store validation guidelines

Docs Build status updates of commit 6bb8895:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/concepts/deploy-and-publish/appsource/prepare/submission-checklist.md
:white_check_mark:Succeeded
View

msteams-platform/concepts/deploy-and-publish/appsource/prepare/teams-store-validation-guidelines.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


[BUG] GET error for volume attachment on node reboot

Describe the bug
After a reboot of a node in a 4 node cluster a user is seeing the following:
Warning  FailedMount  48s (x3 over 4m52s)   kubelet            MountVolume.WaitForAttach failed for volume "pvc-7d2e2124-4b0c-4d79-890a-fcee02a185a1" : volume pvc-7d2e2124-4b0c-4d79-890a-fcee02a185a1 has GET error for volume attachment csi-b21170ee9729a55ec3e64e6bd4ed0a11ac70ac2272e0e3b7bb3f6fdeac262172: volumeattachments.storage.k8s.io "csi-b21170ee9729a55ec3e64e6bd4ed0a11ac70ac2272e0e3b7bb3f6fdeac262172" not found

To recover, the user had to create the volumeattachment object manually for the Pod to mount its storage again
To Reproduce
I have not been able to reproduce this yet unfortunately
Expected behavior
A pod can successfully mount its storage despite a node reboot in the cluster
Log or Support bundle
longhorn-support-bundle_a8118729-480f-4d38-9b91-26a755d2e0cc_2022-06-28T20-34-47Z.zip
Environment

Longhorn version: 1.1.2
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): kubectl(kURL addon - https://kurl.sh/docs/add-ons/longhorn)
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version:  https://kurl.sh/docs/install-with-kurl/

Number of management node in the cluster: 3
Number of worker node in the cluster: 1

Node config

OS type and version: Red Hat Enterprise Linux Server 7.9 (Maipo)
CPU per node: 8
Memory per node: 64GB
Disk type(e.g. SSD/NVMe): (Unsure, but can gather this info if needed)
Network bandwidth between the nodes: (Unsure, but can gather this info if needed)

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): (Unsure, but can gather this info if needed)
Number of Longhorn volumes in the cluster: 5

Hi @diamonwiggins, ref:issues-2629
Did it 'FailedMount' happen for a long time before you created the volumeattachment object?

It could take up to 6 or 7 minutes for Kubernetes to retry creating the volumeattachment object.
How long did the node go down?

@PhanLe1010 The node went down for only minutes. Maybe 5 minutes or so. However it was a full 24 hours before the user manually created the VolumeAttachment objects.

Is there any additional information I can gather to assist here?

@diamonwiggins I don't seem to fingure out why the VA is deleted and never be created automatically as you mentioned earlier that the VA removal feature was removed since Longhorn 1.1.2.
I would suggest upgrading to a newer stable Longhorn version (1.1.3 or 1.2.5) and report back when you hit the issue again

@PhanLe1010 Understood. We've seen a similar issue with another customer after a reboot. The error is slightly different this time with:

MountVolume.WaitForAttach failed for volume "pvc-e25ec426-043d-496d-9ddd-e4920e8c1096" : volume pvc-e25ec426-043d-496d-9ddd-e4920e8c1096 has GET error for volume attachment csi-845807a0d4e3617baaadf26f975d24db606458cb640455aaac527298e9a2c4bd: volumeattachments.storage.k8s.io "csi-845807a0d4e3617baaadf26f975d24db606458cb640455aaac527298e9a2c4bd" is forbidden: User "system:node:ip-10-0-1-200" cannot get resource "volumeattachments" in API group "storage.k8s.io" at the cluster scope: no relationship found between node 'ip-10-0-1-200' and this object

We've confirmed that node names and IP address had not changed, and our customer was able to reproduce this on two separate environments.
longhorn-support-bundle_b27f4748-ec4a-45a1-8d75-04fe278d3584_2022-09-07T18-57-23Z (1).zip
If this warrants a separate github issue let me know and i'll get one opened up.
Environment

Longhorn version: 1.3.1
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): kubectl(kURL addon - https://kurl.sh/docs/add-ons/longhorn)
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version:  https://kurl.sh/docs/install-with-kurl/

Number of management node in the cluster: 1
Number of worker node in the cluster: n/a

Node config

OS type and version: Ubuntu 18.04.6 LTS
CPU per node: 16
Memory per node: 64GB
Disk type(e.g. SSD/NVMe): (Unsure, but can gather this info if needed)
Network bandwidth between the nodes: (Unsure, but can gather this info if needed)

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): AWS
Number of Longhorn volumes in the cluster: 7

cc @joshimoo

we see this issue so often and today also we had a repro where on node restart alertmanager statefulset related pods were not able to mount PVCs
       {
            "apiVersion": "v1",
            "count": 60,
            "eventTime": null,
            "firstTimestamp": "2022-10-12T06:05:07Z",
            "involvedObject": {
                "apiVersion": "v1",
                "kind": "Pod",
                "name": "alertmanager-rancher-monitoring-alertmanager-1",
                "namespace": "cattle-monitoring-system",
                "resourceVersion": "83537",
                "uid": "2088fca6-b6cb-458f-8297-44fa477b0e81"
            },
            "kind": "Event",
            "lastTimestamp": "2022-10-12T07:50:59Z",
            "message": "MountVolume.WaitForAttach failed for volume \"pvc-84933541-a66d-4ca2-a710-6db17e6643ba\" : volume pvc-84933541-a66d-4ca2-a710-6db17e6643ba has GET error for volume attachment csi-0c400de43ff27c65fa12afab1248675317dbb2b8fc07ae6582df5ce218fa6ff7: volumeattachments.storage.k8s.io \"csi-0c400de43ff27c65fa12afab1248675317dbb2b8fc07ae6582df5ce218fa6ff7\" is forbidden: User \"system:node:server1\" cannot get resource \"volumeattachments\" in API group \"storage.k8s.io\" at the cluster scope: no relationship found between node 'server1' and this object",
            "metadata": {
                "creationTimestamp": "2022-10-12T06:05:07Z",
                "name": "alertmanager-rancher-monitoring-alertmanager-1.171d3d2e89354c2e",
                "namespace": "cattle-monitoring-system",
                "resourceVersion": "167371",
                "uid": "1874d80b-43e9-4242-9df3-bc39b68c0cc1"
            },
            "reason": "FailedMount",
            "reportingComponent": "",
            "reportingInstance": "",
            "source": {
                "component": "kubelet",
                "host": "server1"
            },
            "type": "Warning"
        },

@rajivml
Could you help us trouble shooting by providing the reproducing steps and env information?
Environment

Longhorn version:
Installation method (e.g. Rancher Catalog App/Helm/Kubectl):
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version:

Number of management node in the cluster:
Number of worker node in the cluster:

Node config

OS type and version:
CPU per node:
Memory per node:
Disk type(e.g. SSD/NVMe):
Network bandwidth between the nodes:

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal):
Number of Longhorn volumes in the cluster:

HI @PhanLe1010
We are seeing it on both single node and multi-node environments
I will share an environment for your offline analysis via DM over slack
Longhorn version: 1.3.1
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: RKE2
Number of management node in the cluster: 3 nodes which acts as both master + worker
Number of worker node in the cluster: 3 nodes which acts as both master + worker
Node config: 32 Core 128GB RAM
OS type and version: RHEL
CPU per node: 32
Memory per node: 128GB RAM
Disk type(e.g. SSD/NVMe): SSD
Network bandwidth between the nodes: Azure Provided
Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Azure Disks
Number of Longhorn volumes in the cluster: Around 20

I saw this today also. It was after I upgraded k3s from 1.23.4 -> 1.23.13, but that may be coincidence.
Specifically, I saw the User "USER" cannot get resource "volumeattachments" in API group "storage.k8s.io" at the cluster scope error

Longhorn version: 1.3.2
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: k3s 1.23.13

Number of management node in the cluster: 2
Number of worker node in the cluster: 0

Node config

OS type and version: Cent OS 7.9.2009
CPU per node: 8
Memory per node: 16G
Disk type(e.g. SSD/NVMe): SSD (I think?)
Network bandwidth between the nodes: Unknown, but it's a lot

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): They are VMs hosted on prem, but I'm not sure what the service is
Number of Longhorn volumes in the cluster: 13

User "USER" cannot get resource "volumeattachments" in API group "storage.k8s.io" at the cluster scope

This error is not related to this issue. It indicate that the client is missing the RBAC permission. Where did you see that error (from which pods)?

Hi
I have same issue
MountVolume.WaitForAttach failed for volume "pvc-xx" : volume vol-xx has GET error for volume attachment csi-xx: volumeattachments.storage.k8s.io "csi-xx" is forbidden: User "system:node:ip-xx.compute.internal" cannot get resource "volumeattachments" in API group "storage.k8s.io" at the cluster scope: no relationship found between node 'ip-xx.compute.internal' and this object
But we scaledown the statefulset replicas to 0 and scale it back such that volume attachment flow gets triggered again

Also run into this issue. Can confirm that workaround suggested by @simonreddy2001 works.

We need to have a resilience way to recover from this automatically.
cc @derekbit @shuo-wu @PhanLe1010

@diamonwiggins @hedefalk @simonreddy2001 @Orhideous @rajivml
I tried to reproduce the issue using Longhorn v1.3.2 and a StatefulSet with 2 replicas on a 2-node cluster.
Reboot the two nodes repeatedly, but still cannot reproduce the issue.
Could you please provide the reproducing steps? If you run into the issue again, could you provide a support bundle as well? Thanks.

Ref: https://github.com/kubernetes/kubernetes/issues/120571

Theoretically, this issue could be very well related to the upstream issue https://github.com/kubernetes/kubernetes/issues/120571.
However, attempting to reproduce using similar steps as in the upstream issue yields no success. The attempted reproducing steps are:

Install Kubernetes v1.25.15+rke2r2/v1.27.5+rke2r1
Install Longhorn v1.5.3 using this longhorn-manager image phanle1010/longhorn-manager:v1.5.3-injected-detach-error. Thi longhorn-manager image added the logic to artificially inject the detach error into longhorn-csi-plugin to simulate a temporary detach error. This is the code https://github.com/PhanLe1010/longhorn-manager/commit/a29962e468806eb8209ad45b53ec4be204d4266d
Deploy this deployment into the clusterapiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: '1'
  generation: 7
  labels:
    workload.user.cattle.io/workloadselector: apps.deployment-default-test-dep
  name: test-dep
  namespace: default
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      workload.user.cattle.io/workloadselector: apps.deployment-default-test-dep
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        workload.user.cattle.io/workloadselector: apps.deployment-default-test-dep
      namespace: default
    spec:
      affinity: {}
      containers:
        - image: ubuntu:xenial
          imagePullPolicy: Always
          name: container-0
          resources: {}
          securityContext:
            allowPrivilegeEscalation: false
            privileged: false
            readOnlyRootFilesystem: false
            runAsNonRoot: false
          stdin: true
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          tty: true
          volumeMounts:
            - mountPath: /mnt
              name: vol-7rasu
      dnsPolicy: ClusterFirst
      nodeName: phan-v603-pool2-e46dd713-75pnq
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
        - name: vol-7rasu
          persistentVolumeClaim:
            claimName: test-pvc

Use this script to simulate the reproducing steps in the upstream issue#!/bin/bash

set -o errexit
set -o nounset
set -o pipefail
set -x

# Inject detach error
kubectl -n longhorn-system patch -p '{"value": "26"}' --type=merge lhs storage-minimal-available-percentage

# Run application pod
kubectl scale deployment test-dep --replicas 1
kubectl wait --for=condition=available deployment/test-dep
# Delete the app
kubectl scale deployment test-dep --replicas 0
kubectl wait --for=delete pod --selector=workload.user.cattle.io/workloadselector=apps.deployment-default-test-dep

# Wait for detach error
while true; do
    if kubectl get volumeattachment -o json | grep "Simulated detach error"; then
        break
    fi
    echo "Waiting for volumeAttachment to get error..."
    sleep 1
done

# Kill KCM
kubectl -n kube-system delete pod -l component=kube-controller-manager
sleep 2

# Start a new KCM
kubectl -n kube-system wait --for condition=Ready=true pod --selector=component=kube-controller-manager

# there is no way how to wait for KCM to process the volumeattachment...
sleep 13

# Create a new pod *after* KCM started processing volumeattachments
kubectl scale deployment test-dep --replicas 1
sleep 1
kubectl wait --for condition=PodScheduled=true pod --selector=workload.user.cattle.io/workloadselector=apps.deployment-default-test-dep

# Stop injecting errors to detach
kubectl -n longhorn-system patch -p '{"value": "25"}' --type=merge lhs storage-minimal-available-percentage

# Now, the second pod should start, but it's stuck at "no relationship found between node '<IP_ADDRESS>' and this object"

Unfortunately, the end result is new pod always is able to come up so cannot reproduce the issue

Next action
Even though we are not able to reproduce the upstream issue, from code analysis, I do think that the race condition in the upstream issue COULD be the root cause of this ticket. The upstream issue is fixed in:

Kubernetes v1.26 line: >= v1.26.10
Kubernetes v1.27 line: >= v1.27.7
Kubernetes v1.28 line:>= v1.28.3
Kubernetes v1.29 line:>= v1.29.0

Therefore, I think the next step for this ticket would be:

Ask the user to try with these fixed Kubernetes versions to see if the issue still persists. (cc @diamonwiggins Could you try to upgrade Kubenetes to the fixed versions?)
Close this ticket
If the user still hits the issue after upgrading Kubenetes to the fixed versions, we can reopen the ticket

WDYT @derekbit @innobead @ejweber ?
Workaround:
Additionally, from code analysis, I think the workaround may be to scale down the workload, wait for the workload to be fully terminated, then scale back the workload again. Kube-controller-manager should be able to recreate the VolumeAttachment for the new pod

Hi, I have face the similar issue in Kubernetes v1.29

I'm seeing the same behaviour when using Mayastor. In my case I drained the node of Mayastor volumes, restarted the Mayastor  pod (openebs-io-engine-xxx), the uncordoned the node to Mayastor volumes. I then noted that some of the Stateful set volumes were stuck in 'ContainerCreating', reporting an io error with no further details.
The OpenEBS CSI controller was reporting:
I0511 17:34:46.274399       1 csi_handler.go:234] Error processing "csi31ad7af564f89fe04d71d5cc0e2240ee1f5b73d9da88f3e933c1b26d9f501219": failed to  detach: could not mark as detached: volumeattachments.storage.k8s.io"csi31ad7af564f89fe04d71d5cc0e2240ee1f5b73d9da88f3e933c1b26d9f501219" not found

Scaling down the StatefulSet to 0, then scaling back up resolved the issue.
I know Mayastore is an entirely different project, but I thought this would be helpful information for 1) anyone also googling their way here, and 2) adding information towards the "is this an upstream issue" question?

Just adding to the list: also ran into this issue after node reboots on 1.7.2.


Add support to forward containers output to splunk via hec connector

We do currently use docker on Debian 11 and let the containers log on stdout through the
Splunk logging driver (https://docs.docker.com/config/containers/logging/splunk/) via HEC (https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) into Splunk.
In term of docker-compose this gives us the possibility to configure Splunk-logging on a per deployment basis, which is very comfortable as well as independent of any central configuration.
We would like to move to Podman on RHEL8 servers. Everything works fine so far, but we didn't find a way to log stdout of the containers via the HEC interface into Splunk, as the currently available podman version 4 does not provide such a splunk logging driver.
Is there a solution on the part of Podman to log the output of the containers into Splunk.
If not, is it possible to commission a corresponding development against payment?
I opened a feature request in the https://github.com/containers/conmon/issues/340 and I was told to open the request in this repo.

Thanks for opening the issue! the team is going to start by focusing on adding support in podman for conmon-rs, as that will be a required first-step. Once that's done, we can evaluate the work needed for this. In the meantime, anyone can chime in if they're interested in implementing this

We do also use the the splunk logging driver in our docker deployments and we would be very pleased if podman/conmon would support it too. From my point of view this logging driver is an important feature to make podman a real drop-in-replacement for docker in (splunk-specific) production environments.
Thanks in advance :)

Bumping this, as I'm also looking for a better solution than scraping the container logfiles. Is this still waiting on podman changes? Is there a ticket in the podman repo for that?

This still needs conmon-rs changes as well as podman changes. still a lot of work to do here

Raising this issue again, this really need to be implemented or at least have some sort of workaround for it.

@mehdik2023 thank you for the ping! Do you plan to contribute the feature? Right now it seems no real priority for the project but we may raise it since multiple folks looking forward to a solution right now.


Build contributing.md

It should match the contributing md in other uber projects

Made this in https://github.com/uber/react-vis/pull/214


AKShare 接口问题报告 | 调用深圳证券交易所-市场总貌-证券类别统计接口报错

使用版本为：1.15.47。
详细问题描述
执行与报错如下：

stock_szse_summary_df = ak.stock_szse_summary(date="20241216")
Traceback (most recent call last):
File "", line 1, in
File "/home/apple/.local/lib/python3.10/site-packages/akshare/stock/stock_summary.py", line 39, in stock_szse_summary
temp_df = pd.read_excel(BytesIO(r.content), engine="openpyxl")
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 495, in read_excel
io = ExcelFile(
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 1567, in init
self._reader = self._engines[engine](
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_openpyxl.py", line 553, in init
super().init(
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 573, in init
self.book = self.load_workbook(self.handles.handle, engine_kwargs)
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_openpyxl.py", line 572, in load_workbook
return load_workbook(
File "/home/apple/.local/lib/python3.10/site-packages/openpyxl/reader/excel.py", line 346, in load_workbook
reader = ExcelReader(filename, read_only, keep_vba,
File "/home/apple/.local/lib/python3.10/site-packages/openpyxl/reader/excel.py", line 123, in init
self.archive = _validate_archive(fn)
File "/home/apple/.local/lib/python3.10/site-packages/openpyxl/reader/excel.py", line 95, in _validate_archive
archive = ZipFile(filename, 'r')
File "/usr/lib/python3.10/zipfile.py", line 1272, in init
self._RealGetContents()
File "/usr/lib/python3.10/zipfile.py", line 1339, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

使用版本为：1.15.47。
详细问题描述 执行与报错如下：

stock_szse_summary_df = ak.stock_szse_summary(date="20241216")
Traceback (most recent call last):
File "", line 1, in
File "/home/apple/.local/lib/python3.10/site-packages/akshare/stock/stock_summary.py", line 39, in stock_szse_summary
temp_df = pd.read_excel(BytesIO(r.content), engine="openpyxl")
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 495, in read_excel
io = ExcelFile(
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 1567, in init
self._reader = self._engines[engine](
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_openpyxl.py", line 553, in init
super().init(
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_base.py", line 573, in init
self.book = self.load_workbook(self.handles.handle, engine_kwargs)
File "/home/apple/.local/lib/python3.10/site-packages/pandas/io/excel/_openpyxl.py", line 572, in load_workbook
return load_workbook(
File "/home/apple/.local/lib/python3.10/site-packages/openpyxl/reader/excel.py", line 346, in load_workbook
reader = ExcelReader(filename, read_only, keep_vba,
File "/home/apple/.local/lib/python3.10/site-packages/openpyxl/reader/excel.py", line 123, in init
self.archive = _validate_archive(fn)
File "/home/apple/.local/lib/python3.10/site-packages/openpyxl/reader/excel.py", line 95, in _validate_archive
archive = ZipFile(filename, 'r')
File "/usr/lib/python3.10/zipfile.py", line 1272, in init
self._RealGetContents()
File "/usr/lib/python3.10/zipfile.py", line 1339, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

请按 issues 模板提交问题


Accessing LocalDb shows System.InvalidOperationException: Timeout expired.

Description
I have the following yml,
name: Build

on: [push, pull_request]

jobs:
  build:

    runs-on: windows-latest
    strategy:
      matrix:
        dotnet-version: ['5.0.x' ]

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup .NET Core
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: ${{ matrix.dotnet-version }}
      - name: Install dependencies
        run: dotnet restore
      - name: Build
        run: dotnet build --configuration Release --no-restore
      - name: Test
        run: dotnet test --no-restore --verbosity normal

My project depends upon LocalDB. When the Github Action runs, its failes with the below error,
The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

Area for Triage:
.NET Core
Question, Bug, or Feature?:
Question
Virtual environments affected

[x ] Ubuntu 16.04

[x ] Windows Server 2019

Image version
Latest
Expected behavior
The build should be passed.
Actual behavior
The build failed with error,
Application startup exception: System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

On ubuntu, it says PlatformNotSupportedException
Repro steps
Copy paste the above yml and run any dotnet project with LocalDB.

Moved to InMemoryDatabase.


Optimize memory allocations in BlockMerkleRootRule.ComputeMerkleRoot()

Hi,
This PR reduces the number of memory allocations needed in ComputeMerkleRoot().
I tried to measure it in very simplified manner:
[Fact]
public void MerkleRootComputationMutated()
{
	GC.TryStartNoGCRegion(200_000_000);
	long kbAtExecution = GC.GetTotalMemory(false) / 1024;

	for (int i = 0; i < 10_000; i++)
	{
		var leaves = new List<uint256>()
		{
			new uint256("281f5acb40a65640bc48b90b5296a87d09341e3510608b191c9bc3a511f8e436"),
			new uint256("d0249653efaaa999f0278bb390c0f4ec3e5465a10f35264ebcfbb6dd6a677abd"),
			new uint256("7897c117fddbf98ea9749cc868a9d1e663b198dd3ac0ae5837734007f0060b20"),
			new uint256("ced314892a97f342a136269e2842fd0dbd1cab1fa84557bc48420f7cf96f0bc7"),

			new uint256("5b98af3d7554916483bca1a52f16570a93f07c95d6aeb8d08b0794c86cf58128"),
			new uint256("5b98af3d7554916483bca1a52f16570a93f07c95d6aeb8d08b0794c86cf58128"),
			new uint256("5b98af3d7554916483bca1a52f16570a93f07c95d6aeb8d08b0794c86cf58128"),

			new uint256("91be5a63b4b70c8329f20960e49a8bd3adeb77fcbadf78014ac54efaf1647a31"),
			new uint256("ac2dcf5c46d9a801389b6c0630b435ce5c2fa850cfac5456f16937dd8ae697d3")
		};
		bool mutated;
		uint256 root = BlockMerkleRootRule.ComputeMerkleRoot(leaves, out mutated);

		Assert.Equal("95aa5bba66381c3817df338895349acd3fc3e8ce226e04a5e2acbb53db18b9c0", root.ToString());
		Assert.True(mutated);
	}

	long kbAfter1 = GC.GetTotalMemory(false) / 1024;
	GC.EndNoGCRegion();
	long kbAfter2 = GC.GetTotalMemory(true) / 1024;

	Trace.WriteLine(kbAtExecution + " Started with this kb.");
	Trace.WriteLine(kbAfter1 + " After the test.");
	Trace.WriteLine(kbAfter1 - kbAtExecution + " Amt. Added.");
	Trace.WriteLine(kbAfter2 + " Amt. After Collection");
	Trace.WriteLine(kbAfter2 - kbAfter1 + " Amt. Collected by GC.");
}

Results for original code:
2831 Started with this kb.
164423 After the test.
161592 Amt. Added.
2840 Amt. After Collection
-161583 Amt. Collected by GC.

Results for the modified code:
2805 Started with this kb.
147712 After the test.
144907 Amt. Added.
2840 Amt. After Collection
-144872 Amt. Collected by GC.

So the result for this test gives circa 9% in memory savings.
PS: branch list is used for debugging, I guess, as it is used in a write-only way using branch.Add() and it is not consumed anywhere.

Brilliant, it can be a good idea to try it in IBD mode.

@dangershony So I have tried to measure the impact of the patch.

I ran FN twice - first without the patch and second time with the patch applied
It ran in both cases up to circa 525.000th block (it took a bit more than one hour)

On my computer, time spent in the BlockMerkleRootRule rule (summed up) was:

vanilla: 25.8086 seconds
with the patch: 20.505 seconds (circa 20% faster)

This is not statistically correct measurement, but still one can believe that the improvement is "not bad" for the few changed lines.

@dangershony @mixitka These are the types of unwanted allocations that we should be fixing in the code. Good job!
@mixitka can you please sync both Stratis main and test as well Bitcoin to 200k blocks to ensure that everything is still OK?

@mixitka
yes actually it's not just about speed performance improvement we are looking for but memory performance too, so this change is happily welcomed by me

I will sync testnet then push

@fassadlr I have tested on Stratis mainnet & testnet actually.

@dangershony wait to merge, I'd like to ask to @mixitka why not moving hashh declaration at the same position of hash declaration: at a first sight to me it seems it can be moved out of the while loop because it's overwritten everytime it's used in the inner loop, this mean more gain and less allocation.
P.S.
this could be even improved further when we'll move to dotnetcore2.2 target, using Span object

@MithrilMan Actually, you can move hashh out of the while loop as you say.

ok @MithrilMan so anyway we need to wait for 2.2 I say we push this and when we move to span we can move out the hashh otherwise we need to wait for build again and a change will anyway come.


Regenerate CNAME file

Github requires a CNAME file on the gh-pages to set up custom domains e.g for pyviz. If this file is clobbered, DNS breaks and you get a 404 error.
The simplest fix would probably be to declare the domain (e.g pyviz.org) and have the option to generate the CNAME file for each website build (the file is very simple). This option should probably be requested explicitly as always adding a CNAME file will probably have unintended consequences.

Alternative suggestion to consider from Philipp, taken from a conversation elsewhere: "just ensure that the CNAME isn't added in the latest commit".

Note: github auto commits CNAME to the docs branch when you define a custom domain in the repo settings page.
It's possible other such things might happen (i.e. where making a change to project settings via the github website results in modifications to the docs branch).

It's possible other such things might happen (i.e. where making a change to project settings via the github website results in modifications to the docs branch).

I don't think there are too many such things to worry about. I know github supports jekyll themes and things but you won't be using that if you are using nbsite. CNAME is definitely the most important such modification that Github settings might apply that I know about.

Already supported by the travis/gh pages integration, but need to document.

Just to capture this here, if you include fqdn in the travis deploy section then the settings tab on GH will autopopulate with the CNAME. No need to add it manually.


Preview Version With B2C?

Library

[ ]<EMAIL_ADDRESS>or<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[X ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>
Description
Hey team, i've been trying out the new version as requested by @jasonnutter and so i am upgrading from an other client ('oidc-client', which can login/logout etc) but i can not get this new version to work with our Azure B2C.
The flow that i want to implement is the Authentication Code Flow for Azure B2C.
I have registered the callback required for msalService.handleRedirectCallback and i have subscribed to the broadcastService.
I also have registered the library in app.module.ts with the following configuration:

MsalModule.forRoot({

        auth: {
            clientId: 'XXXXXXX-XXXXXXXX',
            authority: "https://my-site.b2clogin.com/tfp/my-site.onmicrosoft.com/B2C_1_signupsignin1",
            validateAuthority: true,
            redirectUri: "https://localhost:5001/oidc-signin",
            postLogoutRedirectUri: "https://localhost:5001/",
            navigateToLoginRequestUrl: true,
        },
        cache: {
            cacheLocation: "localStorage",
            storeAuthStateInCookie: false, // set to true for IE 11
        },
        framework: {
          unprotectedResources: ["https://www.microsoft.com/en-us/"],
        //  protectedResourceMap: new Map(protectedResourceMap)
        },
    },
    {
        popUp: true, // !isIE
        consentScopes: [
            "openid", "https://my-site/read"
        ],
        extraQueryParameters: {}
    }),

When i then execute the .login()  a pop-up is opened and closed instantly and the error is see in the browser console is: Uncaught (in promise): ClientAuthError: Error: could not resolve endpoints. Please check network and try again.
Ofcourse i have a working network but what i dont see is any calls in the network-monitor.
Any advice please or maybe a working example? (i could only find the implicite flow)

@Taats Unfortunately, B2C does not yet support the Auth Code Flow. cc @pkanher617 who can provide more context.

@Taats  Auth code flow is in an alpha in our core library and support for it in Angular would probably not be ready till next quarter at the earliest.

@DarylThayil Thanks for the heads up, guess ill have to look elsewhere then (for now : )


Following the tutorial results in a failed build

Details
I followed the video tutorial, pip installed sphinx, ran sphinx-quickstart, uploaded to github, installed the github hook on readthedocs. The result is a failed build.

Project URL: http://egel.readthedocs.io/
Build URL (if applicable): https://readthedocs.org/projects/egel/builds/5783020/
Read the Docs username (if applicable): egel-lang

Expected Result
A vanilla website.
Actual Result
Failed build.

I just seen your latest builds are successful, were you able to solve the problem?

Closing this as the user hasn't responded or given more information, feel free to reopen with more information if you are still having this problem :).


Add a way to store extra data (JSON) to User profiles

@scott what about it ? http://nandovieira.com/using-postgresql-and-jsonb-with-ruby-on-rails

Yeah I was thinking of this or just serializing into a text column.

I'll open a PR with this enhancement ASAP ok ?

That would be awesome.  I picture using the User#create API as the primary way of adding json data to the User.  Can you also add this?


[Communication] - Use Azure.Core.AccessToken instead of custom type

ApiView: https://apiview.dev/Assemblies/Review/290c367d508340a386fc4a42d7474f36
We were using CommunicationUserToken that used to contain both an access token and a user id. But now our issue token route only returns the token so we can switch to Azure.Core.AccessToken.
Now that the type CommunicationUserToken is not exposed, I also removed the autorest extension that was renaming it from CommunicationIdentityAccessToken
I also removed some of the files that should have been removed in #18066

/azp run net - communication - ci

/azp run net - communication - ci

/azp run net - communication - ci

/azp run net - communication - ci


Allow pathlib for file paths

What does this PR do?
Test CLI & API (bash tests/autotest.sh)
Test APIs used by main.py.

[ ] Test Pass

(Copy and paste the last outputted line here.)

[x] Not Applicable (i.e., the PR does not include API changes.)

Check API Document
If any new APIs are added, please check if the description of the APIs is added to API document.

[ ] API document is updated (linear, nn)
[x] Not Applicable (i.e., the PR does not include API changes.)

Test quickstart & API (bash tests/docs/test_changed_document.sh)
If any APIs in quickstarts or tutorials are modified, please run this test to check if the current examples can run correctly after the modified APIs are released.

Superseded by PR #310


Create Image Tiles for Leaflet

Open Streetmap Tile Provider

vue/src/components/Leaflet/LeafletComp.vue
enthält Beispieldaten.
 These tiles need to be generated.

Probably more useful: https://gis.stackexchange.com/questions/285483/how-can-i-convert-an-image-into-map-tiles-for-leafletjs

To-dos:

[ ] Georeference image from received coordinates
[ ] Generate tiles from image, probably with https://gdal.org/programs/gdal2tiles.html
[ ] Create API Endpoints for getting the tiles, the path should follow: https://leafletjs.com/reference.html#tilelayer

/spend 8h

Closing this, due to because we don't see a huge advantage in a tile server, since the images of the beds are currently pretty small, and we only got one available.


Typescript example

Is your feature request related to a problem? no.
Not finding any typescript example spent about ten hours trying to integrate simple example to my typescript project.
Describe the solution you'd like
Add with-typescript to examples directory.
Additional context
I already manage to add it with typescript; can handle this issue myself.

The main concern with this is duplication of language, and unnecessary dev overhead for maintaining a separate build process in that sort of example.
In reality there is very little difference in implementation and it should absolutely not take ten hours if you have experience with TypeScript.
Please do feel free to share a link to your repo though, I am sure it will be very helpful to others!


Installation error: externally-managed-environment

Hello, I installed certbot on Ubuntu 24.04:
$ sudo snap install --classic certbot
$ certbot --version
certbot 2.11.0

For your plugin, I added Python. Then, I tried to install your plugin but it doesn't work:
$ sudo apt install python3-pip
$ pip install certbot-dns-infomaniak
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

I tried with pipx:
$ sudo apt install pipx
$ pipx install certbot-dns-infomaniak
Note: Dependent package 'certbot' contains 1 apps
  - certbot
Note: Dependent package 'josepy' contains 1 apps
  - jws
Note: Dependent package 'charset-normalizer' contains 1 apps
  - normalizer
Note: Dependent package 'distro' contains 1 apps
  - distro

No apps associated with package certbot-dns-infomaniak. Try again with '--include-deps' to include apps of dependent packages, which are listed above. If you are attempting to install a library, pipx should not be used. Consider
using pip or a similar tool instead.

I'm not used to Python. Could you help me find the right commands to install your plugin?

I think I found a solution using only apt:
sudo apt install certbot python3-certbot-nginx python3-certbot-dns-infomaniak


B-20438 and B-20440

Agility Tickets
B-20440
B-20438
Previous PRs (into INT) Associated with this PR

        PR
Definition

13571
Corrected the package.json from pointing directly to the branch to point at the tag (#v1.2.4) of the react-file-viewer

13543
This is in response to I-13228 and I-13225 (kickbacks)

13520
Due to I-13228 issue the previous merge-commit for these tickets had to be reverted

13508
Resolved an issue with missing QueryClientProvider in tests

13487
Initial merge to allow for rotation/zoom and save capabilities for images associated with docs

Verification that it all works
If you would like to verify that rotation/zoom still works on this branch, make sure you perform the following steps to ensure it will work as designed.

Delete the node_modules folder

rm -rf node_modules

Clear Yarn cache

yarn cache clean

Re-install Yarn modules

yarn install

Start the server and client
After creating a move with an upload (image)
As Office user (SC if counseling is needed.. TOO/TIO/Prime if not), navigate to the move and view the uploads/documents/orders and verify that you are able to rotate the images, zoom in and out on the images, and save the rotation.

Fails

:no_entry_sign:

Danger failed to run dangerfile.ts.

Error SyntaxError
Unexpected token u in JSON at position 0
SyntaxError: Unexpected token u in JSON at position 0
    at JSON.parse (<anonymous>)
    at checkYarnAudit (dangerfile.ts:58:24)
    at Object.<anonymous> (dangerfile.ts:94:5)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at requireFromString (/home/circleci/transcom/mymove/node_modules/require-from-string/index.js:28:4)
    at /home/circleci/transcom/mymove/node_modules/danger/distribution/runner/runners/inline.js:161:68
    at step (/home/circleci/transcom/mymove/node_modules/danger/distribution/runner/runners/inline.js:52:23)
    at Object.next (/home/circleci/transcom/mymove/node_modules/danger/distribution/runner/runners/inline.js:33:53)
    at /home/circleci/transcom/mymove/node_modules/danger/distribution/runner/runners/inline.js:27:71
    at new Promise (<anonymous>)

Dangerfile
53|   // Require new src/components files to include changes to storybook
54|   const hasComponentChanges = danger.git.created_files.some((path) => path.includes('src/components'));
55|   const hasStorybookChanges = allFiles.some(
56|     (path) => path.includes('src/stories') || !!path.match(/src\/.*\.stories.jsx?/),
57|   );
--------------------------^
58|
59|   if (hasComponentChanges && !hasStorybookChanges) {
60|     warn('This PR does not include changes to storybook, even though it affects component code.');
61|   }

  Generated by :no_entry_sign: dangerJS against 983b53e24991da5eebb2410511cf813f149171ae

At the end there had to do a make clean and re-generate the swagger files


Please add appimage for linux/etc

Please add appimage for linux/etc
doesnt seem to work with proton
awakened poe trade appimage works with poe1 under linux

ops


[ci] ignore certificates for kitware apt channel in CUDA jobs (fixes #4646)

Attempts to fix the blocking CI issue on CUDA jobs (#4646). See that issue for more investigation.
Basically, I believe that the nvidia/cuda images (last updated 9/20/2021) are using a client somewhere that is using one or more certificates relying on the Let's Encrypt root certificate that expired 9/30/2021.
This PR patches .ci/setup.sh to say "just trust the kitware apt repository and don't check it's certificate" for CUDA jobs.
Notes for Reviewers
I'm going to open an issue later today on the issues page for the nvidia/cuda images, asking them to update their images. But at least for now, I hope this PR will unblock LightGBM's CI.

Looks like #4645 (Eigen being unavailable from GitLab) has not been addressed by GitLab yet 🙃

remote: A repository for this project does not exist yet.
Error: fatal: repository 'https://gitlab.com/libeigen/eigen.git/' not found
Error: fatal: clone of 'https://gitlab.com/libeigen/eigen.git' into submodule path '/home/guoke/actions-runner/_work/LightGBM/LightGBM/external_libs/eigen' failed
Failed to clone 'external_libs/eigen'. Retry scheduled

https://github.com/microsoft/LightGBM/pull/4648/checks?check_run_id=3792108299

I'm going to open an issue later today on the issues page for the nvidia/cuda images, asking them to update their images.

Thanks! I hope we'll be able to revert this our PR soon.


support operators with type hints

This fix pipeline component parser failure for component __init__ with type hints, default parameters assignment with empty str alongside =  can be fixed as well.

What changes were proposed in this pull request?

How was this pull request tested?

Developer's Certificate of Origin 1.1
   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

Hi @wenjun93! Thank you so much for your contribution - this is an area of the code that we have been wanting to clean up.
I see that this PR is still a WIP, but I have a few suggestions that may or may not help when it's ready for review. I think the area that you want to be looking in is the _parse_properties() function around line 109. Here we check for the presence of default value assignment (=), but we don't make any checks for : and type hints like we should.
I would also recommend adding a test for this particular case since parsing can result in so many edge cases (as the existence of this PR itself proves 😄 ). Our current tests for this functionality are located in elyra/tests/pipeline/airflow/test_component_parser_airflow.py:172-290. These 3 tests check against a dummy operator files located in elyra/tests/pipeline/resource/components (airflow_test_operator.py and airflow_test_operator_no_inputs.py). If you're familiar with pytest and feel comfortable doing so, I think it would be worthwhile to create a new resource file - maybe something like airflow_test_operator_type_hints.py and add a test to test_component_parser_airflow.py as well.
Feel free to reach out with any questions!

@kiersten-stokes thanks for the detailed hints! updated.


🐛 Bug Report: Small UI issue in alignment of Discord icon

📜 Description
There's a small UI issue with the centre alignment of the Discord icon. Wouldn't be too noticeable until we hover over it.
👟 Reproduction steps

Hover over the Discord icon at the bottom of the sidebar

👍 Expected behavior

👎 Actual Behavior with Screenshots

💻 Operating system
MacOS
What browsers are you seeing the problem on?
No response
🤖 What development environment are you experiencing this bug on?
Docker
🔒 Did you set the correct environment variables in the right path? List the environment variable names (not values please!)
No response
📃 Provide any additional context for the Bug.
No response
📖 Relevant log output
No response
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🔗 Are you willing to submit PR?
Yes, I am willing to submit a PR!
🧑⚖ Code of Conduct

[X] I agree to follow this project's Code of Conduct

/assign


Using Calendar with MuiPickersUtilsProvider and useStaticState not working

Environment

Tech
Version

@material-ui/pickers
3.2.10

material-ui
4.9.14

TypeScript

React
16.12.0

Browser
any

Peer library
moment 2.25.3  date-io/moment 2.6.0

Steps to reproduce
Trying to use Calendar with MuiPickersUtilsProvider and useStaticState as per example
Expected behavior
Calendar to be displayed
Actual behavior
Error
Can not find utils in context. You either a) forgot to wrap your component tree in MuiPickersUtilsProvider; or b) mixed named and direct file imports.  Recommendation: use named imports from the module index.
Live example
https://codesandbox.io/s/fancy-paper-x0pcx?file=/src/App.js

This sounds related to, if not a duplicate of #1650. Also, we have moved our effort to v4, we don't support v3. Can you reproduce the issue on http://next.material-ui-pickers.dev/?

thanks for the refence!
I updated my code to use v4, replaced Calendar with StaticDatePicker and MuiPickersUtilsProvider with LocalizationProvider as per new docs, and everything worked.


Add user() helper function to get the currently authenticated user.

Okay, so I'm not sure if this one has ever been suggested, or what the feeling on this will be, but to me this feels like a natural addition to the helpers.php file.
In Laravel we can do this:
$user = Auth::user();

And this:
$user = request()->user();

Why not just have a helper like this?
$user = user();

I use helper functions (response(), redirect(), request(), view()) frequently in my controllers, and being able to access the currently authenticated user in a similar way just feels more consistent.

This was already suggested and declined by Taylor:
https://github.com/laravel/framework/pull/8818

@gmsantos Thanks, I was looking through the existing issues trying to find that, but couldn't.
@taylorotwell Closing this. If you've had a change of heart since, feel free to pull in. ;)


The wav file is so big, how to compress it.

It's 1M+ when I recording only 10s
so big it is, so, how to compress it.

Wav file are bigger by its nature! I cannot do anything about it! But Yeah i foresee the other formats to be supported like mp3, amr, etc.


[static_shock_cli]: Handle SocketException when checking for Pub version (resolves #128)

Handle SocketException when checking for Pub version.
static_shock_cli checks for a newer version of the package on pub.dev when running shock serve. It assumes that the user has an active internet connection, but this is not always the case.
While making a GET request when there is no internet connect, the http library defines and throws a ClientException. At a lower level, this is a dart:io SocketException. In order to avoid importing http, this PR just handles SocketException.
When a SocketException occurs, the code cancels the Pub check and logs a warning. This allows static_shock_cli to continue serving the site using the current local version. The alternative would be to abort the serve and tell the user to restore the internet connection. However, that would make for a bad user experience.
Issues to consider

This PR does not include a test. I didn't see any related tests for this type of functionality, and I'm uncertain how I would mock StaticShockCliVersion to test it.
UpgradeCommand and VersionCommand also check for Pub version updates. I didn't update the code there since that is outside of the scope of issue #128. However, at least VersionCommand should also handle socket exceptions.

This issue probably deserves a more holistic approach. We have multiple plugins that talk to the network, too, such as the GitHub plugin. We probably need a concept of an online vs offline mode. I would imagine that trying to run a standard build without internet should fail, but a special offline mode could be requested via arguments. I believe Flutter works that way.
Feel free to post thoughts about where we might inject this control.

@matthew-carroll
In the static_shock_cli package, I would say the following commands should work in offline mode:

version
serve
create

And these should fail if there is no internet:

build
upgrade

I'm not currently sure what an online/offline mode control would look like that spans both static_shock_cli and static_shock, but here are some comments:

Doesn't serving the site also build it? If running build in offline mode fails, we wouldn't want that to cause serve to fail.
Plugins like the GitHub plugin could individually decide their behavior based on the on/offline status. For example, I think the GitHub plugin should still work in offline mode. It would just return an empty list of contributors.

Doesn't serving the site also build it? If running build in offline mode fails, we wouldn't want that to cause serve to fail.

Why not? If build fails I would expect serve to fail. What's the use case where you'd get meaningful value from serve without building the website?

For example, I think the GitHub plugin should still work in offline mode. It would just return an empty list of contributors.

I'm not sure that would be a desirable behavior in general. When build runs, it deletes the existing build first. So if you happen to run without network access and you didn't realize it, you just deleted the version of your website that had the list of contributors.
Can you review the Flutter CLI and see how much/little works offline without explicitly telling the CLI to go into offline mode?

Why not? If build fails I would expect serve to fail. What's the use case where you'd get meaningful value from serve without building the website?

I'd like shock serve to work in offline mode even though shock serve internally builds the app on every update. That's fine for shock build to fail when building the final production version. I'm trying to differentiate shock build and whatever the internal build mechanism is, which I assume shock serve and shock build both share. Whatever build mechanism shock serve uses should not fail in offline mode just because it's offline.

I'm not sure that would be a desirable behavior in general. When build runs, it deletes the existing build first. So if you happen to run without network access and you didn't realize it, you just deleted the version of your website that had the list of contributors.

Good point. However, this is still shock serve not shock build that I'm talking about. Serving a slightly inferior version of the website during development is still better than not being able to serve anything at all. For me it's not that often that I'm without internet, but I have a friend in rural India who is regularly without internet.

Can you review the Flutter CLI and see how much/little works offline without explicitly telling the CLI to go into offline mode?

These Flutter CLI commands work while offline:
flutter --version
flutter help
flutter channel
flutter config --enable-windows-desktop
flutter analyze
flutter clean
flutter devices
flutter logs
flutter screenshot

flutter doctor works adequately well (note the network error messages):
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.22.1, on macOS 13.6 22G120 darwin-x64, locale en)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.2)
[✓] IntelliJ IDEA Community Edition (version 2022.2.1)
[✓] VS Code (version 1.89.1)
[✓] Connected device (2 available)
    ! Error: Browsing on the local area network for iPhone SE (2nd generation). Ensure the device is unlocked and attached with a
      cable or associated with the same local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)
[!] Network resources
    ✗ A network error occurred while checking "https://pub.dev/": Failed host lookup: 'pub.dev'
    ✗ A network error occurred while checking "https://storage.googleapis.com/": Failed host lookup: 'storage.googleapis.com'
    ✗ A network error occurred while checking "https://maven.google.com/": Failed host lookup: 'maven.google.com'
    ✗ A network error occurred while checking "https://cocoapods.org/": Failed host lookup: 'cocoapods.org'
    ✗ A network error occurred while checking "https://github.com/": Failed host lookup: 'github.com'

If you do explicitly allow offline mode (or flutter pub get is already up to date), these also work:
flutter create myproject --offline
flutter run
flutter test

These CLI commands don't work:
flutter create
flutter upgrade
flutter build
flutter pub get

Looks like Flutter can provide local status information without being online.
For Static Shock that's just shock version.
I wasn't completely clear on your last two categories. You mentioned flutter create in both groups. Also, you said the first group was "explicitly allow offline" but only the first example passed --offline, so I wasn't sure whether each of those commands needed the --offline flag or not.
Assuming the following can't be done offline for Flutter...
flutter create
flutter upgrade
flutter build
flutter pub get

That would probably map to...
shock create
shock template
shock build

Then, finally, flutter run is probably analogous to shock serve.
Action Items:

I think shock create and shock template should be runnable even in offline mode, but when there's no internet, they shouldn't call pub get or shock build automatically. Those are both currently called at the end of the process automatically. We should guard those behaviors on connectivity.
For serve, until we figure out where the actual pipeline code fits into this, the part where serve runs a build should be gated on connectivity, similar to create and template.
We still need to figure out whether we really want every pipeline step to have to check connectivity, itself, or whether the overall pipeline should have a concept of online vs offline mode, and then make it easy for pipeline steps to coordinate with that.

I wasn't completely clear on your last two categories.

Yes, I wasn't very clear with those. The assumptions you made, though, were correct.
Additional clarification:

flutter create needs the --offline flag or it won't work without connectivity.
The -offline flag with flutter create has the following docs: "When "flutter pub get" is run by the create command, this indicates whether to run it in offline mode or not. In offline mode, it will need to have all dependencies already available in the pub cache to succeed."
There is no --offline flag for flutter run or flutter test. They work as long as the required dependencies are available, either through creating a project with the --offline flag or by running flutter pub get while the internet connection was previously available.
There is a --no-pub flag for both flutter run and flutter test. This flag turns off the flutter pub get when executing the command. I didn't experiment with this, though.


How to change the text size of the dialog content, title, and buttons?

I am trying to change the size of the textview based on the screensize and it seems that I cannot manipulate the text size of the texts found in the dialog.

You can't, this library intends to follow the Material Design guidelines.

you can do it like:
dialog.getActionButton(DialogAction.NEGATIVE).setTextSize(18);
It works


Wormhole merge2b

Lots of linting changes.

:tada: This PR is included in version 1.0.0 :tada:
Your semantic-release bot :package::rocket:


Handle multiple amounts

Sometimes both a percentage and a weight is specified, maybe we need to support both:

Double chocolate cookiemix 400 g (89,9%): TARWEbloem [GLUTEN], suiker, palmolie, zonnebloemolie, glucosestroop, magere cacaopoeder 7,2%, KIPHEELEIpoeder (scharrel), MELKeiwit, GERSTemoutbloem [GLUTEN], zout, rijsmiddelen: E500-E450, aroma. Witte chocolade chunks 45 g (10,1%): suiker, volle MELKpoeder, cacaoboter, emulgator: SOJAlecithine, aroma.

We could recognize both percentage and amount (breaking change, though).


Update lastCachedStamp on cache hit

By not invalidating cached CIT values that were recently used, FPS is dramatically improved, particularly in areas with many armor stands.

Please consider merging this PR - we are using a resource pack with more than a thousand items in it, and one particular room on our server with ~10 armor stands gives me 15 FPS without this patch and 250 FPS with it.

This PR is very likely to be merged, I've briefly looked over it and it is definitely a good idea that'll most likely be merged in the next version.
However, my main goal right now with CIT Resewn it to make sure these fixes are applied for every mc version and not just 1.20. So I am not changing anything until it is properly multiversioned which is what I am doing currently.

However, my main goal right now with CIT Resewn it to make sure these fixes are applied for every mc version and not just 1.20. So I am not changing anything until it is properly multiversioned which is what I am doing currently.

I'll be interested to see how you implement this. I've got a simple mod that I'd like to support multiple versions, but I'm not sure how to do so and maintain it with how Fabric patches the Minecraft code.

If you're interested, take a look at recent commits starting from ab4fcee as well as following my other project SHsuperCM/Stonecutter

Neat - so that commit adds the infrastructure to add new versions using placeholders, and this commit adds 1.19 support into the mix? https://github.com/SHsuperCM/CITResewn/commit/4c04ceb7ee4e9b532f3677c1424433e0e3955a08

Essentially, yes. Currently I am also working on some regex stuff to ease the 1.19.2-4 port with stonecutter.

Is their progress on this?

I'm trying to understand this commit right now, is this not essentially just disabling cache invalidation, essentially the same as raising the config for cache time to infinity?
How is this change supposed to handle external changes that dont invalidate the cache?

Actually, it doesn't set the cache time to infinity - any textures that aren't in use get successfully discarded, including discarding textures when reloading resource packs and changes made to the items/mobs. The difference is, if a texture is being rendered before the cache expires, then the time until the texture is discarded is refreshed.
If you were to start up your mod without this patch in an item-heavy area, especially with a lot of armor stands, the frame rate can plummet into the single digits or less. Increasing the cache duration causes the game to play at a reasonable FPS for the duration of the cache, and then have a single frame that takes an excessively long time, repeating once every cache duration.
For best results, the minimum cache time in the config needs to be increased to two frames, preferably more, when this gets merged - doing so completely eliminates the issue. I can record a comparison for reference.

Not the cache time, the cache invalidation time to be pedantic.
Doing some tests with your PR, it appears that the cache is just never invalidated when it should do so periodically.
The only time the cit gets rebuilt for an item is when something fundamentally changes in the item nbt. And it makes sense that it would do that with the PR as every time you check the cache you are resetting the need to refresh the cache, which is the same as setting the cache invalidation time to infinity. You are just counting to 10 but every time you say the next number you reset back to 0, making 10 never appear(unless there is a pre-existing lag spike preventing you from counting up).

Hopefully showing what's happening in video form works better - if not, I'll reply in the morning :)
https://www.youtube.com/watch?v=uEHt29gzgtI

...

Wow. That was not the video I thought I had copied. It's 3 AM here and I'm not operating at my finest. This is the video I meant to attach.
https://www.youtube.com/watch?v=UxpyW9AB1z4

The idea being that the cache only evicts entries if there hasn't been a cache hit in a reasonable amount of time. So in that sense, yes the cache entries can be held indefinitely.

Barring the video link error, I am well aware that it makes performance better to not invalidate the cache. However it still fundamentally breaks intended behavior.
Only thing I'm taking away from this that could potentially help is finding a way to stagger the cache refreshes to happen across multiple frames instead of all in one frame.

I'm sorry for the confusion from the other night. I should have waited to reply until I was well rested and ready to think through my response properly.
Fundamentally, the reason the cache exists is that loading the data without one takes orders of magnitude more time. I agree that storing data in the cache indefinitely would be problematic. However, I find the time-based First In, First Out cache eviction policy is inappropriate for this use case, as textures that were used on one frame are very likely to be used on the next frame.
This PR changes the eviction policy to a Least Recently Used eviction policy. Textures that have been used recently are retained until they have not been used for a certain period of time, after which they are evicted, and are not loaded again until they are next required. Evicting textures from the cache a certain amount of time since they were last used significantly reduces the number of cache misses over evicting them based on the time they were added to the cache, which improves FPS.
I understand your concerns about failing to evict data that is out of date. However, my testing did not find such a case. Enabling, disabling, or reloading the resource packs cause the old data to be evicted properly, and modifications to the items or entities cause immediate updates to the textures used. If you have further concerns that I may have missed, I am happy to address them.


[HUDI-1606]fix HoodieJavaWriteClientExample

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
HoodieJavaWriteClientExample fail with exception, the PR is to fix the issue.
Brief change log
org.apache.hudi.examples.java.HoodieJavaWriteClientExample
Verify this pull request
This pull request is already covered by existing tests: HoodieJavaWriteClientExample.main()
I manually verified the change by running a job locally.
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2560 (dc9fdb0) into master (c30481f) will increase coverage by 18.60%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master    #2560       +/-   ##
=============================================
+ Coverage     50.90%   69.51%   +18.60%
+ Complexity     3167      357     -2810
=============================================
  Files           433       53      -380
  Lines         19806     1929    -17877
  Branches       2032      229     -1803
=============================================
- Hits          10083     1341     -8742
+ Misses         8904      456     -8448
+ Partials        819      132      -687

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
?
?

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
69.51% <ø> (+0.05%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ
Complexity Δ

...di/common/bootstrap/index/HFileBootstrapIndex.java

...i-cli/src/main/java/org/apache/hudi/cli/Table.java

...che/hudi/exception/InvalidHoodiePathException.java

.../java/org/apache/hudi/common/util/HoodieTimer.java

.../common/table/log/block/HoodieLogBlockVersion.java

...util/jvm/OpenJ9MemoryLayoutSpecification64bit.java

...org/apache/hudi/HoodieDatasetBulkInsertHelper.java

...udi/spark3/internal/HoodieWriterCommitMessage.java

...udi/timeline/service/handlers/TimelineHandler.java

...a/org/apache/hudi/common/util/CompactionUtils.java

... and 371 more


Does this work if you do not know the target macs computer password.

I was wondering if this script will work if you do not know the target mac computer passcode

I just tried (Rick Roll) this on my new MacBook pro with M1 chip, OS Monterey version 12.2.1, screen unlocked and could not get to work. Error message popped up asking me to use shorter name for YouTube address and then attempted to change the name of random folder to the name of link.


feat(CiceroMarkToOOXML): render list, listVariable and item

Signed-off-by: Aman Sharma<EMAIL_ADDRESS>These changes render org.accordproject.ciceromark.ListVariable and org.accordproject.commonmark.List. The way they have been rendered are exactly same. Why? Since these lists can only be rendered at once via insertHtml, which takes in HTML as a string, we can't attach variables in some texts inside it.
But still, I feel it's not required since the purpose of making any text a variable in our project is to allow a user to amend the value of variables simultaneously which is not required in a listVariable.
One can render Volumediscountolist and Volumediscountulist to test this feature.

I will merge this after all the dependencies are upgraded.

@irmerk there is one problem with this though.
If we render list and listVariable undistinguished, we won't get the consistent ciceroMark back.

@irmerk there is one problem with this though.
If we render list and listVariable undistinguished, we won't get the consistent ciceroMark back.

Didn't you have them distinguished in the OOXML that was generated in the add-in console that you showed on our call today? In that OOXML you also had the $class which was either commonmark.list of cicermark.listvariable. Were you doing something different there to ensure the correct node was generated?

@irmerk once @DianaLease looks at it. I think we can merge this.

@DianaLease I am merging this.

@DianaLease Shall I merge this?

Yes, looks good!


Authentication without needing a web browser

I'm running onedrive-sdk for python on an embedded device, and I get the following error message:
root@DD-WRT:/opt/bin# python3.5 onedrivesdk3.5.py
Traceback (most recent call last):
  File "onedrivesdk3.5.py", line 16, in <module>
    code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
  File "/opt/lib/python3.5/site-packages/onedrivesdk/helpers/GetAuthCodeServer.py", line 60, in get_auth_code
    s = GetAuthCodeServer((host_address, port), code_acquired, GetAuthCodeRequestHandler)
  File "/opt/lib/python3.5/site-packages/onedrivesdk/helpers/GetAuthCodeServer.py", line 76, in __init__
    HTTPServer.__init__(self, server_address, RequestHandlerClass)
  File "/opt/lib/python3.5/socketserver.py", line 443, in __init__
    self.server_bind()
  File "/opt/lib/python3.5/http/server.py", line 138, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/opt/lib/python3.5/socketserver.py", line 457, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use

That is after I've manually added 'webbrowser.py' to lib dir, otherwise I get a module not found exception.
If I get the code from a PC, by running the auth process provided in an example, I get the following error:
root@DD-WRT:/opt/bin# python3.5 onedrivesdk3.5.py
Traceback (most recent call last):
  File "onedrivesdk3.5.py", line 20, in <module>
    client.auth_provider.authenticate(code, redirect_uri, client_secret)
  File "/opt/lib/python3.5/site-packages/onedrivesdk/auth_provider.py", line 169, in authenticate
    data=params)
  File "/opt/lib/python3.5/site-packages/onedrivesdk/http_provider.py", line 71, in send
    custom_response = HttpResponse(response.status_code, response.headers, response.text)
  File "/opt/lib/python3.5/site-packages/onedrivesdk/http_response.py", line 51, in __init__
    raise Exception(str(message["error"]))
Exception: invalid_grant

Please add support for manual auth (done on a PC) , like in python-onedrive project.

Actually there is already support for what you want! You can authenticate without the help of the GetAuthCodeServer class. Something like this:
import onedrivesdk

redirect_uri = 'http://localhost:8080/'
client_secret = 'your_client_secret'
client_id='your_client_id'
api_base_url='https://api.onedrive.com/v1.0/'
scopes=['wl.signin', 'wl.offline_access', 'onedrive.readwrite']

http_provider = onedrivesdk.HttpProvider()
auth_provider = onedrivesdk.AuthProvider(
    http_provider=http_provider,
    client_id=client_id,
    scopes=scopes)

client = onedrivesdk.OneDriveClient(api_base_url, auth_provider, http_provider)
auth_url = client.auth_provider.get_auth_url(redirect_uri)
# Ask for the code
print('Paste this URL into your browser, approve the app\'s access.')
print('Copy everything in the address bar after "code=", and paste it below.')
print(auth_url)
code = input('Paste code here: ')

client.auth_provider.authenticate(code, redirect_uri, client_secret)

That should do it without any dependencies... except of course that you need a browser, which segues into your first issue. Running on an embedded device is a bit more complicated because Microsoft Account OAuth requires clients to have Javascript available. The easiest way to accomplish that is usually a web browser, but of course Raspberry Pi and other embedded devices do not have that available. One possibility would be to use Selenium and Iceweasel to 'fake' a browser, such as in this StackOverflow thread. That's not done yet, but if you take such an approach definitely submit a pull request!

Confirm this solution/workaround works.

That should do it without any dependencies... except of course that you need a browser, which segues into your first issue. Running on an embedded device is a bit more complicated because Microsoft Account OAuth requires clients to have Javascript available. The easiest way to accomplish that is usually a web browser, but of course Raspberry Pi and other embedded devices do not have that available. One possibility would be to use Selenium and Iceweasel to 'fake' a browser, such as in this StackOverflow thread. That's not done yet, but if you take such an approach definitely submit a pull request!

P.S Browser is not an issue (even though I'm working on an embedded device - ARM7 router), since I'm under the same WAN IP for both my embedded device and a PC, so I simply copy paste between PC & SSH terminal, but thanks for the reference anyway.

Glad to hear that worked for you. I'm going to close the issue - please reopen if you have any other problems.

Just one more quick question:
Is it possible, to (auto) export a refresh token, so that the SDK will auto find it (or manually load it upon run) to skip the auth process for each & every run? I think this feature existed in the old API..

Yes, that's what the Session object is for. The implementation in the SDK is really a demonstration, and not very secure (basically stores everything in plain text). It's recommended that you implement the SessionBase class with a more secure scheme. Anyhow, once you've done that, the pattern is easy.
# When creating your AuthProvider
auth_provider = onedrivesdk.AuthProvider(
    http_provider=http_provider,
    client_id=client_id,
    scopes=scopes,
    session_type=YourSessionClass)

# Do your authentication above
client.auth_provider.save_session(your_args)

# Next time you start up, check for an existing session. If one is found,
client.auth_provider.load_session(your_args)
client.auth_provider.refresh_token()

Hope that helps.

Yes it works.
Thanks for 'plain text' warning!
You may close the issue.

Hi diman82
I had same issue with getting One drive access token from Raspberry PI without browser or UI/Monitor.   You dont need to copy & Paste access toekn from any other system or browser. Check out following link
https://github.com/RaspiRepo/Pi_OneDrive
On the python script you need to replace your Onedrive user name/password and client ID/secret from Microsoft application control panel by adding your application.
This method is bit cracking approach. But once you get access token that can be used for long time by setting up refresh request.

Just a note about that code: it bypasses the acceptance form from Microsoft Account. That means that the app can ask for ANY permissions to your account and you are accepting them. If you choose to use this code for a small side project that's one thing, but please do NOT use this code for a product or app that will be distributed to other users.

Hi,
I am also trying to authenticate using your snippet above.
I am receiving an error after pasting the code into my terminal:
   client.auth_provider.authenticate(code, redirect_uri, client_secret)   File "/usr/local/lib/python3.5/dist-packages/onedrivesdk/auth_provider.py", line 203, in authenticate     data=params)   File "/usr/local/lib/python3.5/dist-packages/onedrivesdk/http_provider.py", line 71, in send     custom_response = HttpResponse(response.status_code, response.headers, response.text)   File "/usr/local/lib/python3.5/dist-packages/onedrivesdk/http_response.py", line 51, in __init__     raise Exception(str(message["error"])) Exception: invalid_request
Any ideas how to fix this? Thanks.

Can you show your code please?

Why is this issue closed?
As per the issue heading Authentication without needing a web browser?, all answer I see hear required browser interaction(other than some solution using selenium).
Do we have anyway in which we can authenticate without using browser also not using selenium?

What if I have accesstoken, I think there should be a way to use that to make required call to api.  I am using onedrive picker code to get file details and accesstoken and would like to request the file download from python backend. While following the code provided in the readme, it thorws Address already in use error.


⚠ GLDDNS Server (ns2.glddns.com) has degraded performance

In 3e73b5f, GLDDNS Server (ns2.glddns.com) (ns2.glddns.com) experienced degraded performance:

HTTP code: 200
Response time: 470 ms

Resolved: GLDDNS Server (ns2.glddns.com) performance has improved in dc7c6bd after 9 minutes.