Default postprocessor fails when encountering "Unknown" stream

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I'm running yt-dlp version 2021.12.27. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Description
I am working on a default extractor for ESPN+ streams. The extractor is not fully functional yet as I have had to hardcode a few auth tokens, but after doing so I am able to successfully download the video. However, I seem to be running into an issue with the postprocessor:
ERROR: Postprocessing: If you want them copied, please use -copy_unknown
This seems to be related to #2 and #9. I am not sure why the -dn option is not ignoring the excess unknown stream, which is 0:2
If I manually add -ignore_unkown to the command, no post-processing is attempted at all, which leaves me in the same position as before, with a .mp4 file that has not been fixed up. I should also mention that I see this same error without doing anything with my new extractor, by that I mean I get the same result by just directly giving yt-dlp the master.m3u8 and the required header.
Verbose log
yt-dlp -f hls-4190 https://www.espn.com/watch/player/_/id/2935a850-ad4b-4f93-adb8-0b4d60821e66 --cookies espn.com_cookies.txt -o luton.mp4 --fixup force -v
[debug] Command-line config: ['-f', 'hls-4190', 'https://www.espn.com/watch/player/_/id/2935a850-ad4b-4f93-adb8-0b4d60821e66', '--cookies', 'espn.com_cookies.txt', '-o', 'luton.mp4', '--fixup', 'force', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.27 [6223f67a8]
[debug] Lazy loading extractors is disabled
[debug] Python version 3.9.9 (CPython 64bit) - macOS-12.1-arm64-arm-64bit
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [WatchESPN] Extracting URL: https://www.espn.com/watch/player/_/id/2935a850-ad4b-4f93-adb8-0b4d60821e66
[WatchESPN] 2935a850-ad4b-4f93-adb8-0b4d60821e66: Downloading JSON metadata
[WatchESPN] 2935a850-ad4b-4f93-adb8-0b4d60821e66: Downloading JSON metadata
[WatchESPN] 2935a850-ad4b-4f93-adb8-0b4d60821e66: Downloading JSON metadata
[WatchESPN] 2935a850-ad4b-4f93-adb8-0b4d60821e66: Downloading JSON metadata
[WatchESPN] 2935a850-ad4b-4f93-adb8-0b4d60821e66: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] 2935a850-ad4b-4f93-adb8-0b4d60821e66: Downloading 1 format(s): hls-4190
[debug] Invoking downloader on "https://vod-akc-na.media.plus.espn.com/ps01/espn/event/2022/01/09/Luton_Town_vs_Harrogate_T_20220109_1641725757031/3500K/3500_complete_aeng-trimmed.m3u8"
[download] luton.mp4 has already been downloaded
[download] 100% of 3.42GiB
[debug] ffprobe command line: ffprobe -hide_banner -show_format -show_streams -print_format json file:luton.mp4
[FixupM3u8] Fixing MPEG-TS in MP4 container of "luton.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i file:luton.mp4 -c copy -map 0 -dn -f mp4 -bsf:a aac_adtstoasc -movflags +faststart file:luton.temp.mp4
[debug] ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-13<IP_ADDRESS>)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[mpegts @ 0x15181b600] Could not find codec parameters for stream 2 (Unknown: none (CUEI / 0x49455543)): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mpegts, from 'file:luton.mp4':
  Duration: 02:14:08.21, start: 3451.648200, bitrate: 3645 kb/s
  Program 1
  Stream #0:0[0x1e1]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
  Stream #0:1[0x1e2]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s
  Stream #0:2[0x4a6]: Unknown: none (CUEI / 0x49455543)
  Stream #0:3[0x4a7]: Data: timed_id3 (ID3  / 0x20334449)
Cannot map stream #0:2 - unsupported type.
If you want unsupported types ignored instead of failing, please use the -ignore_unknown option
If you want them copied, please use -copy_unknown
ERROR: Postprocessing: If you want them copied, please use -copy_unknown
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 3119, in process_info
    replace_info_dict(self.post_process(dl_filename, info_dict, files_to_move))
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 3292, in post_process
    info = self.run_pp(pp, info)
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 3236, in run_pp
    files_to_delete, infodict = pp.run(infodict)
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/postprocessor/common.py", line 22, in run
    ret = func(self, info, *args, **kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/postprocessor/common.py", line 117, in wrapper
    return func(self, info)
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 890, in run
    self._fixup('Fixing MPEG-TS in MP4 container', info['filepath'], [
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 851, in _fixup
    self.run_ffmpeg(filename, temp_filename, options)
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 327, in run_ffmpeg
    return self.run_ffmpeg_multiple_files([path], out_path, opts, **kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 281, in run_ffmpeg_multiple_files
    return self.real_run_ffmpeg(
  File "/opt/homebrew/lib/python3.9/site-packages/yt_dlp/postprocessor/ffmpeg.py", line 320, in real_run_ffmpeg
    raise FFmpegPostProcessorError(stderr.split('\n')[-1])
yt_dlp.postprocessor.ffmpeg.FFmpegPostProcessorError: If you want them copied, please use -copy_unknown

Can you give a free URL where this happens so I can test?

Hi, unfortunately I have never encountered this problem with any other service (and I don't believe any of the ESPN+ videos are free to view).
I'm also pretty reluctant to share credentials, but we can try either of the following things:

I send you a small video that has been downloaded but not postprocessed, so you can try to handle the postprocessing with --fixup force

I share a master m3u8 and an authorization header and you try to download the video yourself. There is usually about a 4-hr window where the auth header stays valid so while we would have to be in sync, it would not need to be instantaneous.

I send you a small video that has been downloaded but not postprocessed, so you can try to handle the postprocessing with --fixup force

We can do that

As I'm pretty sure all of the applicable videos are 25+ minutes long, the smallest I can do is a 45 MB 180p video, which is still too big to post here. How can I share this with you?

you can use any random file upload service (or google drive?). If you want to share the link privately, you can reach me on discord or just email me

See if this works: https://drive.google.com/file/d/1Mo_aPfWrLh_rKxyTnFu1eMlTHvEwCn0p/view?usp=sharing
Hopefully Google Drive doesn't attempt any of its own encoding or anything like that

If I manually add -ignore_unkown to the command, no post-processing is attempted at all, which leaves me in the same position as before, with a .mp4 file that has not been fixed up.

Was that with --fixup force? Adding -ignore_unknown fixes the issue from my tests
diff --git a/yt_dlp/postprocessor/ffmpeg.py b/yt_dlp/postprocessor/ffmpeg.py
index dc20c272d..695468c63 100644
--- a/yt_dlp/postprocessor/ffmpeg.py
+++ b/yt_dlp/postprocessor/ffmpeg.py
@@ -197,7 +197,7 @@ class FFmpegPostProcessor(PostProcessor):
         yield from ('-map', '0')
         # Don't copy Apple TV chapters track, bin_data
         # See https://github.com/yt-dlp/yt-dlp/issues/2, #19042, #19024, https://trac.ffmpeg.org/ticket/6016
-        yield '-dn'
+        yield from ('-dn', '-ignore_unknown')
         if copy:
             yield from ('-c', 'copy')
         # For some reason, '-c copy -map 0' is not enough to copy subtitles

Hmmm, I must have not been adding the argument properly before. Now when I add --postprocessor-args -ignore_unknown to the command, the fixup works. If you don't want to include it in the postprocessor by default, is there any way it can be added as a default argument for a single extractor only?


memo formatting options

allow a select number of formatting tags (bold, italic, underline, colors)
provide formatting buttons
doesn't have to be html tags (in the textarea)
maybe have a mapping file to allow more tags to be mapped

refer to https://github.com/crutchy-/webdb/commit/795870f717c0fba699cdd3e44d1fc7c8b0b09e9e


Create a folder for the frontend and checkin the initial React application files

Date
Activity
Time Spent

01/18/22
Research, Install & Clean Up React Files
30 minutes

01/20/22
Create PR/Documentation
30 minutes

How to test

Clone the repo https://github.com/North-Seattle-College/ad440-winter2022-thursday-repo
Cd into ad440-winter2022-thursday-repo folder
Checkout frontend-folder-initial-react-files-issue11 branch
Cd into frontend folder
Run npm install
Run npm start
The app should automatically start in your web browser with URL http://localhost:3000/ and "Floop App" on the window.


After upgrading to 1.27.0 we get a context deadline exceeded

Expected Behavior
Be able to connect to temporal server
Actual Behavior
2024/07/17 14:00:20 Unable to create client failed reaching server: context deadline exceeded
Steps to Reproduce the Problem

temporal server start-dev
Run this main.go file with go run main.go

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/pborman/uuid"
	"go.temporal.io/sdk/client"
	"go.temporal.io/sdk/workflow"
)

type Activities struct {
	Name     string
	Greeting string
}

// GetGreeting Activity.
func (a *Activities) GetGreeting() (string, error) {
	return a.Greeting, nil
}

// @@@SNIPEND

// GetName Activity.
func (a *Activities) GetName() (string, error) {
	return a.Name, nil
}

func GreetingSample(ctx workflow.Context) (string, error) {
	logger := workflow.GetLogger(ctx)

	ao := workflow.ActivityOptions{
		StartToCloseTimeout: 10 * time.Second,
	}
	ctx = workflow.WithActivityOptions(ctx, ao)

	// @@@SNIPSTART samples-go-dependency-sharing-workflow
	var a *Activities // use a nil struct pointer to call activities that are part of a structure

	var greetResult string
	err := workflow.ExecuteActivity(ctx, a.GetGreeting).Get(ctx, &greetResult)
	if err != nil {
		logger.Error("Get greeting failed.", "Error", err)
		return "", err
	}
	// @@@SNIPEND

	// Get Name.
	var nameResult string
	err = workflow.ExecuteActivity(ctx, a.GetName).Get(ctx, &nameResult)
	if err != nil {
		logger.Error("Get name failed.", "Error", err)
		return "", err
	}

	// Say Greeting.
	var sayResult string
	err = workflow.ExecuteActivity(ctx, a.SayGreeting, greetResult, nameResult).Get(ctx, &sayResult)
	if err != nil {
		logger.Error("Marshalling failed with error.", "Error", err)
		return "", err
	}

	logger.Info("GreetingSample completed.", "Result", sayResult)
	return sayResult, nil
}

// SayGreeting Activity.
func (a *Activities) SayGreeting(greeting string, name string) (string, error) {
	result := fmt.Sprintf("Greeting: %s %s!\n", greeting, name)
	return result, nil
}

func main() {
	// The client is a heavyweight object that should be created once per process.
	c, err := client.Dial(client.Options{
		HostPort: client.DefaultHostPort,
	})
	if err != nil {
		log.Fatalln("Unable to create client", err)
	}
	defer c.Close()

	workflowOptions := client.StartWorkflowOptions{
		ID:        "greetings_" + uuid.New(),
		TaskQueue: "greetings",
	}

	we, err := c.ExecuteWorkflow(context.Background(), workflowOptions, GreetingSample)
	if err != nil {
		log.Fatalln("Unable to execute workflow", err)
	}
	log.Println("Started workflow", "WorkflowID", we.GetID(), "RunID", we.GetRunID())

	// Synchronously wait for the workflow completion.
	var result string
	err = we.Get(context.Background(), &result)
	if err != nil {
		log.Fatalln("Unable get workflow result", err)
	}
	log.Println("Workflow result:", result)
}

go run main.go
2024/07/17 14:02:06 INFO  No logger configured for temporal client. Created default one.
2024/07/17 14:02:11 Unable to create client failed reaching server: context deadline exceeded
exit status 1

If I change in go.mod the sdk version from:
go.temporal.io/sdk v1.27.0
to
go.temporal.io/sdk v1.26.1

everything works:
➜ go run main.go
2024/07/17 14:04:16 INFO  No logger configured for temporal client. Created default one.
2024/07/17 14:04:16 Started workflow WorkflowID greetings_dcf4cf08-cb31-4460-ac0d-cddb7e2f164d RunID 2f4a0e00-7fa9-4bb2-bd64-5f12473cc5c8

Specifications

Version: goSDK: 1.27.0 go version go1.22.5 darwin/arm64
Platform: Mac

On Mac with the same Go Version I am not able to reproduce this at least with
temporal --version
temporal version 0.13.2 (server 1.24.1) (ui 2.28.0)

My only idea would be something interfering with resolving localhost. Could try using the IP of your server or try passthrough:///localhost:7233?

do you mind trying with:
temporal --version
temporal version 0.13.1 (server 1.24.1) (ui 2.27.3)

No issue with temporal version 0.13.1

You were right. Now at my hotel I can't reproduce the issue. It must have been a networking issue at the client. Closing for now.
Thank you.

I'm also getting this error after upgrade


All Sensors stopped working

HA 2023.4.6 and HA 2023.5.1
All Sensors stopped working since yesterday:
Error Log:
2023-05-05 08:32:53.184 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Received status 401 (error=Unauthorized) for https://query1.finance.yahoo.com/v7/finance/quote?symbols=EBK.DE,O2D.DE,LHA.DE,CL=F,EURUSD=X,ERDZ.MU,JPJX.MU,HSIC,0P000151AI.F,LU1019989323.SG,ZZR8.F,FLRG.DE,NRJ.PA,DBK.DE

I have exact the same error since last evening

Same here

I'm also seeing a problem with an android widget that I presume uses Yahoo Finance to source their data...

Same here to. Log:
This error originated from a custom integration.
Logger: custom_components.yahoofinance.coordinator
Source: custom_components/yahoofinance/coordinator.py:307
Integration: Yahoo Finance (documentation, issues)
First occurred: 11:51:32 (2 occurrences)
Last logged: 11:51:38
Received status 401 (error=Unauthorized) for https://query1.finance.yahoo.com/v7/finance/quote?symbols=RR.L,BT-A.L,ARB.L

This is the same issue as https://github.com/iprak/yahoofinance/issues/93. It seems that restrictions have been added at webserver end which seemed to automatically resolve.
I did however attempt to put in a workaround to try/parsing out the authorization crumb. But the the error log indicates that no crumb was passed. Can you enable debug logging, it might tell what is happening.
logger:
  logs:
    custom_components.yahoofinance: debug

Same issue. Here is the debug log:
2023-05-05 07:39:30.006 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Getting crumb from from https://finance.yahoo.com/quote/NQ%3DF
2023-05-05 07:39:31.258 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Crumb=J7juHWu\u002FOPp
2023-05-05 07:39:31.258 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=^GSPC,^IXIC,^DJI,^RUT&crumb=J7juHWu\u002FOPp'
2023-05-05 07:39:31.506 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Received status 401 (error=Unauthorized) for https://query1.finance.yahoo.com/v7/finance/quote?symbols=^GSPC,^IXIC,^DJI,^RUT&crumb=J7juHWu\u002FOPp
2023-05-05 07:39:31.512 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 1.506 seconds (success: False)
2023-05-05 07:39:31.513 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds.
2023-05-05 07:39:31.514 DEBUG (MainThread) [custom_components.yahoofinance.sensor] ^GSPC available=False
2023-05-05 07:39:31.518 DEBUG (MainThread) [custom_components.yahoofinance.sensor] ^IXIC available=False
2023-05-05 07:39:31.518 DEBUG (MainThread) [custom_components.yahoofinance.sensor] ^DJI available=False
2023-05-05 07:39:31.519 DEBUG (MainThread) [custom_components.yahoofinance.sensor] ^RUT available=False

Thanks @drjjr2 The log is what I was hoping to see.
There seems to be some additional authentication involved and maybe this is based on region from where request is being made leading to different data centers being used. I have been able to successfully access data via a crumb from different internet providers. Last time this happened, I noticed that if the path was changed from v7 to v6, then the requests were successful but after a couple hours requests to v6 also started failing. It is as if some internal change in yahoo was propagating to other websites.

Thanks @drjjr2 The log is what I was hoping to see.
There seems to be some additional authentication involved and maybe this is based on region from where request is being made leading to different data centers being used. I have been able to successfully access data via a crumb from different internet providers. Last time this happened, I noticed that if the path was changed from v7 to v6, then the requests were successful but after a couple hours requests to v6 also started failing. It is as if some internal change in yahoo was propagating to other websites.

I can confirm that:
https://query1.finance.yahoo.com/**v7**/finance/quote?symbols=^GSPC,^IXIC,^DJI,^RUT&crumb=J7juHWu\u002FOPp
returns
{"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Crumb"}}}
while
https://query1.finance.yahoo.com/**v6**/finance/quote?symbols=^GSPC,^IXIC,^DJI,^RUT&crumb=J7juHWu\u002FOPp
returns
{"quoteResponse":{"result":[{"language":"en-US","region":"US","quoteType":"INDEX","typeDisp":"Index","quoteSourceName":"Free Realtime Quote","triggerable":true,"customPriceAlertConfidence":"HIGH","tradeable":false,"cryptoTradeable":false,"currency":"USD","exchange":"SNP","shortName":"S&P 500","longName":"S&P 500","messageBoardId":"finmb_INDEXGSPC","marketState":"REGULAR","exchangeTimezoneName":"America/New_York","exchangeTimezoneShortName":"EDT","gmtOffSetMilliseconds":-14400000,"market":"us_market","esgPopulated":false,"regularMarketChangePercent":0.9396169,"regularMarketPrice":4099.38,"firstTradeDateMilliseconds":-1325583000000,"priceHint":2,"regularMarketChange":38.159912,"regularMarketTime":1683293440,"regularMarketDayHigh":4099.43,"regularMarketDayRange":"4084.73 - 4099.43","regularMarketDayLow":4084.73,"regularMarketVolume":66093059,"regularMarketPreviousClose":4061.22,"bid":4094.59,"ask":4104.44,"bidSize":0,"askSize":0,"fullExchangeName":"SNP","regularMarketOpen":4084.73,"averageDailyVolume3Month":4292687741,"averageDailyVolume10Day":3953081000,"fiftyTwoWeekLowChange":607.7998,"fiftyTwoWeekLowChangePercent":0.17407586,"fiftyTwoWeekRange":"3491.58 - 4325.28","fiftyTwoWeekHighChange":-225.8999,"fiftyTwoWeekHighChangePercent":-0.052227814,"fiftyTwoWeekLow":3491.58,"fiftyTwoWeekHigh":4325.28,"fiftyDayAverage":4039.1782,"fiftyDayAverageChange":60.20166,"fiftyDayAverageChangePercent":0.014904433,"twoHundredDayAverage":3969.841,"twoHundredDayAverageChange":129.53882,"twoHundredDayAverageChangePercent":0.03263073,"sourceInterval":15,"exchangeDataDelayedBy":0,"symbol":"^GSPC"},{"language":"en-US","region":"US","quoteType":"INDEX","typeDisp":"Index","quoteSourceName":"Free Realtime Quote","triggerable":true,"customPriceAlertConfidence":"HIGH","tradeable":false,"cryptoTradeable":false,"currency":"USD","exchange":"NIM","shortName":"NASDAQ Composite","longName":"NASDAQ Composite","messageBoardId":"finmb_INDEXIXIC","marketState":"REGULAR","exchangeTimezoneName":"America/New_York","exchangeTimezoneShortName":"EDT","gmtOffSetMilliseconds":-14400000,"market":"us_market","esgPopulated":false,"regularMarketChangePercent":0.92007303,"regularMarketPrice":12076.498,"firstTradeDateMilliseconds":34612200000,"priceHint":2,"regularMarketChange":110.09961,"regularMarketTime":1683293442,"regularMarketDayHigh":12078.509,"regularMarketDayRange":"12072.087 - 12078.509","regularMarketDayLow":12072.087,"regularMarketVolume":162578119,"regularMarketPreviousClose":11966.398,"bid":0.0,"ask":0.0,"bidSize":0,"askSize":0,"fullExchangeName":"Nasdaq GIDS","regularMarketOpen":11997.339,"averageDailyVolume3Month":5037841451,"averageDailyVolume10Day":5159296000,"fiftyTwoWeekLowChange":1987.668,"fiftyTwoWeekLowChangePercent":0.1970167,"fiftyTwoWeekRange":"10088.83 - 13181.09","fiftyTwoWeekHighChange":-1104.5918,"fiftyTwoWeekHighChangePercent":-0.08380125,"fiftyTwoWeekLow":10088.83,"fiftyTwoWeekHigh":13181.09,"fiftyDayAverage":11824.833,"fiftyDayAverageChange":251.66504,"fiftyDayAverageChangePercent":0.021282757,"twoHundredDayAverage":11493.624,"twoHundredDayAverageChange":582.874,"twoHundredDayAverageChangePercent":0.050712813,"sourceInterval":15,"exchangeDataDelayedBy":0,"symbol":"^IXIC"},{"language":"en-US","region":"US","quoteType":"INDEX","typeDisp":"Index","quoteSourceName":"Free Realtime Quote","triggerable":true,"customPriceAlertConfidence":"HIGH","tradeable":false,"cryptoTradeable":false,"currency":"USD","exchange":"DJI","shortName":"Dow Jones Industrial Average","longName":"Dow Jones Industrial Average","messageBoardId":"finmb_INDEXDJI","marketState":"REGULAR","exchangeTimezoneName":"America/New_York","exchangeTimezoneShortName":"EDT","gmtOffSetMilliseconds":-14400000,"market":"us_market","esgPopulated":false,"regularMarketChangePercent":0.9529283,"regularMarketPrice":33443.42,"firstTradeDateMilliseconds":694362600000,"priceHint":2,"regularMarketChange":315.6836,"regularMarketTime":1683293444,"regularMarketDayHigh":33449.46,"regularMarketDayRange":"33248.55 - 33449.46","regularMarketDayLow":33248.55,"regularMarketVolume":11491663,"regularMarketPreviousClose":33127.74,"bid":33420.83,"ask":33461.1,"bidSize":0,"askSize":0,"fullExchangeName":"DJI","regularMarketOpen":33248.55,"averageDailyVolume3Month":313024032,"averageDailyVolume10Day":297041000,"fiftyTwoWeekLowChange":4782.4824,"fiftyTwoWeekLowChangePercent":0.16686411,"fiftyTwoWeekRange":"28660.94 - 34712.28","fiftyTwoWeekHighChange":-1268.8594,"fiftyTwoWeekHighChangePercent":-0.036553614,"fiftyTwoWeekLow":28660.94,"fiftyTwoWeekHigh":34712.28,"fiftyDayAverage":33076.184,"fiftyDayAverageChange":367.23828,"fiftyDayAverageChangePercent":0.011102801,"twoHundredDayAverage":32708.07,"twoHundredDayAverageChange":735.35156,"twoHundredDayAverageChangePercent":0.022482267,"sourceInterval":120,"exchangeDataDelayedBy":0,"symbol":"^DJI"},{"language":"en-US","region":"US","quoteType":"INDEX","typeDisp":"Index","quoteSourceName":"Delayed Quote","triggerable":true,"customPriceAlertConfidence":"HIGH","tradeable":false,"cryptoTradeable":false,"currency":"USD","exchange":"WCB","shortName":"Russell 2000","longName":"Russell 2000","messageBoardId":"finmb_INDEXRUT","marketState":"REGULAR","exchangeTimezoneName":"America/New_York","exchangeTimezoneShortName":"EDT","gmtOffSetMilliseconds":-14400000,"market":"us_market","esgPopulated":false,"regularMarketChangePercent":-1.177074,"regularMarketPrice":1718.811,"firstTradeDateMilliseconds":558279000000,"priceHint":2,"regularMarketChange":-20.472656,"regularMarketTime":1683232208,"regularMarketDayHigh":1728.3916,"regularMarketDayRange":"1704.8181 - 1728.3916","regularMarketDayLow":1704.8181,"regularMarketVolume":0,"regularMarketPreviousClose":1739.2837,"bid":0.0,"ask":0.0,"bidSize":0,"askSize":0,"fullExchangeName":"Chicago Options","regularMarketOpen":1727.2416,"averageDailyVolume3Month":4292687741,"averageDailyVolume10Day":3953081000,"fiftyTwoWeekLowChange":77.341064,"fiftyTwoWeekLowChangePercent":0.047116954,"fiftyTwoWeekRange":"1641.47 - 2030.05","fiftyTwoWeekHighChange":-311.239,"fiftyTwoWeekHighChangePercent":-0.15331593,"fiftyTwoWeekLow":1641.47,"fiftyTwoWeekHigh":2030.05,"fiftyDayAverage":1792.1952,"fiftyDayAverageChange":-73.384155,"fiftyDayAverageChangePercent":-0.04094652,"twoHundredDayAverage":1826.6267,"twoHundredDayAverageChange":-107.815674,"twoHundredDayAverageChangePercent":-0.05902447,"sourceInterval":15,"exchangeDataDelayedBy":15,"symbol":"^RUT"}],"error":null}}
I get data.

Disabled NordVPN - sensors began working again.

It works when I access this url on my browser: https://query2.finance.yahoo.com/v1/test/getcrumb
copy the crumb
and paste it at the end of the url like this: https://query1.finance.yahoo.com/v7/finance/quote?symbols=^GSPC,^IXIC,^DJI,^RUT&crumb=YOURCRUMBHERE
Not sure how to translate this in code though

@justinodunn That only seems to work if you have accessed Yahoo Finance website once. https://query2.finance.yahoo.com/v1/test/getcrumb doesn't return anything for a brand new request i.e. incognito instance.

Seems like you need to provide the cookie(s) that are provided once you visit the yahoo finance website in the getcrumb request. Otherwise the getcrumb returns nothing, which you observe in an incognito instance. There is however a cookies consent form you need to pass. This can be a challenging step...

Don't know much about shell scripts but they seem to have a workaround here.. https://github.com/pstadler/ticker.sh/blob/acquire-yahoo-finance-session/ticker.sh

@jeroen84 , @justinodunn
That ticker.sh script establishes a session by navigating to finance.yahoo.com, then gets the crumb which is later passed to ticker requests. My component does something similar, crumb+cookies are extracted out of a request for Nasdaq symbol.
Logs provided by other folks confirm that crumb is being supplied in the requests. Can you enable debug logging to verify that crumb is being passed?

Well it started working again for me and I changed nothing.

That is what happened last time too. It suddenly stopped working and started working by itself. It feels that something might be going on at the Yahoo server backend.
One thing I did notice in my experimentation that sometimes the request would fail with invalid crumb. But if I made the same request again, with same crumb+cookies then it succeeded. As if a single re-try might be good idea.

Still not working here. Looks like it can't get the crumb.
2023-05-07 12:34:17.001 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Getting crumb from from https://finance.yahoo.com/quote/NQ%3DF
2023-05-07 12:34:17.567 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=^GSPC
2023-05-07 12:34:17.679 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Received status 401 (error=Unauthorized) for https://query1.finance.yahoo.com/v7/finance/quote?symbols=^GSPC

But logging in the code is not perfect, there are three places where it can fail silently. I will try to enhance the logging.

I see the request for getting the crumb but then in the next request the crumb is not added, is this not necessary?
2023-05-07 15:21:00.001 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Getting crumb from from https://finance.yahoo.com/quote/NQ%3DF
2023-05-07 15:21:00.599 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=STZ.PA,SPYK.DE,ABBN.SW,EW,MSFT,REP.MC,ANTO.L,ACLN.SW,EURUSD=X,EURCHF=X,EURDKK=X,EURGBP=X,EURNOK=X,BAR.BR,BMW.DE,AIR.PA,KBC.BR,KIN.BR,MELE.BR,ACKB.BR,UCB.BR,XFAB.PA,VUSA.AS,MDLZ,COLR.BR,CYAD.BR,GREEN.BR,AD.AS,EUNL.DE,DEME.BR,STK.PA,MMM,ABT,ADBE,ALGN,GOOGL,AMZN,AXP,AMT,AAPL,ASML.AS,AZN,BAC,BLK,SQ,BNP.PA,AVGO,CAT,COP,COST,ACA.PA,BN.PA,DSY.PA,DOCU,LLY,EQNR.OL,EL.PA,HEI,HMC,ISP.MI,INTU,ISRG,JPM,LRCX,OR.PA,MA,MBG.DE,MRK,META,NFLX,NEE,NKE,NSC,NOVO-B.CO,NVDA,ORCL,PYPL,RIO,ROP,CRM,SAN.PA,SLB,NOW,SIE.DE,SONY,TSM,TGT,FTI,SCHW,GS,HD,DIS,TMO,TT,UBER,UNA.AS,VRTX,V,WMT,XZMJ.DE,XYL,ZM,IBE.MC,AAL.L,RMS.PA,DTG.DE,HLT,BKNG,VWS.CO,C,MS,DPWA.DE,AMD'
2023-05-07 15:21:00.792 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Received status 401 (error=Unauthorized) for https://query1.finance.yahoo.com/v7/finance/quote?symbols=STZ.PA,SPYK.DE,ABBN.SW,EW,MSFT,REP.MC,ANTO.L,ACLN.SW,EURUSD=X,EURCHF=X,EURDKK=X,EURGBP=X,EURNOK=X,BAR.BR,BMW.DE,AIR.PA,KBC.BR,KIN.BR,MELE.BR,ACKB.BR,UCB.BR,XFAB.PA,VUSA.AS,MDLZ,COLR.BR,CYAD.BR,GREEN.BR,AD.AS,EUNL.DE,DEME.BR,STK.PA,MMM,ABT,ADBE,ALGN,GOOGL,AMZN,AXP,AMT,AAPL,ASML.AS,AZN,BAC,BLK,SQ,BNP.PA,AVGO,CAT,COP,COST,ACA.PA,BN.PA,DSY.PA,DOCU,LLY,EQNR.OL,EL.PA,HEI,HMC,ISP.MI,INTU,ISRG,JPM,LRCX,OR.PA,MA,MBG.DE,MRK,META,NFLX,NEE,NKE,NSC,NOVO-B.CO,NVDA,ORCL,PYPL,RIO,ROP,CRM,SAN.PA,SLB,NOW,SIE.DE,SONY,TSM,TGT,FTI,SCHW,GS,HD,DIS,TMO,TT,UBER,UNA.AS,VRTX,V,WMT,XZMJ.DE,XYL,ZM,IBE.MC,AAL.L,RMS.PA,DTG.DE,HLT,BKNG,VWS.CO,C,MS,DPWA.DE,AMD
2023-05-07 15:21:00.792 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 0.792 seconds (success: False)

Works again for me with changes from #97.

@eddysteurs That will only happen if crumb response did not contain crumb or the parsing failed. Debug log would contain a message about that.

https://github.com/iprak/yahoofinance/pull/98/files

Had the same issue.
Edited the const.py file (in config/custom_components/yahoofinance) and changed  https://query1.finance.yahoo.com/v7/...... to https://query1.finance.yahoo.com/v6/...... and works again now.
Will monitor and see when v7 is working again.

Change to v6 works for me😁


Local Target Platform

Hi,
Is there a possibility to use a Local Target Platform?
thanks,
Tom

Might wanna take a look at Goomph and its example project.  Currently on 3.0.0-SNAPSHOT, actively seeking feedback before 3.0.0 release.


Feature: UploadMediaFromReader (using io.Reader to upload procedurally-generated/ephemeral files directly)

A rough implementation of a client.UploadMedia derivative which takes an io.Reader instead of a filepath string. I make a lot of bots and programs that generate content procedurally and having to use and manage ioutil.TempFile stuff can be a pain.
Implementation is simple as it comes - mostly adapting the already existing code for dealing with filepaths in doAPI to handle taking a reader directly.
This could be packed into the UploadMedia function pretty cleanly as well (using interface{} and a type switch) but that change could be breaking and would probably need more documentation to avoid confusion.
Let me know how I can improve this. First time contributing code to a public repo.

Coverage decreased (-0.8%) to 90.879% when pulling 3fc01eeeeb762f6343c188502b95fb60efc3ef63 on bclindner:master into efa05aa949f5602163868a351e1e20a988506925 on mattn:master.

Thank you. Looks good to me.


stateful widget error, A constant constructor can't call a non-constant super constructor of 'State'.

here is my public git.
https://github.com/CrazyBunnyz/Sociominer_V2/
there are 2 branches on that git.
the main is my original code with stateless for most of the part. the problem comes when I try to integrate my code with ExpansionTile where I need state to detect on onExpansionChanged. i try to implement the example but I keep getting the error below. can anyone please help. you can see both code before and after the error from both branches.

Hi @CrazyBunnyz
This issue doesn't seem to describe a bug or a feature request. Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow and if you need help with your code, please see https://www.reddit.com/r/flutterhelp/
Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you
PS: this can be fixed by removed const before ChatMember on line 14. Updated code should look like:
ChatMember({Key? key, this.deviceScreenType}) : super(key: key);

This is working as intended

thank you, i already try your suggestion before posting this but it return other error.

on other part after fixing it like this

ill try to post it on stackoverflow as well, hope it will be solved soon. thanks

You need to use MyStatefulWidget instead of ChatMember. ChatMember is a State, not a Widget


visionOS : Invalid redeclaration of 'lock()'

Since the latest 2.8.8, we can't build the library for visionOS

i have the same error in my visionOS project

Fixed in 2.9.0

I can confirm it works on visionOS without issues


feat: add html serializer and rules for Slate

Adds another way to create Slate values from HTML instead of from JSON.
Example is in the comments:

This required $myEntity and [optional $otherEntity ] ending.

Creates this:

Could store this on the server if we wanted, although I'm not sure if it has much more value. Maybe slightly more readable but still should not have server modifying it.

Tried to add some tests, but I can't get it to work and spent too much time not getting anywhere. It works when I test it in the browsers as thats how I was able to create the picture in PR description but when I tried in the test it fails to deserialize and thinks it's an empty input.

Ok, will work on adding that.

:tada: This PR is included in version 0.383.0 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Implement action types pause, resume and restart

Describe the solution you'd like
If a media file is played, it is possible to say "stop" to stop playing. Similarly, with the following action types, it should be possible to

pause the playing of a media file (youtube, mp3 etc) with the action type pause
resume the playing of a media file which has been paused with the action type resume
restart the playing of a media file (either currently playing or paused) with the action type restart

Additional context
Examples:
https://api.susi.ai/susi/chat.json?q=pause
"actions": [{"type": "pause"}]

https://api.susi.ai/susi/chat.json?q=resume
"actions": [{"type": "resume"}]

https://api.susi.ai/susi/chat.json?q=restart
"actions": [{"type": "restart"}]

@Orbiter Currently, youtube video etc is playing over chat screen so the user can't use these actions without closing youtube player. If we play video/MP3 within app then there will be multiple instances of Player, which is not a good practice.


Correct some Javadoc references

Noted that these pointed to the wrong class; probably updated at some point.
Looking at the code it appears these should probably point to these classes instead.

Hi @dwd,
Thanks for fixing the javadoc links. We would really like to merge your pull request in, but before we can accept your pull request, we will need you to agree to our OSS Contributor License. Once you have read through them there is a link at the bottom to send an email to confirm. After we have received the email I will add you as an outside collaborator then I can merge your changes in.
Thanks!

Oh, dear lord. Now I have to pass this by legal.

I've just noticed that your changes are applied to our master branch and we had actually already fixed the javadoc links on our develop branch a few weeks ago.
I'm going to close this pull request as these javadoc fixes will be merged into master when we next do a release (0.5.0). Sorry not to able to accept your pull request, but I hope this won't put you off from raising other pull requests in the future.
Please see our Ways of Working wiki page for information on our git branching strategy and creating pull requests.


H2O4GPU crashing jupyter kernel with message thrust::system::detail::bad_alloc

Customer is reporting H2O4GPU models (GBM and ElasticNet) produced an error message ‘terminate called after throwing an instance of 'thrust::system::detail::bad_alloc’’ and eventually broke jupyter notebook kernel. Retrying results in similar behavior.
The instance is a GCP VM with 32 vCPUs and 4 Tesla P4 GPUs. Dataset used is http://kt.ijs.si/elena_ikonomovska/data.html, which has about 116 million records and is 5.76 GB.
Is this related to #311 ?

bad_alloc occurs in case there's not enough memory. Not related to #311
I was able to run GBM on g3.16xlarge(4 GPUs with 8Gb of RAM as P4).  Can I get code example? Also attached notebook that works.
Elastic net indeed fails at https://github.com/h2oai/h2o4gpu/blob/master/src/gpu/matrix/matrix_dense.cu#L1963 as a result of allocation attempt of the whole matrix on a single GPU. We need to improve this part.

Untitled.zip

If they use one-hot encoding for categories it's efficient and should be avoided.

Hi @sh1ng , I shared the python notebook with you on slack.

xgboost part can be solved by setting 'n_gpus': -1, Don't really know why it's not set by default.
xgb_params = {'max_depth':8,
              'objective':'binary:logistic',
              'min_child_weight':30,
              'eta':0.1, #learning rate
              'scale_pos_weight':2,
              'gamma':0.1, #min_split_loss
              'reg_lamda':0.5, #L2-regularization term
              'tree_method':'gpu_hist',
              'n_gpus': -1, }

Elastic net issue is going to be fixed in #763


Fails when the tasks list is inside a group

Let's take this Reminders setup example,

When adding a task to the "INBOX" list, which is not in a Group, everything works well.
Then, if I set "Work INBOX" which is inside "Work" Group as default reminders list, the workflow script fails with NSAppleScriptErrorNumber = "-1728" with NSAppleScriptErrorMessage = "Reminders got an error: Can\U2019t get list \"Work INBOX\".";.
Error thrown by Run Script line 45.

This issue is still present on the latests Alfred version, latests Alfred Reminders version and Big Sur.


feat: cloud armor - upgrade preconfigured waf list to v33

update cloud armor policy to use v33 of preconfigured waf rules. There is a limitation for each rule where it can include a maximum of 5 expressions thus explaining the additional rule.

lfi-v33-stable
rfi-v33-stable
rce-v33-stable
methodenforcement-v33-stable
scannerdetection-v33-stable
protocolattack-v33-stable
sessionfixation-v33-stable
xss-v33-stable

The list below are not included in this example as they are more application specific and will be evaluated on a per application basis

nodejs-v33-stable
java-v33-stable
php-v33-stable
sqli-v33-stable
cve-canary(Apache Log4j CVE-2021-44228)
json-sqli-canary(JSON-formatted content SQLi)

credit to Yan Bellerose for this repository https://github.com/ybellerose/GCP-Cloud-Armor/tree/main
closes #730

@ybellerose I would appreciate your input on this PR

/lgtm
Rules as-is will work.  observation: you could prepare the rules for the tuning phase by adding the severity level and the opt-out list. Such as:  evaluatePreconfiguredWaf('xss-v33-stable', {'sensitivity': 4, 'opt_out_rule_ids': []})
Results is the same as the actual configuration, simply prepare for the next stage when opt-out rule will be required by some application.


runtests.py is missing in pypi distfile

HACKING.rst.txt says that one should run python runtests.py -v for testing, but the file is missing in the tarball uploaded to pypi. Please add it.

When I manually download that file from github, running the tests also complains about a missing .coveragerc, please add that too.

Note that you can also do python -c "import scipy; scipy.test('full')".

To be fair, that section of HACKING.rst.txt is in the "Recommended Development Setup" section and has just told the user they must clone the Git repo. ISTM that runtests.py is a development tool and not something that should be uploaded to pypi.

@person142 that sounds right to me. On the other hand we do upload tox.ini and tools/, which isn't really needed either. So it's a bit inconsistent.
We should probably just add runtests.py for consistency and archiving. There may be other files missing. A test script that checks all files in paver sdist vs all files under version control would be useful.


Attempt to add Storybook

I am trying to add Storybook to my Phenomic app. Has anyone tried this before maybe?
As a test I added the <Footer /> component, but the css modules don't seem to work. No classnames are available on the imported css in Footer.js. The global css works by just importing the files in the Storybook main.js, so I'm guessing the postcss setup and css loader are ok. Below is the webpack file I made for Storybook, by copying the relevant parts from the Phenomic config.
Does anyone have  a clue what might be wrong with this?
// you can use this file to add your custom webpack plugins, loaders and anything you like.
// This is just the basic way to add addional webpack configurations.
// For more information refer the docs: https://goo.gl/qPbSyX

// IMPORTANT
// When you add this file, we won't add the default configurations which is similar
// to "React Create App". This only has babel loader to load JavaScript.

import path from 'path'

const postcssPlugins = () => [
  require('stylelint')(),
  require('postcss-cssnext')({
    browsers: 'last 2 versions',
    features: {
      customProperties: {
        variables: {
          mainColor: '#111',
          mainColorContrasted: '#eee'
        }
      }
    }
  }),
  require('postcss-reporter')(),
  require('postcss-browser-reporter')()
]

module.exports = {
  plugins: [
    // your custom plugins
  ],

  // webpack 1
  postcss: postcssPlugins,

  module: {
    loaders: [
      // add your custom loaders.
      //
      // *.css => CSS Modules
      {
        test: /\.css$/,
        exclude: /\.global\.css$/,
        include: path.resolve(__dirname, '../src'),
        // webpack 1
        loaders: [
          'style-loader',
          'css-loader?modules&localIdentName=[path][name]--[local]--[hash:base64:5]',
          'postcss-loader'
        ]
      },
      // *.global.css => global (normal) css
      {
        test: /\.global\.css$/,
        include: path.resolve(__dirname, '../src'),
        // webpack 1
        loaders: [
          'style-loader',
          'css-loader',
          'postcss-loader'
        ]
      },
      // svg as raw string to be inlined
      {
        test: /\.svg$/,
        loader: 'raw-loader'
      }
    ]
  }
}

I never used Storybook yet, so I can't help.

We use storybook and absolutely love it. I think I can get some help from @arunoda the storybook creator.

we also use this config file to load the stories. You can ignore the purple stuff which is Meteor specific

we also have an index file which imports all our stories which are defined in component folders
import './accounts/stories';
import './work-in-progress-views/stories';
import './shared/stories';
...

@Maxhodges Thanks! I'll give it another try 👍

User land problem.


How to send a message to a broadcast list

I am trying to send a message to a broadcast group using /send-message method, but I am not sure how to do that with a broadcast group. It seems that a broadcast is not considered a group nor a contact. This are the params I am sending to the method:
url: http://localhost:8080/api/testsession/send-message
params:
{
"phone": "id@broadcast",
"isGroup": true,
"isNewsletter": false,
"message": "Test message"
}
It sends a success message when the api is sent. But it does not send the message.
Does anyone know if there is some kind of "isBroadcast" parameter when sending a message?

Hello, we do not support broadcast.

You can't send to broadcast from whatsapp web, only from mobile, also they must register your number to receive the message


FPS

Is the fps tested in mmdetection too high? Below are the two models I tested, which look very high。
cascader-rcnn：Overall fps: 85.9 img/s, times per image: 11.6 ms/img
retinanet：   Overall fps: 102.3 img/s, times per image: 9.8 ms/img

@BIGWangYuDong
@hhaAndroid


build: rename css vars util to shorten name

#862  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


How to generate text with the Megatron-LM model trained with DeepSpeed

DeepSpeed tutorial https://www.deepspeed.ai/tutorials/megatron/ does not have any guidance regarding text generation after training the model. All the information in the tutorial deals with training, which implies that logic for generation does not change.
I tried running the Megatron-LM script for text generation, (which works fine with the model trained using Megatron-LM scripts), but it was unable to load the checkpoint generated by the DeepSpeed wrapper.
Are checkpoints generated by Megatron-LM and DeepSpeed wrapper of Megatron-LM binary compatible? Do I need to wrap the text generation code with deepspeed module they way it was done with the pre-training logic? Are there examples of text generation based on models trained with DeepSpeed?
Thank you,
Michael

I was able to modify the text generation utility generate_samples.py to wrap Megatron-LM model in DeepSpeed engine and use it to generate text with the checkpoint created by DeepSpeed Megatron-LM wrapper.
I can create a pull request for the change, but the code for Megatron-LM in this repository is over a year old and I think that fixing it will not serve much purpose. The preferred solution is for the Microsoft team to complete their effort of incorporating the more recent version of Megatron-LM in DeepSpeed Examples repo. I left a note to the developers working on that branch appraising them of this bug.
Please note that another bug in DeepSpeed makes it hard to wrap Megatron-LM model in DeepSpeed engine for text generation. Text generation does not require an optimizer. The deepspeed.initialize() call states that the optimizer is optional. In fact, passing None for an optimizer leads to an exception. Trying to create a "fake" optimizer for text generation just to make DeepSpeed code to work complicates the code and makes it hard to test the code on a machine with small amount of GPU memory.
My team fixed the issue with optimizer and sent a pull request to the Microsoft team. Once it is merged, it will be much easier to use text generation with the DeepSpeed code.

Hi @msmolyak, many many thanks to you and your team for the multiple contributions along this line.
Let me work on your submitted bugfix and ramp back up on the DSE side of things. We'd love to get the text generation PR from you once we're setup to properly track Megatron-LM's upstream branch.

Hi, @msmolyak , I'd like to ask you a question,I modify the  generate_samples.py，to use the setup_model_and_optimizer() function in pretrain_gpt2.py. It can load model parallel checkpoint success, but it can't generate any text,it will stuck in logits = model(tokens, position_ids, attention_mask)  without any error log, Do you know what that might be?
def main():
    """Main training program."""

    print('Generate Samples')

    # Disable CuDNN.
    torch.backends.cudnn.enabled = False

    # Timer.
    timers = Timers()

    # Arguments.
    args = get_args()

    # Pytorch distributed.
    initialize_distributed(args)

    # Random seeds for reproducability.
    set_random_seed(args.seed)

    # get the tokenizer
    tokenizer = prepare_tokenizer(args)

    # Model, optimizer, and learning rate.
    if args.deepspeed:
        model = setup_model_and_optimizer(args)
    else:
        model = setup_model(args)

    # setting default batch size to 1
    args.batch_size = 1

    # generate samples
    generate_samples(model, tokenizer, args, torch.cuda.current_device())

Hi @hujian233,
Here are the notes I took when trying to make text generation work in the current version of DeepSpeedExamples (I did not commit the code anywhere, it was just a proof of concept effort):
https://docs.google.com/document/d/1My6UA-2n_MHMZO8w-xwnXoKBU1KWEr4B4_iYveFoXPs/edit?usp=sharing
Step 6 dealing with the changes to generate_samples has three changes. The document contains the diff with my changes.

Update the model using deepspeed.initialize()
Obtain an optimizer since the call above does not work without an optimizer (it is not used in text generation)
Fixed the logic for computing the vocabulary size

The second change was a hack to create an optimizer, which is totally superfluous. The preferred approach here would be to modify the DeepSpeed code to allow initializing a model without an optimizer. (My colleague submitted a pull request to that effect).
This code was able to generate text from the DeepSpeed-trained model. If this document does not offer any clues, let me know and I will try to run text generation with your code in my test environment.

Hi @msmolyak , thank you. I have already generate text normally.

Hi, @msmolyak , Have you solved the problem you raised. If now can use deepspeed initialize without optmizer?
this is my code:
def get_model(args):
    """Build the model."""

    print_rank_0('building GPT2 model ...')
    model = GPT2Model(num_layers=args.num_layers,
                      vocab_size=args.vocab_size,
                      hidden_size=args.hidden_size,
                      num_attention_heads=args.num_attention_heads,
                      embedding_dropout_prob=args.hidden_dropout,
                      attention_dropout_prob=args.attention_dropout,
                      output_dropout_prob=args.hidden_dropout,
                      max_sequence_length=args.max_position_embeddings,
                      checkpoint_activations=args.checkpoint_activations,
                      checkpoint_num_layers=args.checkpoint_num_layers,
                      parallel_output=False)

    if mpu.get_data_parallel_rank() == 0:
        print(' > number of parameters on model parallel rank {}: {}'.format(
            mpu.get_model_parallel_rank(),
            sum([p.nelement() for p in model.parameters()])), flush=True)

    if args.deepspeed and args.fp16:
        model.half()

    # GPU allocation.
    model.cuda(torch.cuda.current_device())

    # Fp16 conversion.
    if args.fp16:
        model = FP16_Module(model)

    # Wrap model for distributed training.
    model = DDP(model)

    return model

def get_parameters(model):
    """Set up the optimizer."""

    # Build parameter groups (weight decay and non-decay).
    while isinstance(model, (DDP, FP16_Module)):
        model = model.module
    param_groups = gpt2_get_params_for_weight_decay_optimization(model)

    # Add model parallel attribute if it is not set.
    for param_group in param_groups:
        for param in param_group['params']:
            if not hasattr(param, 'model_parallel'):
                param.model_parallel = False
    return param_groups

def setup_model_deepspeed(args):
    """Setup model without optimizer."""

    model = get_model(args)
    param_groups = get_parameters(model)
    if args.deepspeed:
        print_rank_0("DeepSpeed is enabled.")

    model, _, _, _ = deepspeed.initialize(
        model=model,
        model_parameters=param_groups,
        args=args,
        mpu=mpu,
        dist_init_required=False
    )

    if args.load is not None:
        _ = load_checkpoint(
            model, None, None, args)

    return model

It doesn't word well with the latest deepspeed version. do you know what I'm missing?  how the deepspeed config file set?

Closing this issue as it appears to be stale.  If you are hitting new issues/have more questions, please open a new issue with the latest DeepSpeed/Megatron-DeepSpeed repo and we would be happy to take a look.  Thanks!


TP: Allow users to assign topology-based DS to ORG servers

I'm submitting a ...

new feature / enhancement request

Traffic Control components affected ...

Traffic Portal

Current behavior:

Traffic Portal prevents users from creating deliveryservice server assignments for topology-based delivery services by going to Server -> Manager Delivery Services.
New behavior:

To support Multi-Site Origin, Delivery Services must must always be directly assignable to ORG servers in Traffic Portal regardless of whether or not the Delivery Service employs a topology.
Minimal reproduction of the problem with instructions:

In Traffic Portal:

Create a topology
Create a delivery service with the topology
Create a server of type ORG
Try to assign the delivery service to the ORG server from Server -> Manage Delivery Services

Note that this does not block 5.0

@zrhoffman - didn't i already to this? i forget :)

You added TP support for assigning ORG servers to a DS in #5164. This issue is for assigning DSes to an ORG server from the Server -> Manage Delivery Services page.

You added TP support for assigning ORG servers to a DS in #5164. This issue is for assigning DSes to an ORG server from the Server -> Manage Delivery Services page.

oh the other way around, gotcha


Fix bugs in how free dimension session options are set:

Describe your changes

The “unet_text_embeds_size” free dimension should always be overridden with a value of 1280 irrespective of image resolution - this value is same as the text_encoder_2 hidden dimension for both SD-XL base and refiner model. Currently using a resolution other than 1024 results in error.
With the optimum pipeline, the right keyword to pass session options is “session_options” and not “sess_options”. This change improves perf by 40% for 512x512 resolution and about 10% for 1024x1024 resolution.

@microsoft-github-policy-service agree company="NVIDIA"

/azp run


How to return the timestamp with the returned values

Dear All,
I modified the KeepVelocity class to save the values of the location, acceleration, velocities of the 2 cars in NoSignalJunctionScenario. However, I want to return the values accompanied with the timestamp where these values occurred.
Any ideas how to do that.
Thanks.

You could use a py_trees blackboard or add a pure static class with static getters and setters. I would recommend using a blackboard, as it is the cleaner way :)
Here is a link with additional info on py_trees blackboards:
https://py-trees.readthedocs.io/en/devel/blackboards.html


Attribute list

Hi, I want to re-implement your work. Can you please let me know how you have created the attribute list? Is it manually or you used any code for this?

@amena6490 You can check the attribute list in the second line of list_attr_celeba.txt file. You can download the list_attr_celeba.txt  by running the bash download.sh.
In the StarGAN code, you can add or remove attributes (or attribute values) here. Note that some part of the code was hardcoded to cover the case of hair color (black/blond/brown). The code ensures that the target label does not have more than one hair color at the same time.
FYI, these are all available attributes:
'5_o_Clock_Shadow', 'Arched_Eyebrows', 'Attractive', 'Bags_Under_Eyes', 'Bald', 'Bangs', 'Big_Lips',
'Big_Nose', 'Black_Hair', 'Blond_Hair', 'Blurry', 'Brown_Hair', 'Bushy_Eyebrows', 'Chubby',
'Double_Chin', 'Eyeglasses', 'Goatee', 'Gray_Hair', 'Heavy_Makeup', 'High_Cheekbones', 'Male',
'Mouth_Slightly_Open', 'Mustache', 'Narrow_Eyes', 'No_Beard', 'Oval_Face', 'Pale_Skin', 'Pointy_Nose',
'Receding_Hairline', 'Rosy_Cheeks', 'Sideburns', 'Smiling', 'Straight_Hair', 'Wavy_Hair', 'Wearing_Earrings',
'Wearing_Hat', 'Wearing_Lipstick', 'Wearing_Necklace', 'Wearing_Necktie', 'Young'


Remove spurious space from directory path

@flcdrg : Thanks for your contribution! The author, @heaths, has been notified to review your proposed change.

#sign-off

@gewarren: I'm sorry - only the author of this article, @heaths, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.


Activating extension 'antfu.i18n-ally' failed: Incorrect locale information provided.

Describe the bug
Seems like the extension fail to activate.
Extension Version
2.3.19
Framework/i18n package you are using
nuxt-i18n
To Reproduce
Steps to reproduce the behavior:

Upgrade i18n-ally, and reopen old projects.

Device Infomation

OS: win10
Version:
VS Code Version:1.49.0

Extension Log
Go to View -> Output -> i18n Ally, and paste the content below. You should mask any sensitive information
rejected promise not handled within 1 second: RangeError: Incorrect locale information provided
extensionHostProcess.js:1048
stack trace: RangeError: Incorrect locale information provided
    at Object.getCanonicalLocales (<anonymous>)
    at BCP47.lookup (.././src/tagSystems/bcp47.ts:28:26)
    at eval (.././src/core/loaders/LocaleLoader.ts:138:75)
    at Array.map (<anonymous>)
    at LocaleLoader.guessDirStructure (.././src/core/loaders/LocaleLoader.ts:138:14)
    at processTicksAndRejections (internal/process/task_queues.js:94:5)
    at LocaleLoader.init (.././src/core/loaders/LocaleLoader.ts:80:35)
    at Function.initLoader (.././src/core/Global.ts:197:9)
    at Function.update (.././src/core/Global.ts:276:13)
    at Function.updateRootPath (.././src/core/Global.ts:220:13)

Change tag system from none to legacy solve this problem. I didn't explicitly set this attribute before. Don't know why now default to  becp.

@yubaoquan Sorry for the delay, wondering what's your locale code that triggers the error? Thanks

@yubaoquan Sorry for the delay, wondering what's your locale code that triggers the error? Thanks

No code change. Before upgrade, the extension worked just fine. Now I set ally with with Language Tag System: legacy, everything's ok.


refactor carstore meta into a class

This should be the "no functional change, just code moving" change.
Future change likely to be making an interface and alternative implementation.

There is also a next patch for turning carstore.CarStore into an interface which carstore.FileCarStore (formerly known as CarStore) implements. Could be merged into this or merged to main sequentially.
https://github.com/brianolson/indigo/pull/1


Missing the milliseconds in Graylog

Line 146 in message.go (v2.0 branch) is currently:
		TimeUnix: float64(time.Now().Unix()),
Unix time is in seconds since 1.1.1970, thus, log entries in Graylog get the milliseconds truncated.
		TimeUnix: float64(time.Now().UnixNano()) / float64(time.Second.Nanoseconds()),
works nicely.
Best regards.

there's  a merge request for this already: #23. Although I hardcoded the time.Second.Nanoseconds() as 1e9... I must admit that your solution is more expressive, hence in my opinion betterer.
Not sure why it hasn't been merged yet though.

Could you add time.Second.Nanoseconds() also for TLS branch?
Thanks!

After a quick look, it seems to me that the tls branch can easily be rebased on top of the current v2 branch, and merged afterwards - in my humble opinion, there's not even any need to cut a new release, as this would simply constitute an extension of the currently existing API, without any (breaking) changes in existing behaviour... gonna submit a merge request : )

Greate!
I'll waiting.
And on this moment merging v2 and tls branches works fine.
Thanks!

Hello.
I am waiting too.


Doc's check_password()'s setter and preferred arguments.

Follow up to 90e05aaeac612a4251640564aa65f103ac635e12.

Thanks for reviews :+1:


add optimism CLI type

it's not reasonable to make the top level cli struct reusable, hence we should roll a standalone struct Cli for optimism that mimics the current one:
https://github.com/paradigmxyz/reth/blob/7dab636b0488b916d602f890537252a6771d107d/bin/reth/src/cli/mod.rs#L33-L38
this way we can get rid of all op specific commands and can select what commands we actually want for OP.
TODO
add op cli struct first

duplicate Cli in optimism-cli crate

In followups we can define the op binary in optimism-bin
FYI @loocapro

May I get this?

I think we should also move the Noargs type to reth-cli-commands first


Using IPX with NestJS

Has anyone managed to successfully integrate IPX with NestJS?

Yes, you can register it as a Global Middleware in main.ts


change to paper name bert-mpu

SequenceClassificationPipeline is a common name for sequence classification.
rename it to let users know about what is the actual classifier they are using


Console says "total is not defined" even without any piece of code.

Challenge Waypoint: Iterate Through an Array with a For Loop has an issue.
User Agent is: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.71 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:
// Example
var ourArr = [ 9, 10, 11, 12];
var ourTotal = 0;

for (var i = 0; i < ourArr.length; i++) {
  ourTotal += ourArr[i];
}

// Setup
var myArr = [ 2, 3, 4, 5, 6];

// Only change code below this line

The console says "total is not defined" when without adding any piece of code. Tested on: Firefox 43.0.4, Chrome 48.0.2564.71 and 	34.0.2036.25.

Hi @freecates,
This is interesting. Good catch I would say. With your link, or upon resetting the code, it should have the test cases, however it doesn't. Some how the test cases are not displayed below the console window, even when I opened in a fresh browser window (no cache-ed data).

Needs looking into. :+1:
It should have the test cases like below:

@freecates thanks for posting this issue!
This is expected. Here is a part form the description:

Instructions
Declare and initialize a variable total to 0. Use a for loop to add the value of each element of the myArr array to total.

So you haven't defined and used the variable total and that's why you get that error.
@raisedadead test cases doesn't show up because there is an error. Also total is not defined and total is undefined are totally different things.
Closing this issue. If you have problems with your code, please, post a message in our Gitter chat room first, thanks.

Please review the Guidelines for Contributing, thank you!.

Also total is not defined and total is undefined are totally different things.

@bugron Yup I get that its clearly mentioned in the console output :smile:
But I do feel that at the start, i.e when the user lands on the challenge for first time he should see the test cases as in other challenges, this may be a shocker to newbies especially when they are used to seeing the green and red icons and the test cases.
A sudden console error may be misleading to some, in comparison to the previous challenges.
Don't you agree?

All I can say this is how things work. Follow instructions, if there are problems post in our Gitter chat room, only if others can reproduce the problem you're experiencing post an issue here.

Haha... its okay @bugron.
 I was just highlighting the fact a newbie may face, with things he is used to.
Have a good one!
P.S: Have done a couple of contributions before, I am aware of the guidelines :wink:

I'm posting the guidelines almost in every comment so it's not just for you ;)

Please review the Guidelines for Contributing, thank you!.

Haha Sorry @bugron :laughing: ! My bad. Its actually a good idea..., I too see a lot of issues that could be solved by the chat.
Good work :+1:

So sorry, but I haven't realized that I couldn't use any name for my
variable to complete the Waypoint, so I got confused with an error even
if I hadn't written any piece of code.
El 11/01/16 a les 15:46, Mrugesh Mohapatra ha escrit:

Haha Sorry @bugron https://github.com/bugron :laughing: ! My bad.
Its actually a good idea..., I too see a lot of issues that could be
solved by the chat.
Good work :+1:
—
Reply to this email directly or view it on GitHub
https://github.com/FreeCodeCamp/FreeCodeCamp/issues/6052#issuecomment-170574734.


sin^2 fails when getting LaTeX

Steps to reproduce:
Enter \sin^ in the Try part of the Forms example project, or attempt to enter sin^2 in the Editor part.
Result:
Exception thrown: 'System.ArgumentNullException' in CSharpMath.dll
An exception of type 'System.ArgumentNullException' occurred in CSharpMath.dll but was not handled in user code
LaTeX Symbol name must not be null.

The atom gets constructed correctly, but getting latex fails.
Variables to powers and numbers to powers work, but functions to powers do not.

{Large Operator sin} is cast to op:LargeOperator and MathAtoms.LatexSymbolNameForAtom(op) returns null.

Sounds like duplicate of #67


Code cleanup #3064: reorganize imports, other minor pep8 fixes

Just a little warm-up. I also fixed some other issues like empty lines with spaces because imports where not so bad in this case.

Coverage remained the same at 94.444% when pulling a723f4b08f4a04940919b2012a740095ffc00547 on mmarzantowicz:refactor into 77304e2cc67752735241439d6069a51bd996a29c on divio:develop.

Coverage remained the same at 94.444% when pulling a723f4b08f4a04940919b2012a740095ffc00547 on mmarzantowicz:refactor into 77304e2cc67752735241439d6069a51bd996a29c on divio:develop.

Coverage remained the same at 94.444% when pulling a723f4b08f4a04940919b2012a740095ffc00547 on mmarzantowicz:refactor into 77304e2cc67752735241439d6069a51bd996a29c on divio:develop.


Drop phantom fields in 1.16?

@sledorze @raveclassic the internals of fp-ts don't rely on the presence on those phantom fields anymore (_A, _L, etc...), do we want to try to remove them and ship version 1.16? This would allow to test this change in the wild (and overall it's quite easy to revert in case of issues)

Yeah, let's try to drop them. However I won't be able to test the changes in production env/projects soon.

I can try on our code base this week.
It will surely breaks some stuff.
:)


CSS Grid

Description
In this PR I will briefly explain CSS Grid and its properties
Kanishk Chhabra
Front-End Web Development using HTML, CSS and JS - Batch 3
Domain
Frontend Dev HTML/CSS/JS
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/aasign

/assign

/assign


Forward the GET request to next handler if contains no query

@glucaci  I have changed the target branch and will first write a few tests that proof the new behaviour.


sbt doc fails on @typeclasses methods with scaladocs

Supposing the following implementation:
package example

import simulacrum.typeclass

/**
  * My formatter
  *
  * @tparam A
  */
@typeclass trait Show[A] {
  /**
    * Formats the model
    * @param a
    * @return
    */
  def show(a: A): String
}

object Shows {
  implicit val intShow = new Show[Int] {
    override def show(a: Int) = a.toString
  }
}

object Hello extends App {
  import Shows._
  import Show.ops._

  println(1.show)
}

Build.sbt:
lazy val root = (project in file(".")).
  settings(
    inThisBuild(List(
      organization := "com.example",
      scalaVersion := "2.12.2",
      version      := "0.1.0-SNAPSHOT"
    )),
    name := "Hello",
    addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full),
    libraryDependencies ++= Seq(
      "com.github.mpilquist" %% "simulacrum" % "0.10.0"
    )
  )

It compiles successfully but running sbt doc will return the following error:
[error] .../simulacrum-doc-bug/src/main/scala/example/Hello.scala:29: value show is not a member of Int
[error]   println(1.show)
[error]             ^

It might have something to do with macro paradise.

I was able to write a typeclass just now that has scaladocs on both the trait and its methods.


[WIP] benchmark for groupby operation

#2210 pr is too large and it contains too many topics and therefore it's hard to be reviewed.
I separate benchmark test code from #2210 in order to polish it first and then share it among other branches.
Let's make it into master first, and then I can start bench testing other ideas of optimization by branching from master to share the same benchmark test code.

If we have more benchmark code, then we need to move the benchmark dir under top-level dir. Currently, I don't think we need.

I think this is ready -- want to merge it?

I still have some changes. Let me clean them up and then we are good to go!

Refactored my code and it should be very easy to add more benchmark test cases.
sample output. A force update is due to I mistakenly amended all my changes to a previous commit.
=========== count on binary categorical data =============

nrows: 100000
reps: 3
avg time to run w/  2 threads: 42.1337 ms.
avg time to run w/  4 threads: 30.247 ms.
avg time to run w/  8 threads: 28.779 ms.
avg time to run w/ 12 threads: 26.7163 ms.
========================== END ===========================

=========== bench_test_aggreate_min summary ============

nrows: 100000
reps: 3
users: 1000
avg time to run w/  2 threads: 187.639 ms.
avg time to run w/  4 threads: 163.525 ms.
avg time to run w/  8 threads: 154.321 ms.
avg time to run w/ 12 threads: 145.579 ms.
========================== END ===========================

=========== bench_test_aggreate_avg summary ============

nrows: 100000
reps: 3
users: 1000
avg time to run w/  2 threads: 180.494 ms.
avg time to run w/  4 threads: 158.512 ms.
avg time to run w/  8 threads: 156.701 ms.
avg time to run w/ 12 threads: 146.346 ms.
========================== END ===========================

ready to go! @hoytak

added data generator for skewed data distribution.
It will be later used for validating the rwlock implementation perf since if a key appears a lot more frequent, then it will cause more contentions when sframe performs aggregations under multithreading context.

Looks good!  Go ahead and merge it!


[8.x] Fulltext Index

This PR adds the ability to create fulltext indexes in your migrations when using MySQL. In the future we may expand this to Postgres.

Looks like the future is now. Checkout https://github.com/laravel/framework/pull/39875 by @tpetry


Percy snapshot issue on the build details page

The build command box is incorrectly sized.

I've looked into this today and am going to close it. There's nothing obvious in the snapshot that would be causing it, and it's showing fine on both Mac and Windows browsers in the "real world".


Override the Classname

Our convention is lowercase, hyphened class names. But this library is rendering uppercase classes which is not desired. Is it possible to change this?
.FormInput {
}
I also cannot add classes to this div, which is very limited considering I want to apply different :after selectors to the div.

You can always create your own custom Fields using the FormField. This is what i have done in the past to format and style them how i want. Also note, react-form V2 is almost ready and has docs that show you how to write these custom fields. Stay tuned :)

V2 is ready and you can customize the fields however you like. Closing because we no longer add classes ourselves, it is now up to the dev to create there own inputs or pass styles to the default ones. See the example custom input in the docs.


API for botlist.space is deprecated

Error occurred while posting to botlist.space on shard 18:
Error: 404 Not Found
Response included with the error: All API endpoints on this URL have become deprecated. Please visit https://botlistspace.gitbook.io/api/ for the new API documentation.

Resolved in b6f7d55


build: Adopt outputexports

This patch simplifies the integration with the zephyr build system
whilst at the same time enriching the set of build targets to allow
zephyr to be configured using its kbuild features.
It works by exploiting "make outputexports", a feature of the zephyr
build system that makes the zephyr compiler configuration available to
other build systems in an easily accessible manner.
Whilst looking at the build we also correct the implementation of clean
so that it no longer destroys any custom zephyr configuration. Like any
other zephyr application one must use "make pristine" to remove the
config too.
JerryScript-DCO-1.0-Signed-off-by: Daniel Thompson<EMAIL_ADDRESS>

@pfalcon: I think you were interested in this...

@daniel-thompson : Thanks for submitting it!
Put simple, @daniel-thompson's patch allows to build JrS for any board supported by Zephyr, present or future.
I built and booted JrS on the following Zephyr boards:
qemu_x86
qemu_cortex_m3
frdm_k64f

I tested building for the following boards (don't have them to boot):
96b_nitrogen
altera_max10
arduino_101
arduino_due

Few other boards are tried didn't build because they don't have full support in Zephyr, few other due to lack of RAM.
So, definitely +1 from me.
@sergioamr , @poussa : FYI.

Of the 21 boards supported by mainline zephyr:
13 build correctly (only 12 build correctly before this patch)
4 build but fail to link because there is insufficient RAM and/or FLASH
3 fail to build because JerryScript tickles a compiler bug in the ARC toolchain
1 fails because CONFIG_UART_INTERRUPT_DRIVEN is set in the .config (I suspect this is a zephyr bug, the config option is not requested by prj.conf)

LGTM

LGTM


Project with dev-release was not created

From @derhasi on January 13, 2015 8:7
My project https://www.drupal.org/project/drutils is not available as drupal/drutils.
What is the procedure to create those projects that have no release yet and did not get any attention via commitlog?
Copied from original issue: webflo/packagist#9

Looks like commit log parsing is broken atm.

From @derhasi on January 13, 2015 11:15
As the last code change was from September, I do not think packagist could have recognized it via the commitlog. As there is no release, releases.tsv is not a source either. So I'm wondering where packagist could have got the information from to get to know that dev release. Maybe you can point me to the relevant code, so I can look for the problem on my own.

From @derhasi on January 14, 2015 16:25
Now I even got the case, that a dev release that existsed is not present anymore: see http://drupal-packagist.webflo.io/packages/drupal/master . Two days ago 7.3.x-dev was available, which is not the case anymore.

The requested package drupal/master 7.3.x-dev could not be found.

From @derhasi on January 20, 2015 15:23
I just recognized that anonymous users in the commitlog get different results than logged in users, see screenshot:

I recognized the problem while looking after the my latest commit I pushed in my master 3.x-dev dev branch. There is still not such a branch in the packagist.


Package Requested

[X] I would like someone else to build the manifest.
[ ] I would like help so I can submit the manifest.
[X] I have performed a search and couldn't find this package.
[ ] I have checked that the installer for this package is not a script file
[ ] I think there is a new version available and I have provided the URL.

Please provide the following information
Publisher: OneLogin, Inc
Publisher Url: https://www.onelogin.com
Package Name: OneLoginDesktopProInstaller
Description: Provides OneLogin Desktop Pro
Package Version: <IP_ADDRESS>5
Installer URL: https://onelogin-desktop-release.s3.amazonaws.com/windows/OneLoginDesktopProInstaller-<IP_ADDRESS>5.msi
Current Package Identifier (for new versions):

Package fails to install silently due to this interactive prompt

The publisher would need to provide Installer switches to bypass this prompt for this package to be added to WinGet.
@stephengillie - Interactive-Only-Installer label please.


Human-mode sorting in performance table

In GitLab by @mcastril on Jan 4, 2024, 16:54
The performance view is not sorting tasks by hierarchical or submission order (I think it's using jobname alphabetical order), which is confusing.

The # therefore only indicates the rowid, which can be misleading.
I think it would be enough to internally pad the CHUNK number in the jobname with zeroes.

In GitLab by @mcastril on Jan 9, 2024, 10:20
Thanks Luiggi, I think it's a good approach

In GitLab by @ltenorio on Mar 15, 2024, 15:29
This was implemented a long time ago, but it is now available in production (ES and DE) along with the "Export to CSV" option from #95.

In GitLab by @ltenorio on Jan 25, 2024, 15:38
mentioned in commit a9cba65c5bcecc7061da8c6804f6f44c170d9c45

In GitLab by @ltenorio on Jan 5, 2024, 08:42

... (I think it's using jobname alphabetical order), which is confusing.

Yes, it is just sorted by name

The # therefore only indicates the rowid, which can be misleading.
I think it would be enough to internally pad the CHUNK number in the jobname with zeroes.

I think we can replace or remove the # column, and add the chunk number column (if exists). (This is a modification to the API too)
Then, do the order by chunk and then by name (if in the same chunk).


Add Spotify support to core.ts

This was already merged into the development branch via https://github.com/nraboy/ng2-cordova-oauth/pull/52.
For the future, please make your PR to the development branch, not the master branch.
Best,


Depedency conflicts with @react-native-firebase/crashlytics

What happened?
Failed building for ios when @react-native-firebase/crashlytics is present due to depency resolve. After the firebase updates 21.4.1, most of their libraries updated past GoogleDataTransport 10.0, so the error extends to analitics, auth, in-app-messaging, and probably others.
Since I'm using EAS to build, I can't test changing the GoogleDataTransport (< 10.0, >= 9.4.1) requirement to try fixing that, or changing podfiles, as they are created by eas when building.
EAS error:
[!] CocoaPods could not find compatible versions for pod "FirebaseCoreExtension":
In Podfile:
RNFBAuth (from ../node_modules/@react-native-firebase/auth) was resolved to 21.6.1, which depends on
Firebase/Auth (= 11.5.0) was resolved to 11.5.0, which depends on
FirebaseAuth (~> 11.5.0) was resolved to 11.5.0, which depends on
FirebaseCoreExtension (= 11.5)
RNFBCrashlytics (from ../node_modules/@react-native-firebase/crashlytics) was resolved to 21.6.1, which depends on
FirebaseCoreExtension
CocoaPods could not find compatible versions for pod "GoogleDataTransport":
In Podfile:
RNFBCrashlytics (from ../node_modules/@react-native-firebase/crashlytics) was resolved to 21.6.1, which depends on
Firebase/Crashlytics (= 11.5.0) was resolved to 11.5.0, which depends on
FirebaseCrashlytics (~> 11.5.0) was resolved to 11.5.0, which depends on
GoogleDataTransport (~> 10.0)
RNMLKitTextRecognition (from ../node_modules/@react-native-ml-kit/text-recognition) was resolved to 1.5.2, which depends on
GoogleMLKit/TextRecognitionDevanagari (= 6.0.0) was resolved to 6.0.0, which depends on
GoogleMLKit/MLKitCore (= 6.0.0) was resolved to 6.0.0, which depends on
MLKitCommon (~> 11.0.0) was resolved to 11.0.0, which depends on
GoogleDataTransport (< 10.0, >= 9.4.1)
Version
@react-native-ml-kit/barcode-scanning: version@react-native-ml-kit/face-detection: version@react-native-ml-kit/identify-languages: version@react-native-ml-kit/image-labeling: version@react-native-ml-kit/text-recognition: version@react-native-ml-kit/translate-text: version
Which ML Kit packages do you use?

[ ] @react-native-ml-kit/barcode-scanning
[ ] @react-native-ml-kit/face-detection
[X] @react-native-ml-kit/identify-languages
[ ] @react-native-ml-kit/image-labeling
[X] @react-native-ml-kit/text-recognition
[ ] @react-native-ml-kit/translate-text

What platforms are you seeing this issue on?

[ ] Android
[X] iOS

System Information
ystem:
OS: macOS 14.5
CPU: (8) arm64 Apple M2
Memory: 344.08 MB / 8.00 GB
Shell:
version: "5.9"
path: /bin/zsh
Binaries:
Node:
version: 20.18.1
path: /usr/local/bin/node
Yarn: Not Found
npm:
version: 10.8.2
path: /usr/local/bin/npm
Watchman: Not Found
Managers:
CocoaPods:
version: 1.16.2
path: /opt/homebrew/bin/pod
SDKs:
iOS SDK:
Platforms:
- DriverKit 24.1
- iOS 18.1
- macOS 15.1
- tvOS 18.1
- visionOS 2.1
- watchOS 11.1
Android SDK: Not Found
IDEs:
Android Studio: Not Found
Xcode:
version: 16.1/16B40
path: /usr/bin/xcodebuild
Languages:
Java: Not Found
Ruby:
version: 3.3.6
path: /opt/homebrew/bin/ruby
npmPackages:
"@react-native-community/cli": Not Found
react:
installed: 18.2.0
wanted: 18.2.0
react-native:
installed: 0.74.5
wanted: 0.74.5
react-native-macos: Not Found
npmGlobalPackages:
"react-native": Not Found
Android:
hermesEnabled: Not found
newArchEnabled: Not found
iOS:
hermesEnabled: Not found
newArchEnabled: Not found
Steps to Reproduce
Install @react-native-firebase/crashlytics, @react-native-ml-kit/text-recognition and build with eas.

@thacio any fixes??

Not really. I uninstalled crashlytics, set all the firebase services to 21.3.0, and will use analytics meawhile. Considering later creating a class simulating crashlytics but using analytics instead, so it would be easier to migrate if this ever gets fixed.

@thacio any fixes??
Well, good news and bad news.

The problem was GoogleMLKit. Other people were going through this problem, they solve it releasing the version 7.0.0.
https://github.com/googlesamples/mlkit/issues/877
So, I bumped every GoogleMLkit dependency to 7.0.0 as was able to compile with firebase 2.6.1 without a problem. Also, the deploymentTarget for ios increased to 15.5.0, so it needs to be added to expo app.json.
Now, the bad news is that it just stopped working altogether for me, but I can't say if it was the bump to 7.0.0 or something else. Since yesterday I've been trying to upgrade to expo sdk 52, so it might be another package that broke the functionalities. But probably it was something with react-native-mlkit, since if I don't upgrade it or firebase, it still works.
If you want to give it a shot, you can clone the repository, look for the GoogleMLKit and bump from 6.0.0 to 7.0.0. I've forged the repo with the bump. https://github.com/thacio/react-native-ml-kit
To install, you would need to clone it locally.
'''
git clone https://github.com/thacio/react-native-ml-kit
npm install ./react-native-ml-kit/text-recognition
'''
'''
{
"expo": {
"plugins": [
[
"expo-build-properties",
{
"ios": {
"deploymentTarget": "15.5.0"
}
}
]
]
}
}
'''

@thacio any fixes??
Well, good news and bad news.

The problem was GoogleMLKit. Other people were going through this problem, they solve it releasing the version 7.0.0. googlesamples/mlkit#877 So, I bumped every GoogleMLkit dependency to 7.0.0 as was able to compile with firebase 2.6.1 without a problem. Also, the deploymentTarget for ios increased to 15.5.0, so it needs to be added to expo app.json.
Now, the bad news is that it just stopped working altogether for me, but I can't say if it was the bump to 7.0.0 or something else. Since yesterday I've been trying to upgrade to expo sdk 52, so it might be another package that broke the functionalities. But probably it was something with react-native-mlkit, since if I don't upgrade it or firebase, it still works.
If you want to give it a shot, you can clone the repository, look for the GoogleMLKit and bump from 6.0.0 to 7.0.0. I've forged the repo with the bump. https://github.com/thacio/react-native-ml-kit To install, you would need to clone it locally.
git clone https://github.com/thacio/react-native-ml-kit
npm install ./react-native-ml-kit/text-recognition

{
  "expo": {
    "plugins": [
      [
        "expo-build-properties",
        {
          "ios": {
            "deploymentTarget": "15.5.0"
          }
        }
      ]
    ]
  }
}

This is good news. I'll check it out.

Just confirming it, text_recognition breaks with GoogleMLKit 7.0.0


Logger for LabVIEW

Logger framework for LabVIEW, similar to log4j.
https://logging.apache.org/log4j/2.x/

https://github.com/fieldrndservices/logger


Update package.json

Fix:
Version update: 8.0.0 -> 9.0.0
Keyworks: separte "discord music bot"

Yeah thanks


Black screen screenshot/video when using RDP

When trying to take screenshot or record video in a headless RDP session the output is always a black screen - nothing is captured.

An alternative when running on Windows is the usage of VNC Server.

I am also facing same issue. Screenshots are not generating if i am doing execution in Linux machine with Headless Chrome.


Push Notifications

Is there an easy way to integrate push notifications into this starter app?

At this time you'll need to do the integration manually, however we recommend using Amazon Pinpoint for Push Notifications and Analytics with React Native. You can see an example here: https://github.com/awslabs/aws-pinpoint-sample


Removing unused apis phase 2

Are these APIs no longer needed, or migrated to the new managed embedding layer? Mostly asking if these tests have been migrated over to a different place, or if we are fine just deleting them. cc @bholmes @mrvoorhe

The ones I've removed were already migrated so I figured while I was in there I'd remove them. I didn't remove all the ones that were migrated though.


added user_id back so FE has follow functionality they can use.

Had to add this back so FE could continue to work on follow feature which is part of MVP.


Running sast-scan on M1 mac does not find anything.

Hi all,
running sast-scan via docker:
docker run --rm -e "WORKSPACE=${PWD}" -v ~/.m2:/.m2 -v "$PWD":/app -v "$PWD/.shiftleftscan-reports":/app/reports shiftleft/scan:arm scan --src /app --type java

on apple M1 mac works without errors, but does not find anything. Any idea why this happens or if this is a bug?
(full script at https://github.com/darkspirit510/BenchmarkJava/blob/adapt-to-m1/scripts/runShiftLeftScan.sh)

Could you set the environment variable -e SCAN_DEBUG_MODE=debug and share the logs?

Sure! Here is the result:
███████╗ ██████╗ █████╗ ███╗   ██╗
██╔════╝██╔════╝██╔══██╗████╗  ██║
███████╗██║     ███████║██╔██╗ ██║
╚════██║██║     ██╔══██║██║╚██╗██║
███████║╚██████╗██║  ██║██║ ╚████║
╚══════╝ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═══╝

[11:07:51] INFO     Scanning /app using plugins ['java']
           DEBUG    ⚡ Executing class "java -jar /opt/spotbugs/lib/spotbugs.jar -textui -include /usr/local/src/lib/../spotbugs/include.xml
                    -exclude /usr/local/src/lib/../spotbugs/exclude.xml -noClassOk -auxclasspathFromFile /tmp/tmpju5ay69y -sourcepath /app
                    -quiet -medium -xml:withMessages -effort:max -nested:false -output /app/reports/class-report.xml /app"

[11:08:05] DEBUG    SARIF and HTML report written to file: /app/reports/class-report.sarif, /app/reports/class-report.html 👍
           DEBUG    ⚡ Executing source-java "/opt/pmd-bin/bin/run.sh pmd --no-cache --fail-on-violation false -language java -d /app -r
                    /app/reports/source-java-report.csv -f csv -R /usr/local/src/lib/../rules-pmd.xml"

[11:23:02] DEBUG    SARIF and HTML report written to file: /app/reports/source-java-report.sarif, /app/reports/source-java-report.html 👍
[11:23:02] DEBUG    Aggregate report written to /app/reports/scan-full-report.json

           INFO     Baseline file written to /app/reports/.sastscan.baseline
                      Security Scan Summary
╔══════════════════════╤══════════╤══════╤════════╤═════╤════════╗
║ Tool                 │ Critical │ High │ Medium │ Low │ Status ║
╟──────────────────────┼──────────┼──────┼────────┼─────┼────────╢
║ Class File Analyzer  │        0 │    0 │      0 │   0 │   ✅   ║
║ Java Source Analyzer │        0 │    0 │      0 │   0 │   ✅   ║
╚══════════════════════╧══════════╧══════╧════════╧═════╧════════╝

Thanks @darkspirit510. Perhaps the project has to be compiled first or scan has to be executed with SCAN_AUTO_BUILD=true as explained in https://slscan.io/en/latest/getting-started/#environment-variables
Since scan invokes pmd and spotbugs separately the score would simply match either pmd or spotbugs so this exercise isn't going to yield anything interesting. plus this project is deprecated so best to look for any alternatives for any serious apps.

Is this different in the tags latest and arm? because it works as expected on latest using an intel pc.

@prabhu you were right! i forgot to build the project. SCAN_AUTO_BUILD=true did not help, but a manual mvn compile did. Thank you!


Use Wii Numbchuck as controller?

Could you use the Wii Numbchuck  with the ps move service tracking a ball of light on top? This would theoretically give you all the degrees of movement

The Wii Nunchuk has to be plugged in to something. Also, it doesn't have a full IMU. e.g., see here


Tree status not updating on github PRs

Type of Request
bug
Infrastructure Environment
flutter/cocoon
What is happening?
The flutter/flutter tree status check on Github is not updating after the tree went green about ~40 minutes ago.
I landed a docs only change to give things a kick just in case (https://github.com/flutter/flutter/pull/148777)
The autosubmit bot will not merge changes if the tree status does not update.
Steps to reproduce
See build dashboard, tree is green (as of filing): https://flutter-dashboard.appspot.com/#/build?repo=flutter&branch=main
PRs like https://github.com/flutter/flutter/pull/148790 do not reflect this.
Expected results
The tree status on Github PRs should be green.

Looks like it just needed a kick! All is well.


Update README.md

Signed-off-by: Divyansh Bansal<EMAIL_ADDRESS>Read then delete

Make sure to use a concise title for the pull-request.
Use #patch, #minor #majora or #none in the pull-request title to bump the corresponding version. Otherwise, the patch version
will be bumped. More details

TL;DR
Please replace this text with a description of what this PR accomplishes.
Type

[x] Bug Fix
[x] Feature
[x] Plugin
[X] correction

Are all requirements met?

[x] Code completed
[x] Smoke tested
[x] Unit tests added
[x] Code documentation added
[x] Any pending items have an associated Issue

Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
https://github.com/flyteorg/flyte/issues/
Follow-up issue
NA
OR
https://github.com/flyteorg/flyte/issues/

Hi @idivyanshbansal thabk you for the PR, please follow the DCO bot to see how to sign off


Swagger API documentation nicknames should include unique for each operation

cc @smarterclayton
As I understand it, the nickname for each operation should be unique[1]. I searched the open issues, and could not find an open issue relating to this, however my apologies if this is a duplicate.
Since the introduction of 'namespaced' apis (/api/v1/services v.s./api/v1/namespaces/{namespaces}/services), there are operations with duplicate nicknames [2][3].
This means when we try to consume APIs via their swagger definition, the client overwrites the namespaced version of the API with the non-namespaced version.
Maybe listService and listServiceByNamespace might be the convention to go by?
Thanks @webron for the help in IRC confirming this
[1] https://github.com/swagger-api/swagger-spec/issues/64
[2] https://github.com/GoogleCloudPlatform/kubernetes/blob/master/api/swagger-spec/v1.json#L9961
[3] https://github.com/GoogleCloudPlatform/kubernetes/blob/master/api/swagger-spec/v1.json#L10863

@bgrant0607 @nikhiljindal
Can describe more about how this affects you?  For example, you cannot make a client in language X using library Y that depends on a valid swagger spec.   This will help us prioritize.

Question above directed at @cianclarke

Yes I believe that was Cian's problem (I sent him towards using the swagger
JS client, which uses nicknames for the methods it uses internally).
On Tue, Jul 14, 2015 at 1:07 PM, Eric Tune<EMAIL_ADDRESS>wrote:

Question above directed at @cianclarke https://github.com/cianclarke
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/11229#issuecomment-121310485
.

--
Clayton Coleman | Lead Engineer, OpenShift

Thanks @cianclarke.
Yes I just checked and we have duplicate nicknames, which we should not.
I am surprised because, last I checked, our swagger spec passed swagger validation, which it should not if we have duplicate nicknames.

Umm, how did you run the validation? :)

@erictune Sure - when attempting to integrate with k8 APIs using https://github.com/swagger-api/swagger-js, I discovered that due to the duplicated nicknames, I was unable to utilise a swagger client to call the APIs I needed access to. In particular, we need to operate on namespaced APIs.
Somewhat unrelated to this issue, for the immediate future we probably won't be using the Swagger API client to build this integration, so it's not blocking us - just would be a nice to fix :-)

$ swagger-tools validate api/swagger-spec/resourceListing.json api/swagger-spec/api.json api/swagger-spec/v1beta3.json api/swagger-spec/v1.json api/swagger-spec/version.json
We do it as a one-off right now, so it is possible that it might have broken since I last checked.
I am working on adding it as part of our continuous tests: https://github.com/GoogleCloudPlatform/kubernetes/issues/5356

@whitlockjc - does swagger-tools check for duplicate nicknames in 1.2 definitions?

I just ran swagger-tools validate again and didnt get any error, so I can confirm that it is not failing on duplicate nicknames.

Not yet, it's on the list: https://github.com/apigee-127/swagger-tools/issues/219


fixes typo in ReadMe text

"...a background script to collect ads they see." ==> "...a background script runs to collect..."

That sentence is a mess thank you!


Antifraud act

Neither the alignment assemblies based process used to develop it nor the substantive insights of the Taiwan Antifraud Act recently passed are reflected in the book.  The former would be a worthy addition to Chapter 5-4 and the latter to something in Part 4, likely 4-1.  @mashbean would appreciate your leadership here.

 Gov4Git notices
On Friday, 06-Sep-24 18:14:05 UTC by Gov4Git dev
Notice ctxpfx
Started managing this issue, using the Plural Management Protocol v1,  as Gov4Git concern 1004 with initial priority score of 0.000000.
This project is managed by Gov4Git, a decentralized governance system for collaborative git projects.
To participate in governance, install the Gov4Git desktop app.
Notice crodwh
This issue's priority score is now 0.000000.
The cost of priority is 0.000000.
The projected bounty is now 0.000000.
Notice e6x6ip
The set of eligible proposals claiming this issue is empty.


feat: SECP related hints

Context: port of the Starknet OS to Rust.
We need an implementation of the hints used by the Starknet OS in the SECP syscalls. These hints rely on private primitives in cairo-vm and must be implemented in this repository.
Checklist

[ ] Linked to Github Issue
[x] Unit tests added
[ ] Integration tests added.
[ ] This change requires new documentation.

[ ] Documentation has been added/updated.
[x] CHANGELOG has been updated.

Hi @odesenfans !!
Sorry the delay, we didn't saw this PR
Can you merge to main and solve the conflicts?

@odesenfans some more points:

Since the VM is only allowed to run whitelisted hints in production we cant enable the VM to execute this hints in production. There are two ways to solve this, leaving the hints under a feature flag like cairo-0-secp-hints`, the other way is that you implement your own hint processor in your repo
There are some lints error in the CI
There are some unwraps where you can handle the error, like the pop().unwrap(); and some more cases
Can you add some integration tests for the new hints?

@pefontana The requested changes and integration tests have been added. Please feel free to run the workflows and review the code.

@JulianGCalderon @pefontana Let me know if there is anything else needed. Would be nice to get the tests passing.

Hi @whichqua, two jobs seem to be failing:

Compute memory and execution traces with cairo-lang. This job executes the programs with cairo-lang
Compute memory and execution traces with cairo-vm. This job executes the programs with cairo-vm.

The second job fails because the hint is not found. This is because this job on the CI runs all the standalone files in cairo_programs and compares it's execution. You could try to move these files to another directory cairo_programs/cairo-0-secp-hints-feature and see if it fixes it. Keep in mind that this will probably fix the CI, but it will not fix the underlying error that is breaking the first job.

@JulianGCalderon Is there a reason we cant add the feature cairo-0-secp-hints flag to cairo-vm-cli?

Hi @whichqua, the compute memory and execution traces with cairo-lang is working OK now. On the other hand, compute memory and execution traces with cairo-vm is still failing. To fix this, could you move the added programs to another directory cairo_programs/cairo-0-secp-hints-feature?
The reason you should do this is because the failing workflow compares the execution of all the standalone programs in the directory with the default features (not including the new added feature). There is another job in the workflow that already tests with other features, so there is no need to test them in this job also.


Linker error with latest Mac OS XCode update: filesize exceeds vmsize

Since the latest XCode CLI Tools v15.0 getting the following error:
../target/debug/deps/my-contract-1d02595d33e6ca8a" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: ld: segment '__DWARF' filesize exceeds vmsize in '../target/debug/deps/libinjectivetesttube.dylib'
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

Quick fix for now:

$ sudo rm -rf /Library/Developer/CommandLineTools

Download XCode CLI Tools 14.3.1 here

For fix see https://github.com/osmosis-labs/test-tube/pull/41


Please add a support to view and downnload mp4 videos

i am using sd-webui in a cloud computing, so your extension has been a solution for me to download, delete and move files and folders
I tried searching for my videos made on deforum using this extension, but the output folders only contains images
Or you can create another  sdui extension for files explorer
Sorry my english

Sounds good, but there's quite a lot of work to be done, such as video thumbnail generation.

Already supported. See https://github.com/zanllp/sd-webui-infinite-image-browsing/wiki/Change-log for more information.

Thank you!