The value was get with GetString, and it contained Phonetic in Japanese.

The value was get with GetString, and it contained Phonetic in Japanese.
How to disable include Phonetic?
Excel

ExcelDataReader result
0: GetFieldType=System.String GetString=田中タナカ, GetExcelValue=田中タナカ, GetName=NAME, GetValue=田中タナカ
Bold font value is not needed
string excelPath = @"";

FileStream s = File.Open(excelPath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
ExcelFileType? excelFileType = ExcelFileType.FindForFilename(excelPath);

if (excelFileType == null)
{
    throw new ArgumentException("Unsupported file type", nameof(excelPath));
}

Sylvan.Data.Excel.ExcelDataReader reader = Sylvan.Data.Excel.ExcelDataReader.Create(s, excelFileType.WorkbookType, new ExcelDataReaderOptions { OwnsStream = true });

while (reader.Read())
{
    for (int i = 0; i < reader.FieldCount; i++)
    {
        var str = reader.GetString(i);
        var excelValue = reader.GetExcelValue(i);
        var name = reader.GetName(i);
        var value = reader.GetValue(i);
        var type = reader.GetFieldType(i);
        Debug.WriteLine($"{i}: GetFieldType={type} GetString={str}, GetExcelValue={excelValue}, GetName={name}, GetValue={value}");
    }
}

await reader.DisposeAsync();

Debug Excel file
PhoneticIssue.xlsx

Thank you very much! I was able to solve the problem with the new version.

Phonetic appears when there is a comma in the cell.

田中,人ヒト
Bold font is not needed.
Excel file is here
PhoneticIssue.xlsx
Thanks


Bump framework version

$Subject

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6611709639


CLI documentation link points to 404 page[🐞]

Which component is affected?
Qwik City (routing)
Describe the bug
If you make a mistake using the routeAction$, the linked documentation page is a 404 - resource not found.
 16:22  warning routeAction$()can only be declared inlayout.tsx, index.tsxandplugin.tsxinside thesrc/routes directory, instead it was declared in "C:/Users/Josch/Desktop/qwik-test/src/components/starter/counter/counter.tsx". Please check the docs: https://qwik.builder.io/qwikcity/loader  qwik/loader-location
Cheers
Reproduction
https://qwik.builder.io/qwikcity/loader
Steps to reproduce
I tried this inside of the default counter component, that's how the error popped up:
   const useRequest = routeAction$((props) => {     console.log('make req')   })
System Info
Windows 10, not relevant tho

Additional Information
No response

It has been resolved.
https://github.com/BuilderIO/qwik/pull/4153


HOW TO DEPLOY DJANGO PROJECT WITH DATABASE USING HEROKUAPP HOISTING SERVICES.

Article checklist
ATTENTION: In your PR - add the ISSUE (topic form) # that this PR corresponds to. You can do this by also adding a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.
NOTE: (Please ensure that you have only one open issue + linked pull request at a time. This will ensure that we complete the article in a timely manner from inception to publishing.)
If you have not already please go over our Resources Page for more tips and tricks on how to improve our overall technical writing.
Preliminary Checks - Formatting and Structure

[ ] Is the article you are submitting an in-depth and unique article? Does it go beyond what is in the official docs and what is covered in other blog sites. See these articles as examples.
[ ] Does your article follow any of the suggested structure formats? - see example formats
[ ] Is your article properly formatted in Markdown? - see Github Markdown guide
[ ] Have you used the correct folder and file structure? - see Contribution Guidelines
[ ] Is your article over 750 words? - Tool: Wordcounter.net
[ ] Have you used our preferred heading structure? - H3 (###) and up
[ ] Does your article provide enough value and detail about your topic? - Articles should be clear, accurate, and fully explained.
[ ] Can your article be understood by beginners? - Assume the audience is smart but has no prior exposure to the common terminology in your article.
[ ] Have you included a hero image that is 600x400 pixels and have the copyright to use it? - Tools: Pixlr Image Editor for resizing and Unsplash for Creative Commons images
[ ] In your PR - add a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.

Grammar & plagiarism checks

[ ] Have you spellchecked and grammar checked your article? - Tools: Write&Improve - ProWritingAid -  Grammarly
[ ] Please place your article through a 3rd party plagiarism checker? We suggest using Quetext, this tool is free and has a daily limit. We typically accept articles with 10% or less.
[ ] Have you checked your article for readability? - Tool: Hemmingway
[ ] Have you added sources for quotes and images that aren't yours?

Technical checks

[ ] Are your code snippets properly formatted for syntax highlighting - see Syntax guide

Contribution guidelines
For first-time contributors and for more details, see Contributing Guidelines

What about this sir @hectorkambow @marienjus

Good afternoon and thank you for submitting your article for publishing.
Your article has been entered into our queue and should be reviewed (and published pending any changes) as soon as a peer reviewer is finished with the ones in the queue before it. :rocket:
Link your topic-suggestion, please.
Thank you. @Horlawhumy-dev

Oh! My bad. Thanks @ahmadmardeni1

Hope I am not missing any info on the review. @ahmadmardeni1 @zolomohan

@Horlawhumy-dev This PR has MULTIPLE index files.
Multiple articles on it. It should only have the articles to be published. One a time - each articles should be its own branch.

@Horlawhumy-dev
I'll be running a final review on your PR when ready.
Upon running your article through our 3rd party plagiarism checker it seemed to raise a few flags and the % was higher than we typically accept.
Please see attached PDF - and revisit the article to ensure we are contributing wholly unique and original content.
Be sure to see our resources page to see more info on plagiarism and what is considered as such.
HOW TO DEPLOY DJANGO PROJECT WITH DATABASE USING HEROKUAPP HOISTING SERVICES. #3225.pdf

Sorry about that sir. I have done the corrections.

Thank you @Horlawhumy-dev @hectorkambow. I'll start the preliminary revisions soon.


Updates future/10 with latest changes from 9.5.0

Merges latest changes from 9.5.0 into future/10

@mitchelsellers and @bdukes as discussed, the two last commits are the merge issues we where having. One about the new default portal template and the other about the command that fires up the frontend build. I built this and did not spot anything wrong but yeah because of the folder moves, it looks like 6000 files but each move counts as 1 deleting and one creation, soooo. yeah, fun times, but it finally builds fine.

@valadas I just pushed a commit that resolved a bunch of differences between my merge and this merge.  I haven't yet done any deep comparison of other files that moved (in the Build and Dnn.AdminExperience folders).

Wow, I have very bad internet right now, but yeah, I will try another build when able and take a look

If I look at your commit, I see like 6000 changes, is that right?

If you look at it in the context of the PR, it still shows the number of changed files for the whole PR.  If you look at it as an individual commit, it's 208 files changed.  Most of those are copyright headers.

@bdukes cool, got it, makes sense to me and I ran a build locally and tested around, looks fine 2 me...

@valadas yeah, if it's passing checks, I'm okay to merge it.  I'll try to find some time next week to do a couple more comparisons to make sure we didn't miss anything, and can submit any fixes as separate PRs

Would you be fine if we merge this before the conference and fix later if we find any issues with it though?


feat: populate internal path with content type field [CU-2jtv8xj]

Ticket
NA
Summary
TODO
Test Plan
TODO

Task linked: CU-2jtv8xj Link content type field to path

@ltsNotMike resolve conflicts please and we will merge it


Features/3 setup testing

Closes #3

Added unit tests
Converted to TypeORM
Added migrations

@kzkaneoka Can I get a review? I changed how things work

@TheBeege Quick question just in case. when running npm run start:dockerLocal locally, I got:
Error during migration run:
Error: connect ECONNREFUSED <IP_ADDRESS>:3306

Am I doing wrong, or will this be done by [Chore] Setup Proper Database Migration #4? If so, LGTM.

@kzkaneoka Ah... I believe that's due to the migration container coming up before the database container has gotten up and running. I'll come up with a way to make that more reliable today. Until then, if you run it a second time, it should work well. Were you able to get it running that way?

@kzkaneoka Try now. I want to make sure that part is working before we merge.

@TheBeege I could run it from my end. Thanks!

@kzkaneoka Feel free to. I've added you to the team with proper permissions. I think the only strong rule we should have is that people shouldn't merge their own code. So please merge whenever you're ready.

@TheBeege Merged!


CLI - API Inquiry

Just curious if this CLI is able to create a new project and get it's auto subdomain after its creation/deployment? Basically, just wanted to know if what I can do inside the Coolify paid plan I can do through the API. Thanks.

Don't know if you know this, but there is a Discord community for Coolify. If you don't get a response quick here, you could and should try there.
https://coolify.io/discord


Implement async TensorFromTransportOptions for GDR

Instead of blocking on completion of an RDMA op, RecvTensor client will now post a work request to the NIC send queue and return immediately.
The GDR background polling thread will handle the callback after the corresponding RDMA op is completed, i.e. polled from the completion queue on NIC. The old epoll based mechanism is removed to trade higher CPU usage for improved throughput and lower latencies for RDMA ops.
The maximum numbers of work request (WR) in the send/recv queues on NIC are increased to entertain the increased number of concurrent RDMA ops. The threshold of tensor size below which we pass the tensor content in metadata is also increased to reduce the pressure to send/recv queues on NIC.
This fixes #23933.
Signed-off-by: Bairen Yi<EMAIL_ADDRESS>

@poxvoculi I was blocked on #23933 for the past week, and I am glad I can fix it now.
I will go back to refactoring of networking plugins in the next couple of days.

I've rebased to latest master and cleanup the code.
Note that the checksum part is removed, as it is not guaranteed that the async RDMA client will read the same tensor content through network compared with gRPC. It turns out that sometimes the worker will RDMA read weights in the next steps from PS after the checksum is transmitted though transport_options (which was computed with stale weights on RecvTensor server). I am not sure it is a bug or not, but the same trick of sharing backed tensor_buffer could be found in gRPC as well, so I'll just leave it there. The original purpose of checksumming (i.e. debugging) should be achieved via some other approaches (e.g. unit testing).

@harshini-gadige Mind to kick off Kokoro testing again?

@harshini-gadige Mind to kick off Kokoro testing again?

Done

@harshini-gadige Ready to pull? It’d be great if I could make it into the 1.13 source release.

@googlebot CLA shall be fine. Let me double check.

@harshini-gadige Ready to pull? It’d be great if I could make it into the 1.13 source release.

@harshini-gadige Ready to pull? It’d be great if I could make it into the 1.13 source release.

Done


Raise better error on notebook_launcher

Currently we rely on PyTorch's standard error message when the notebook_launcher has issues, and it is exceedingly cryptic how to continue when an import is causing issues. I specifically mean the Cannot re-initialize CUDA in a forked subprocess....
Especially for the user, it doesn't actually explain what is truly going on. As a result, this PR introduces a better error message.
Example trace:
---------------------------------------------------------------------------
ProcessRaisedException                    Traceback (most recent call last)
File ~/accelerate/src/accelerate/launchers.py:138, in notebook_launcher(function, args, num_processes, mixed_precision, use_port)
    137 try:
--> 138     start_processes(launcher, args=args, nprocs=num_processes, start_method="fork")
    139 except ProcessRaisedException as e:

File ~/miniconda3/envs/accelerate/lib/python3.9/site-packages/torch/multiprocessing/spawn.py:197, in start_processes(fn, args, nprocs, join, daemon, start_method)
    196 # Loop on join until it returns True or raises an exception.
--> 197 while not context.join():
    198     pass

File ~/miniconda3/envs/accelerate/lib/python3.9/site-packages/torch/multiprocessing/spawn.py:160, in ProcessContext.join(self, timeout)
    159 msg += original_trace
--> 160 raise ProcessRaisedException(msg, error_index, failed_process.pid)

ProcessRaisedException:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/zach_mueller_huggingface_co/miniconda3/envs/accelerate/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
    fn(i, *args)
  File "/home/zach_mueller_huggingface_co/accelerate/src/accelerate/utils/launch.py", line 514, in __call__
    self.launcher(*args)
  File "/tmp/ipykernel_1146463/1934231147.py", line 10, in train_loop
    accelerator = Accelerator()
  File "/home/zach_mueller_huggingface_co/accelerate/src/accelerate/accelerator.py", line 358, in __init__
    self.state = AcceleratorState(
  File "/home/zach_mueller_huggingface_co/accelerate/src/accelerate/state.py", line 542, in __init__
    PartialState(cpu, **kwargs)
  File "/home/zach_mueller_huggingface_co/accelerate/src/accelerate/state.py", line 137, in __init__
    torch.cuda.set_device(self.device)
  File "/home/zach_mueller_huggingface_co/miniconda3/envs/accelerate/lib/python3.9/site-packages/torch/cuda/__init__.py", line 350, in set_device
    torch._C._cuda_setDevice(device)
  File "/home/zach_mueller_huggingface_co/miniconda3/envs/accelerate/lib/python3.9/site-packages/torch/cuda/__init__.py", line 235, in _lazy_init
    raise RuntimeError(
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In [1], line 14
     11     print(accelerator.state)
     13 from accelerate import notebook_launcher
---> 14 notebook_launcher(train_loop, num_processes=2)

File ~/accelerate/src/accelerate/launchers.py:141, in notebook_launcher(function, args, num_processes, mixed_precision, use_port)
    139         except ProcessRaisedException as e:
    140             if "Cannot re-initialize CUDA in forked subprocess" in e.args[0]:
--> 141                 raise RuntimeError(
    142                     "CUDA has been initialized before the `notebook_launcher` could create a forked subprocess. "
    143                     "Please make sure no code ran initializes CUDA by checking `import torch; torch.cuda.is_initialized()`, "
    144                     "and ensure that no outside imports are causing issues after `notebook_launcher()` is called."
    145                     ) from e
    147 else:
    148     # No need for a distributed launch otherwise as it's either CPU, GPU or MPS.
    149     if is_mps_available():

RuntimeError: CUDA has been initialized before the `notebook_launcher` could create a forked subprocess. This likely stems from an outside import causing issues once the `notebook_launcher()` is called. Please review your imports and test them when running the `notebook_launcher()` to identify which one is problematic.

Note that in this case it is directly from an import, as earlier we check if cuda was initialized

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@sgugger it actually bypasses it somehow (not the best answer I know)... which is what makes this a super subtle bug. Because we already do:
+            if torch.cuda.is_initialized():
+                raise ValueError(
+                    "To launch a multi-GPU training from your notebook, you need to avoid running any instruction "
+                    "using `torch.cuda` in any cell. Restart your notebook and make sure no cells use any CUDA "
+                    "function."
+                )

            # torch.distributed will expect a few environment variable to be here. We set the ones common to each
            # process here (the other ones will be set be the launcher).
            with patch_environment(
                world_size=num_processes, master_addr="127.0.01", master_port=use_port, mixed_precision=mixed_precision
            ):
                launcher = PrepareForLaunch(function, distributed_type="MULTI_GPU")
                print(f"Launching training on {num_processes} GPUs.")
                try:
                    start_processes(launcher, args=args, nprocs=num_processes, start_method="fork")
                except ProcessRaisedException as e:
                    if "Cannot re-initialize CUDA in forked subprocess" in e.args[0]:
                        raise RuntimeError(
                            "CUDA has been initialized before the `notebook_launcher` could create a forked subprocess. "
                            "This likely stems from an outside import causing issues once the `notebook_launcher()` is called. "
                            "Please review your imports and test them when running the `notebook_launcher()` to identify "
                            "which one is problematic."
                        ) from e


Add feature gate DisableScalingClassesForShoots to assign Shoot kube-apiserver resource requests statically instead of using a ScalingClass

How to categorize this PR?

/area auto-scaling
/area control-plane
/kind enhancement
What this PR does / why we need it:
Changes how the initial resource requests for Shoot kube-apiservers are computed for Seeds which have HVPA enabled. Currently, this is done by assigning a ScalingClass based on the maximum Node count over all worker pools. As this wastes a lot of resources for Shoots which see little usage, this PR introduces a feature gate to switch to a static assignment for the initial resource requests.

Introduce a new feature gate DisableScalingClassesForShoots, in alpha state and defaulted to false
If the feature gate is enabled, kube-apiservers for new Shoots get a static amount of resources assigned initially
If the feature gate is disabled, kube-apiservers for new Shoots get assigned resources determined by the ScalingClass

Which issue(s) this PR fixes:
Fixes #7985
Release note:

Introduce a new alpha feature gate on `gardenlet`, called `DisableScalingClassesForShoots`. If turned on, initial resource requests for `kube-apiserver`s of shoot clusters running on seed clusters which enable the `HVPA` feature gate are assigned statically and no longer by a `ScalingClass` determined by maximum Node count. This helps to reduce resource waste for clusters with little usage.

/cc @andrerun

/retest

/retest

@voelzmo Can you please check failing integration test - https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/8003/pull-gardener-integration/1668160780123508736#1:build-log.txt%3A109

/retest

/retest

@voelzmo integration test will not pass please check https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/8003/pull-gardener-integration/1668160780123508736#1:build-log.txt%3A109 and accordingly adapt test.

/approve
changes look good except for https://github.com/gardener/gardener/pull/8003#issuecomment-1587158473

/retest


run_electrum_nmc not included in Windows binary?

Receiving a fatal error on Windows 10. The run_electrum_nmc script does not appear to be included.

Same issue using the installer, can't find the run_electrum_nmc file in folder.

Can you post the exact text and a screenshot of the error you're getting?

I did an initial look-through of the Windows build code and I don't see anything that would be likely to cause this symptom; a screenshot and exact text would be helpful in tracking down the bug.

Traceback (most recent call last):
File "run_electrum_nmc", line 359, in
File "electrum_nmc\daemon.py", line 348, in init_gui
ModuleNotFoundError: No module named 'electrum_nmc.gui.qt'

Hmm, with the installer version, can you provide a screenshot of the contents of the directory you installed it to?  (The directory that has the electrum_nmc and contrib subdirectories in it.)

Ah, hmm, I'm looking at the line that the traceback mentions.  It's definitely a piece of code that I fiddled with recently.  Is there a file called daemon.py anywhere in the installation folder (using the installer)?  If so, can you look at line 348 of it, and try changing the electrum_nmc on that line to electrum?  There are two instances of it on that line; try changing each one individually, and also try changing them both, and let me know which sets of changes make a difference (if any do).

I can't find daemon.py unfortunately.

I can reproduce the dialog in the screenshot on my Windows machine.  What exactly did you do to produce the traceback that you pasted?

Interestingly, the Python zip works fine on Windows, so I assume there's an issue with the .exe packaging scripts somewhere.  Not sure exactly where though.

(this is where it gets annoying for you i think )using bash ./electrum-nmc-nc3.3.6.exe

hmm, I wonder if the PE header trick I used for certutil will be able to approximate the behavior you're achieving via bash....

For context, I had used <IP_ADDRESS>.1.(1) on this machine with no issue.

I am somewhat indisposed so may take time to respond. Apologies.

Well, I figured out how to extract a PyInstaller exe file, and I definitely see something wrong.  Electrum-NMC's exe file doesn't have any of the Python modules for the electrum_nmc.gui package, whereas upstream Electrum (same version) does have them.  I have no idea why exactly they're missing though.  This certainly explains the traceback you got.

I have a lead, but it's not a very likely one.  Will hopefully have more info on whether that lead has panned out in circa 30 minutes.

Pursuing another couple of leads...

Relatively sure I see what's wrong.  Fix will hopefully be out tomorrow.


total is not set if there is an error

Error events are not listened for, so if the request or response terminates due to an error, total will not be set.

I think you also need to listen for response's error event. For example, if there's an error reading the response body.

Fixed in 9fac585ca5863be0fec278dd2d772fd39f9e32a6

Was there a reason you wanted to patch emit? prependListener allows you to move to the front of the line if other listeners have already been attached.

Was there a reason you wanted to patch emit?

Yup. If someone added prependOnceListener after we did so, origin.listenerCount('error') would return 0. But we could fix that by listening for the newListener event. Wait, but we need the count, right? So we need to listen for the removeListener event too. It looks like a mess [a bit]. The whole implementation would look something like this:
let errorListeners = origin.listenerCount('error');

origin.on('newListener', event => {
	if (event === 'error') {
		errorListeners++;
	}
});

origin.on('removeListener', event => {
	if (event === 'error') {
		errorListeners--;
	}
});

origin.once('error', error => {
	...
	if (errorListeners === 0) {
		throw error;
	}
});

I just feel better with the current solution, but if you want to send a PR, feel free (it's ok too) :)

Pardon my very late response, but the approach I had shown was incorrect. I've updated the post above. Please take a look at it! :)


embedding visualization

Signed-off-by: ChengZi<EMAIL_ADDRESS>

Codecov Report

Merging #1426 (dd93b71) into main (70762ad) will increase coverage by 0.21%.
The diff coverage is 73.68%.

:exclamation: Current head dd93b71 differs from pull request most recent head 001a5db. Consider uploading reports for the commit 001a5db to get more accurate results

@@            Coverage Diff             @@
##             main    #1426      +/-   ##
==========================================
+ Coverage   68.40%   68.62%   +0.21%
==========================================
  Files         296      302       +6
  Lines       16265    16678     +413
  Branches     2646     2702      +56
==========================================
+ Hits        11126    11445     +319
- Misses       4400     4478      +78
- Partials      739      755      +16

Impacted Files
Coverage Δ

...dels/multiscale_vision_transformers/create_mvit.py
38.46% <ø> (+1.42%)
:arrow_up:

...ee/models/visualization/embedding_visualization.py
73.68% <73.68%> (ø)

towhee/utils/git_utils.py
51.89% <0.00%> (-0.88%)
:arrow_down:

towhee/hub/repo_manager.py
23.96% <0.00%> (-0.82%)
:arrow_down:

...whee/models/bridgeformer/bridge_former_training.py
51.54% <0.00%> (ø)

towhee/models/bridgeformer/bridge_former.py
53.12% <0.00%> (ø)

...odels/bridgeformer/bridge_former_training_block.py
92.67% <0.00%> (ø)

towhee/models/bridgeformer/bridge_former_utils.py
50.00% <0.00%> (ø)

towhee/models/bridgeformer/__init__.py
100.00% <0.00%> (ø)

towhee/models/clip/clip.py
87.20% <0.00%> (+1.77%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 70762ad...001a5db. Read the comment docs.

/lgtm
/approve


ClipOval for native view is not working on Android & IOS

I'm trying to use the ClipOval to clip in circle my native views from PlatformView which is not working at all on Android and partially on IOS :
ANDROID :

IOS :

The native view is on top in the screenshots.
The native view (alpha blue color) is the child of a pink container which is Clipped and we can see that for both the render is not correct.
For Android the square is full and for IOS partially.
Notice this example use ClipOval but this is also not working with CustomClipper in circle in flutter.
A workaround would be to create the clip itself inside the native code for both which I want to avoid at any cost for performance reasons.
The full example code is ready to be cloned and run
A fix for this would be great!
Thanks everyone
FLUTTER DOCTOR
[✓] Flutter (Channel stable, 3.3.4, on macOS 12.6 21G115 darwin-x64, locale fr-FR)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.2)
[✓] VS Code (version 1.71.2)
[✓] Connected device (2 available)
[✓] HTTP Host Availability

• No issues found!

Triage report
I can reproduce this issue on Master (3.5.0-5.0.pre.1). Tested on Android 13 emulator (Pixel 5) and iOS 16 simulator (iPhone 14 Pro Max).

These are unhandled or partially handled mutator stack items.
The issue on iOS is the same as https://github.com/flutter/flutter/issues/35062 and there is a potential fix linked. cc @jmagman
And the Android mutator stack entry isn't implemented at all. cc @stuartmorgan

This is unfortunately still the case using flutter 3.7 and impeller


Create fatecsenairs.txt

Description of a local university at Porto Alegre / RS / Brazil under the domain .org.br.

Are you sure about this domain (fatecsenairs.org.br)? I can't find any MX record for it...

PR has been updated to match a domain with a MX record, appears to be a 'college' of some sort providing post-18 education.
https://translate.google.com/translate?sl=auto&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fwww.senairs.org.br%2F&edit-text=

Hello, actually Senai is an education institution that goes from the basic technical/instrumentation education and technical high school to college and graduation.

@gugabrand Sorry for confusion; I was implying that I believe it meets the criteria to be included :smile:


Add Flash Question Answering tutorial

Before submitting

[x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?

What does this PR do?
Fixes # (issue).
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

@karthikrangasai mind checking the failing tests? :]

@karthikrangasai could you pls remind me what I have promised here to do to unblock it? :rabbit:

Hello @Borda ,
We had spoken about

the Kaggle API key getting outdated and updating it
enrolling the lightinig kaggle testing account into the competition.

Downloading and sharing the dataset again might have licensing issues.
After this, I don't think we had arrived to a conclusive solution.

the Kaggle API key getting outdated and updating it

just re-generated Key and the same issue, also check that the user approved dataset conditions... need to check deeper


strtoull(3) usage walk-around for the libc of NetBSD

strtoull(3) is a part of POSIX 2008 and included with: _POSIX_C_SOURCE 200809L
The current NetBSD LibC (as of pre 8.x release) enables strtoull(3) with:

_ISOC99_SOURCE
STDC_VERSION >= 199901L
_NETBSD_SOURCE
__cplusplus >= 201103L

Go for -std=c99.
Please merge this with radare2.

NetBSD issues reported upstream
http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=49685

The NetBSD team reports that this is fault of missing C99 defined here. So my issue is valid not just NetBSD. I will update the revision log.

I think js0n.c uses some c99 thing too, so its ok to enforce that i guess

On 22 Feb 2015, at 14:41, krytarowski<EMAIL_ADDRESS>wrote:
The NetBSD team reports that this is fault of missing C99 defined here. So my issue is valid not just NetBSD. I will update the revision log.
—
Reply to this email directly or view it on GitHub.

Thanks!


Download Citrix sources using manifest "source" field, rather than with wget

The download links for Citrix are dynamic, so they can't be hard-coded in the manifest. They have to be obtained at build-time by scraping the downloads page for a unique link. However, I don't know take that unique link and use it in the module "source" field, so instead the installers are downloaded via wget (which in turn requires giving the build environment network permissions). I think it'd be best to download the installers via the "source" field, if possible.

On second thought, the build environment would still need network permissions to do obtain the download links, even if installer downloads are able to be done via manifest "source" field.


DirectSoundOut.PlaybackStopped Event Not Firing at End of Stream

I read on the wiki that you can use the PlaybackStopped event to detected whether a WaveStream (WaveFileReader) has reached the end. When I play the audio and wait for the end, it does not fire. I'm not sure if I am doing something wrong, or if this is bug. Here is my AudioPlayer code.
public class AudioPlayer : IDisposable
{
    // Events
    public event EventHandler<StoppedEventArgs> PlaybackStopped;

    /// <summary>
    ///     Gets the current time in the song.
    /// </summary>
    public TimeSpan CurrentTime => Clock.Elapsed;
    /// <summary>
    ///     Gets whether the audio has reached the end.
    /// </summary>
    public bool IsFinished => WaveStream.CurrentTime == WaveStream.TotalTime;

    /// <summary>
    ///     Gets the output controller.
    /// </summary>
    private DirectSoundOut Output { get; set; }
    /// <summary>
    ///     Gets the WaveStream in use.
    /// </summary>
    private WaveStream WaveStream { get; set; }
    /// <summary>
    ///     Gets the Stopwatch used for keeping track of
    ///     how far in the WaveStream the AudioPlayer is.
    ///
    ///     This is used instead of WaveStream.CurrentTime,
    ///     so the SampleStream can get times between the latency.
    /// </summary>
    private Stopwatch Clock { get; } = new Stopwatch();

    /// <summary>
    ///     Loads audio from a file into the AudioPlayer.
    /// </summary>
    /// <param name="filename">The location of the file.</param>
    public void LoadAudio(string filename)
    {
        UnloadAudio();

        WaveStream = new WaveFileReader(filename);
        Output = new DirectSoundOut();

        Output.Init(new WaveChannel32(WaveStream));
        Output.PlaybackStopped += (object sender, StoppedEventArgs e) =>
        {
            if (e.Exception != null) Console.WriteLine(e.Exception.GetType() + "\n" + e.Exception.StackTrace);
            Console.WriteLine(WaveStream.CurrentTime + " == " + WaveStream.TotalTime + "?");

            PlaybackStopped?.Invoke(this, e);
        };
    }
    /// <summary>
    ///     Disposes of the resources from the current audio file
    ///     and playback controller.
    /// </summary>
    public void UnloadAudio()
    {
        if (Output != null)
        {
            Output.Stop();
            Output.Dispose();
            Output = null;
        }
        if (WaveStream != null)
        {
            WaveStream.Dispose();
            WaveStream = null;
        }
        Clock.Reset();
    }

    /// <summary>
    ///     Begin playback.
    /// </summary>
    public void Play()
    {
        if (Output.PlaybackState == PlaybackState.Stopped) Clock.Reset();
        if (Output.PlaybackState != PlaybackState.Playing)
        {
            Output.Play();
            Clock.Start();
        }
    }
    /// <summary>
    ///     Pause playback.
    /// </summary>
    public void Pause()
    {
        Clock.Stop();
        Output.Pause();
    }
    /// <summary>
    ///     Stop playback.
    /// </summary>
    public void Stop()
    {
        Clock.Stop();
        Output.Stop();
    }

    /// <summary>
    ///     Disposes of all resources.
    /// </summary>
    public void Dispose()
    {
        UnloadAudio();
    }
}

Something strange comes up when I call LoadAudio and audio is already playing. The PlaybackStopped event is fired with a NullPointerException and the message below is 0:00:000 == <Time of New Audio>. The new audio will still play.

It's not stopping because you're using WaveChannel32 which produces never-ending audio by default. There's no real need for it so I suggest you remove it.
On the unload, its generally a good idea not to restart playback until you've received the PlaybackStopped event. Alternatively, try WaveOutEvent instead of DirectSoundOut which might be more reliable.

Doing what you suggested gave me the results I was looking for. Thanks for getting back to me so quickly.


opentracing-cpp: 1.5.1 -> 1.6.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/opentracing/opentracing-cpp/releases.
meta.description for opentracing-cpp is: "C++ implementation of the OpenTracing API"
meta.homepage for opentracing-cpp is: "https://opentracing.io"
meta.changelog for opentracing-cpp is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.6.0 with grep in /nix/store/7fp174pp3cpijmk5jwa61ymkbjr6qjax-opentracing-cpp-1.6.0

directory tree listing: https://gist.github.com/d6a511dd809235a63bd5d4c4fe39faf9

du listing: https://gist.github.com/bd93b26a1be2aaa18b6bfcf0a8d51d15

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
lightstep-tracer-cpp
opentracing-cpp

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/7fp174pp3cpijmk5jwa61ymkbjr6qjax-opentracing-cpp-1.6.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A opentracing-cpp https://github.com/r-ryantm/nixpkgs/archive/9014113a103faffe2284ac96a238c7873777e7ab.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/7fp174pp3cpijmk5jwa61ymkbjr6qjax-opentracing-cpp-1.6.0
ls -la /nix/store/7fp174pp3cpijmk5jwa61ymkbjr6qjax-opentracing-cpp-1.6.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review 1

1 package failed to build:

lightstep-tracer-cpp

1 package built:

opentracing-cpp

Maintainer pings
cc @rbvermaa for testing.

This is a semi-automatic executed nixpkgs-review which is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 104567 run on x86_64-linux 1

1 package failed to build and are new build failure:

lightstep-tracer-cpp: log https://termbin.com/me8w

1 package built:

opentracing-cpp

This is a semi-automatic executed nixpkgs-review which is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 104567 run on x86_64-linux 1

1 package failed to build and are new build failure:

lightstep-tracer-cpp: log https://termbin.com/me8w

1 package built:

opentracing-cpp

Still blocked on lightstep-tracer-cpp switching to the 1.6.x OpenTracing release.

looks like no updates for 3 years: https://github.com/lightstep/lightstep-tracer-cpp/pull/215

looks like no updates for 3 years: lightstep/lightstep-tracer-cpp#215

https://github.com/lightstep/lightstep-tracer-cpp/commit/0dcb9160e9298191199896870c84547dfb843623

❗ This instrumentation is no longer recommended. Please review documentation on setting up and configuring the OpenTelemetry C++ API and SDK for more information on migrating to updated instrumentation.

Oh I just missed that commit. So can we maintain two versions of opentracing-cpp, 1.6.0 (or newer if any) and 1.5.1 for lightstep-tracer-cpp or mark lightstep-tracer-cpp broken (Perhaps this is a more aggressive option, but remove lightstep-tracer-cpp since it seems as if the package is not referenced by any other packages in this repository)?

The problem is way worse:
Opentracing was essentially abandoned.
It makes few to no sense to keep it.
However, too many files inside Nixpkgs depend on it. I am trying to remove some of them, but others are machine-generated.

Since this package is now rendered useless, I will merge it just because.

@ofborg build opentracing-cpp


PageInject changes the absolut UI5 element ID

Have a look at the picture:

The page reload changes the id's.
Relative Id's shouldn't be used for testing.
Therefore i made the page-reload optional so the user can decide which way he wan'ts to go.


Ignore spurious gamma emission from nuclear de-excitations included in betaminusdecays.txt

These appear to be related to nuclear excited-state transitions, which are not handled by artis. Ignoring these gives good agreement with the total energy release rates (Qdot) from the network calculation.
Should not be significant for the published model, since (with uranium data fixed as it was) the abnormally high gamma emission is only seen after one day when gamma ray thermalisation is very inefficient. Will be more significant for higher-mass models.

Running this on the Just+2023 sym-n1-a6 model shows no significant difference. energy release is still exceeding Qdot and should be fixed.


Rancher Harvester Dashboard Cypress test

I added in the tests to validate that you can load the dashboard from the link to Harvester in Rancher.
I also fixed some missing semicolons and extended out the dashboard page object so that we can validate that the header has loaded.

#1213 https://github.com/harvester/tests/issues/1205


cups may need systemd network-online.target instead of network.target

I'm not sure what the current policy of duplicate entries to the apple/cups issus is but for discussion I reopen a duplicate of https://github.com/apple/cups/issues/5709 here as well also to discuss if more Linux distributions are affected.
Running Arch Linux with cups built from the OpenPrinting/Cups fork of todays master I still can't reach my network cups server using its webinterface and it doesn't offer the printer to the network after fresh reboot. A "systemctl status cups" shows it active and properly started though. A manual systemd restart cups.service solves it. My local solution using in Arch Linux is to further modify the service file
--- cups-2.3.0/scheduler/cups.service.in        2019-08-23 17:19:38.000000000 +0200
+++ cups-2.3.0/scheduler/cups.service.in.new    2019-08-25 20:20:25.805339727 +0200
@@ -1,8 +1,9 @@
 [Unit]
 Description=CUPS Scheduler
 Documentation=man:cupsd(8)
-After=network.target sssd.service ypbind.service
+After=network-online.target sssd.service ypbind.service
 Requires=cups.socket
+Wants=network-online.target

 [Service]
 ExecStart=@sbindir@/cupsd -l

Acording to https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ this shouldn't need be needed and I'd prefer to find out why network.target isn't enough to make cups properly working here.
Attaching here a cups debug error_log.txt using network.target make the start failing and the working
error_log_working.txt
Maybe you have an idea why network.target isn't working for me while it does seem to do so in different distributions.

Hi @andyrtr ,
from a brief look in logs you use a remote access (you connects to <IP_ADDRESS> and there is a message - 'Remote access is enabled.')- which means you have cupsd set to connect to cupsd on remote machine. It is set probably via ServerName/ServerAlias directives in /etc/cups/client.conf or /etc/cups/cupsd.conf.
And based on network.target and network-online.target definition - the former indicates only network management stack is up. Cupsd usually listens on localhost:631 so it needs only port 631 to be prepared, so network.target suffices.
On the other hand, since you enabled a remote access, you need to have a network really working. The definition of 'working network' here is based on network management software (f.e. NetworkManager), so network-online.target is reached when this specific software is started and it configured the network (mostly by assigning IP).
Putting network-online.target into cups.service by default would delay startup (especially when you aren't in any network), so I wouldn't advice to make it default for all.
However, if you really need the remote access and cupsd working this way, you can create /etc/systemd/system/cups.service.d/<name>.conf drop-in with:
[Unit]
After=network-online.target
Wants=network-online.target

and then do systemctl daemon-reload. You can check the result via systemctl cat cups.

The remote system with cupsd running has IP <IP_ADDRESS> and is setup with
Listen <IP_ADDRESS>:631 and Allow 192.168.1.* in Location directivities in cupsd.conf. Isn't this the common way to setup a cups server to offer its service to the local network? Then it should work out of the box with files shipped by the distribution. I can confirm that the cups webinterface can be accessed using "links http://localhost:631" via ssh on that system with only network.target. But it needs either a service restart or the network-online.target enabled to allow the local network access working.
Either my setup is not correct to work the way it should with only network.target or I would expect cupsd to detect any later change to the network and allow the desired remote access.
I haven't seen any delay reported by Arch users with network-online.target used. But I guess it's more a workaround.

Aha, I guess I misunderstood the issue - I thought you have a problem on the client, not on the server.
Then it makes more sense. We have a downstream patch in Fedora (planning to send it upstream in a while) to use IP_FREEBIND socket option which fixes this behavior according to (systemd documentation)[https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/]

@andyrtr The "normal" way to enable remote access (for printer sharing, etc.) is to listen on the "any" address (*), either using the Listen directive or the Port directive:
Listen *:631

Port 631

Listening to a specific address can lead to the kind of failures that you are seeing. As I recall, there was an old feature request to be able to listen to all addresses for a given network interface, and to adapt to interface changes, but that request was never implemented due to a lack of OS support. Even now, using DBUS notifications would only work on platforms that use systemd-networkd (often true for desktop/server systems, not so much for embedded systems/NAS boxes that care about networking standards conformance...)

Aha, I guess I misunderstood the issue - I thought you have a problem on the client, not on the server.
Then it makes more sense. We have a downstream patch in Fedora (planning to send it upstream in a while) to use IP_FREEBIND socket option which fixes this behavior according to systemd documentation.

@zdohnal - the clients can't access the servers webinterface and can't print to the servers printer queues until the cups server on the server has been restarted.
@michaelrsweet: This is confusing to me.
Listen <IP_ADDRESS>:631 can be accessed by clients after the daemon restart but
with either
Listen *:631 or Port 631 or both together the server cannot be reached even after daemon restart.

@zdohnal - I guess it's this patch here: https://src.fedoraproject.org/rpms/cups/blob/master/f/cups-freebind.patch
I'm going to try that one out if it helps.

@zdohnal - thanks for the hint. this patch makes it work. Now network.target + Listen <IP_ADDRESS>:631 works as intended. Feel free to push the patch.
@michaelrsweet - please reconsider the new release version. Our distribution package version comparison tool "vercmp" reports a version donwgrade 2.3.3 > 2.3.3op1. Why don't you use 2.3.4 or 2.4.0 ?

@andyrtr The issue with that patch is that it is sometimes useful to get an error if you try to bind to the wrong address. Usually that is an error (or a mistake, if you will), so it should require an explicit override.
Using Listen *:631 or Port 631 should work in all cases, unless you have some sort of security setting/firewall in place that prevents incoming connections without an explicit source (listen) address.
As for the versioning, OpenPrinting CUPS isn't replacing Apple CUPS. And in fact Apple does have a v2.3.4 that ships with their newest version of macOS (still not pushed to Github...) I'll bring this up amongst the OpenPrinting maintainers - an alternative would be a fourth version number (e.g. <IP_ADDRESS> instead of 2.3.3op1) but I know there are other environments that might have a problem with that... :/

Using Listen *:631 or Port 631 should work in all cases, unless you have some sort of security setting/firewall in place that prevents incoming connections without an explicit source (listen) address.

Got it. Listen *:631 doesn't work when there's also Listen localhost:631 before in the configuration file. Maybe man cupsd.conf is misleading here saying
Listen /path/to/domain/socket
           Listens to the specified address and port or domain socket path for connections.  **Multiple Listen directives can be  provided**
           to  listen on multiple addresses.  The Listen directive is similar to the Port directive but allows you to restrict access to
           specific interfaces or networks.

@andyrtr Maybe this should be made more explicit, however the issue of multiple Listen lines isn't that they aren't supported, it is just that Listen *:631 binds to all IP addresses (so you can't have a duplicate address).

Reopening to track documentation change.

[master fc9167da3] Update cupsd.conf man page to talk about Listen * vs Listen address (Issue #53)

@michaelrsweet Doesn't setting Port 631 in cupsd.conf have a downside of opening the port for wide internet if the server has a IP which is reachable via wide internet?
I know this can be narrowed down by setting Allow from to a proper subnet, but still clients from outside can connect to the port and send a bunch of requests, which will keep cupsd busy when throwing them away, right?
I started using 'Listen :631' to prevent possible requests from outside since I read the article about how many print servers are reachable on the wider internet (about half a year ago or so), I thought it is the correct solution for this + setting Allow from properly.

@zdohnal The vast majority of systems running CUPS are going to be behind a router that does NAT and provides basic firewall capabilities. That combined with the default Deny all/Allow @LOCAL ACLs will quite effectively limit access to the LAN.
Limiting the Listen to the address of the (only) network interface in that configuration doesn’t make it more secure since remote access is being limited by the router and not by cupsd. Then it comes down to whichever holes you decide to configure, e.g., so-called DMZ hosts.
Indeed, a far more common problem is not making things open enough and dealing with any onboard firewall configuration underneath cupsd...
DoS attacks from the Internet are only a problem if you are “brave” enough to provide a public-facing print server. In my experience those public-facing servers are a result of misconfiguration or trying to do everything with a single server. Using Listen with an address in those situations won’t help unless the server in question has multiple physical network interfaces.

DoS attacks from the Internet are only a problem if you are “brave” enough to provide a public-facing print server. In my experience those public-facing servers are a result of misconfiguration

By misconfiguration you mean network's misconfiguration right? Or cupsd's?

or trying to do everything with a single server.

If that's the case, is there any other mitigation than other network interface and configuring cupsd to listen on only that selected interface? (meaning settting cupsd to listen on IP which is assigned to the interface)

Using Listen with an address in those situations won’t help unless the server in question has multiple physical network interfaces.

@zdohnal By misconfiguration I mean the network (router), or the server, or cupsd. It can happen anywhere, and usually the first step is "I can do this all on a single server with a single physical interface". The first rule of network security is that you don't connect a server to a network unless you want it accessible via that network.
The place for Listen is on multi-interface servers, e.g.:
LAN (private IP) <--eth0--> Server <--eth1--> Internet/WAN (public IP)

If you had such a server (and I am by no means recommending this configuration!) you can configure cupsd to only listen on its private IP over eth0 with appropriate allow/deny lines and server firewall configuration. Provided you don't get caught by side-channel or proxy attacks (...) this will effectively limit cupsd traffic to the LAN. Mind you, the server itself can be attacked from the public-facing interface, potentially exposing job history/files and other sensitive information, which is why you don't use this configuration if you care about security.
Things like Docker containers, virtual machines, and other VPS solutions are more commonly used to provide isolation and flexibility, and there you are back to using "Listen *:631" or "Port 631"...
(I have seen Listen used to provide multiple cupsd instances on a single server with multiple interfaces/addresses to support different organizations/groups and isolate printers and jobs, but these days you can spin up a Docker container for that much more easily and without the hassle of creating multiple cupsd configurations on a single OS image...)

Thank you for the explanation! Then the baseline is 'don't do everything on one server if the server is public one, or you may experience various problems.' :)


Add support for reconfig cluster membership operation (issue #234)

Signed-off-by: Raul Gutierrez S<EMAIL_ADDRESS>

this passes for me locally fwiw (given the tests aren't happy in travis)

This is really exciting- thanks for adding it!


Remove outdated Travis CI & Jenkins configuration

Motivation
The repository contains outdated Travis CI & Jenkins configuration files which can cause confusion. Since the configuration isn't maintained, it's better to remove it.


Authentication error

Hi,
I get the following error message:
https://login.live.com/err.srf?lc=1033#error=unauthorized_client&error_description=The+client+does+not+exist.+If+you+are+the+application+developer%2C+configure+a+new+application+through+the+application+management+site+at+https://manage.dev.live.com/.
This is my configuration in https://apps.dev.microsoft.com

What could be the problem?

Can you show the code you're using to get this error? Make sure to remove client secret information.

import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
redirect_uri = "http://localhost:8080/"
client_secret = "34eOf5NSECRETAduP5so8Jn"
client = onedrivesdk.get_default_client(client_id='your_client_id',
scopes=['wl.signin',
'wl.offline_access',
'onedrive.readwrite'])
auth_url = client.auth_provider.get_auth_url(redirect_uri)
#this will block until we have the code
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
client.auth_provider.authenticate(code, redirect_uri, client_secret)

In this line:

client = onedrivesdk.get_default_client(client_id='your_client_id',
scopes=['wl.signin',
'wl.offline_access',
'onedrive.readwrite'])

you need to replace the fake string 'your_client_id' with your actual client ID. Try this:
client = onedrivesdk.get_default_client(client_id='0000000044194611',
scopes=['wl.signin',
'wl.offline_access',
'onedrive.readwrite'])

Additionally, your redirect_uris do not match. You need to either update the RedirectURI on your app registration page (the page you screenshot), or you need to update the redirect_uri variable to match what's on your app registration page.


update packages

:x: Build TheIdServer 3.0.1-fix-dependencies1-0069 failed (commit https://github.com/Aguafrommars/TheIdServer/commit/69beeb7688 by @)

:white_check_mark: Build TheIdServer 3.0.1-fix-dependencies1-0070 completed (commit https://github.com/Aguafrommars/TheIdServer/commit/b5da52c89f by @)


Update docs on docker-based installation

Closes #1657
This small fix addresses some shortcomings in the docs for the
docker installation. Thanks to feedback from @tesnimab for pointing
out the issues.

Auto merge enabled, so please don't approve if you'd like to see some changes.


Support uploading symlinked folders

We can do this now since https://github.com/Azure/azure-storage-azcopy/issues/1174 was fixed. Here's the code in question:
https://github.com/microsoft/vscode-azurestorage/blob/0d7252fee3b4474ae9c4ed2ef15f3d1292e7aecb/src/commands/azCopy/azCopyTransfer.ts#L36-L38

This seems to be working for me. I can upload symlinked folders to a File Share from macOS. Symlinks were created using ln.

Tried it with Windows, using mklink /J SymLink OriginalFolder. Seems to upload alright for me as well.

Also worth mentioning, but this was also closed: https://github.com/Azure/azure-storage-azcopy/issues/1174

We should add a test case in OneNote for this


Add ability to password protect visibility of fields when inside an entity

Sometimes there are fields that are sensitive.  Fields that should not be easily accessed or viewed.  This could be in bulk or in on an individual basis.
Even if a user has permissions to view these types of fields, as a general policy for security and sensitivity of the data, there is a common scenario where the field should not be visible unless explicitly requested by the user.  Common examples could be: Credit Card numbers, SIN/SSN, verbal code phrases, and other data that can compromise a individual.
A field should be able to be configured to be "Sensitive" (or some other word presenting this idea) that restricts the viewing and access of the field:

Could restrict Export to CSV or request user to Re-Auth if they want to export this data.
When viewing a Entity Record the field should be masked until the user "unlocks" the field and has to re-auth in order for the field to load.
When viewing this field in a data grid, the field should be masked unless the user Re-Auth's to expose that field.

numbers 1 and 3 above i think are more sensitive use cases. But number 2 is a perfect use case for having the secondary auth requirement.  Example if there was a SIN/SSN number against a contact that field should not be visible unless the user explicitly "unlocks" the field, and when the field is unlocked, that should generate a log.
Over time this could be extended for extra security/audit features such as a user is only able to unlock so many fields per hour, day, between the hours of X and Y, etc.  Or if they unlock more than X fields in a given amount of time then they are flagged for review.
This likely has some carry over with the current ACL work being done at the field level #336 #337
edit: typo fix for ALC to ACL.

Thanks for this request, we already added similar item to next release backlog and planning to enable fields encryption on storage level. We'll extend the scope with view options as well.

👍

This in 2.0 release?

No, next one will be 1.10

@StephenOTT unfortunately we had to reconsider our plans, so this won't be released in 1.10, but we'd still like to ship it by the end of the year.


USAi Labs AUDACITY

without videos

Awesome, looks great, @JHPHELAN ! Only thing is that the pull request contains two videos of almost 100MB which makes cloning the repo slow. You can drag and drop a video so it's hosted on Github's servers rather than include it in the source code. To do that, navigate to the file in your branch (link, select the pencil button on the top right to edit the file, then drag and drop the video in place. I'd only add one so we can have many builds without the page loading slowly.
Since you have 4 images, you can display those in a 2x2 grid using the table rather than a 4x1 like you have now, but I can help with that if you don't know how.

@Achllle Following examples moved images to 2x2 and made link to Youtube as video was too large to drop.  Preview looks ok, hope it translates.

I removed the video files and did some formatting on the table. I also created a thumbnail for the youtube video rather than just displaying the link. I'll merge this in, if you want to make additions, you're welcome to make another PR!

Thanks for the clean-up!  Looks good.


Problem with select fields attributes generation in django admin

I'm starting with django-parsley, it's a very nice app to develop django projects.  It's very easy to configure and to use. But I have a problem using django-parsley to generate the client parsley validation in the django admin, I have this bug with django 1.6.1 and 1.6.7. The problem is related with select field parsley generation, all input fields are generated correctly, and  the parsley.js validation works fine, but the data-parsley attribute of the select fields are missing.
I debugged for long time the django source and the html tags generated for the select fields don't contain the data-parsley attributes.
For now, I'm not using django-parsely for the django admin in my applications because not all fields are validated correctly.
I've searched online and I can't find any other developer comment regarding this problem. If anybody have this problem, or is able to find the solution, I'll be happy to use it.

@aaboffill Which attribute of select field is missing?
I tested the following with Django 1.6 and it seems to work for me:
First test using "python manage.py shell"
from parsley.tests.forms import FormWithCustomChoices
f = FormWithCustomChoices()
print f.as_ul()

Output

State: <select data-parsley-required="true" data-parsley-required-message="This field is required." id="id_state" name="state">
<option value="NY">NY</option>
<option value="OH">OH</option>
</select>

So you can see that data-parsley attributes are there
Second test in admin
I created a model in example/core/models.py
class SelectModel(models.Model):
choices = models.CharField(choices=[("1", "1"), ("2", "2")], max_length=1)

I created a ModelAdmin for this in example/core/admin.py
class SelectAdmin(ParsleyAdminMixin, admin.ModelAdmin):

    class Meta:
        model = SelectModel

    class Media:
        js = (
        "//code.jquery.com/jquery-latest.min.js",
        "parsley/js/parsley.min.js",
        "parsley/js/parsley.django-admin.js"
        )

admin.site.register(SelectModel, SelectAdmin)

At http://localhost:8000/admin/core/selectmodel/add/, if I try to "Save" the form without selecting a choice, then parsley.js correctly raises an error "This field is required."


Implement Redis 7 command LATENCY HISTOGRAM

Implement the Redis 7 command LATENCY HISTOGRAM as specified at https://redis.io/commands/LATENCY-HISTOGRAM.

Closing as this is auto generated. It has no value in a client library. This command is meant for cli users.


Integration with kedro: serve as the DataSet implementation and integrate with the scheduler plugin

Describe your problem
as titled.
Additional context
Add any other context about the problem here.

Implemented in #1287 and #1423.


Cache architecture information

Since we call arch() all over the place, it is better to get it once,
report on it and cache the value for future use.

lgtm

@dagwieers I think this breaks ARM tests.

I made a pull request that should fix this caching problem with unit tests: https://github.com/emilsvennesson/script.module.inputstreamhelper/pull/322


(Population Report) City & non City Populations

As an organisation we need the population of people, people living in cities, and people not living in cities for the following locations:
a continent
a region
a Country

As someone who is developing properties in a country, I can view how many people live in cities compared to those not living in cities in order to make detailed market research for potential property developments.


[FEATURE REQ] Samples for AppService Deployment

Library name
Azure.ResourceManager.AppService
Please describe the feature.
We have samples for website deployment in Track 1 SDK, that is based on a fluent style SDK, it is very important to have the same samples for track 2 given the fact that deploy an webapp in appserivce is  a hero scenario and little bit complex by directly calling our SDK

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'App Services:0.5455895,ARM:0.06951551,Extensions:0.06403308'

Do you have a first example to share or not yet?

@TechWatching  we released our first stable version of AppService package last month and we decide to have some wrapped method for the deployment, it is in discussion and our samples would base on that

@xboxeer Are the new wrapped methods for deployment available or not yet?

@TechWatching not yet implemented since there is some higher priority features in our todo list. meanwhile we are also batch generating SDK samples and publishing them in Azure Rest API reference site, for example: https://learn.microsoft.com/en-us/rest/api/cdn/endpoints/create?tabs=dotnet#examples, you can search in this site to find the samples

I have been trying to figure out how to deploy to an existing app service as well, is this something that is being worked on and when do you expect it to be available?

@xboxeer Any news? Are the new wrapped methods for deployment in the backlog?


[YouTube] Add "most replayed" aka heatmap data

Checklist

[X] I'm reporting a site feature request
[X] I've verified that I'm running yt-dlp version 2022.05.18 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region
Germany
Example URLs
https://www.youtube.com/watch?v=Z8Z51no1TD0
Description
Since Mai 18 YouTube started to roll out a feature adding a "most replayed" graph (internal name seems to be "heatmap") to the progress bar after they were experimenting with it for at least two years. (see this tweet)

The data of this new feature seems to be missing right now. I'm currently helping myself laboriously with a self-written web extension. I hope what I found out so far is somehow helpful:
YouTube's implementation on the web page is relatively straight forward and easy to extract (using an extension inject). A SVG tag on the page (svg.ytp-heat-map-svg 1000x100) contains a path defined with cubic Bézier curves (a C followed by three x,y pairs).
Every third x,y parameter after a C, where x ends with 5.0, is a usable data point:
x is the time stamp in percent. Just compute (x-5)/1000 for a value from 0 to 1.
y is the heat value for this time period. Just compute  (100-y)/100  for a value from 0 to 1.
Example:
Here's the SVG tag for this video:
<svg class="ytp-heat-map-svg" height="100%" preserveAspectRatio="none" version="1.1" viewBox="0 0 1000 100" width="100%" style="height: 40px;">
  <defs>
    <clipPath id="hm_1_0">
      <path class="ytp-heat-map-path" d="M 0.0,100.0 C 1.0,80.0 2.0,5.6 5.0,0.0 C 8.0,-5.6 11.0,57.4 15.0,71.8 C 19.0,86.1 21.0,71.3 25.0,71.8 C 29.0,72.2 31.0,73.0 35.0,74.0 C 39.0,75.1 41.0,76.3 45.0,77.0 C 49.0,77.7 51.0,77.0 55.0,77.5 C 59.0,78.0 61.0,79.0 65.0,79.7 C 69.0,80.4 71.0,80.9 75.0,81.0 C 79.0,81.1 81.0,80.3 85.0,80.2 C 89.0,80.1 91.0,80.3 95.0,80.7 C 99.0,81.0 101.0,81.5 105.0,81.8 C 109.0,82.2 111.0,82.1 115.0,82.4 C 119.0,82.6 121.0,82.7 125.0,82.9 C 129.0,83.1 131.0,83.5 135.0,83.4 C 139.0,83.4 141.0,82.7 145.0,82.4 C 149.0,82.2 151.0,82.3 155.0,82.3 C 159.0,82.2 161.0,82.7 165.0,82.1 C 169.0,81.4 171.0,79.3 175.0,78.9 C 179.0,78.6 181.0,80.2 185.0,80.4 C 189.0,80.5 191.0,79.7 195.0,79.7 C 199.0,79.6 201.0,80.4 205.0,80.1 C 209.0,79.8 211.0,78.5 215.0,78.2 C 219.0,77.9 221.0,78.5 225.0,78.8 C 229.0,79.0 231.0,79.6 235.0,79.6 C 239.0,79.6 241.0,78.6 245.0,78.7 C 249.0,78.8 251.0,79.8 255.0,80.0 C 259.0,80.2 261.0,79.8 265.0,79.8 C 269.0,79.8 271.0,79.9 275.0,80.1 C 279.0,80.2 281.0,80.7 285.0,80.6 C 289.0,80.5 291.0,79.7 295.0,79.5 C 299.0,79.3 301.0,79.9 305.0,79.8 C 309.0,79.7 311.0,78.9 315.0,78.8 C 319.0,78.8 321.0,79.2 325.0,79.5 C 329.0,79.8 331.0,80.2 335.0,80.2 C 339.0,80.3 341.0,80.0 345.0,79.8 C 349.0,79.6 351.0,79.4 355.0,79.3 C 359.0,79.2 361.0,79.3 365.0,79.2 C 369.0,79.2 371.0,79.1 375.0,79.0 C 379.0,79.0 381.0,79.1 385.0,79.1 C 389.0,79.1 391.0,79.3 395.0,79.0 C 399.0,78.7 401.0,78.0 405.0,77.7 C 409.0,77.5 411.0,77.7 415.0,77.8 C 419.0,77.9 421.0,78.3 425.0,78.3 C 429.0,78.3 431.0,78.1 435.0,77.9 C 439.0,77.6 441.0,77.3 445.0,77.1 C 449.0,76.8 451.0,77.0 455.0,76.7 C 459.0,76.4 461.0,75.7 465.0,75.5 C 469.0,75.4 471.0,76.3 475.0,76.1 C 479.0,75.9 481.0,74.9 485.0,74.4 C 489.0,73.8 491.0,73.6 495.0,73.3 C 499.0,73.0 501.0,72.8 505.0,72.6 C 509.0,72.5 511.0,72.8 515.0,72.5 C 519.0,72.1 521.0,71.2 525.0,70.9 C 529.0,70.6 531.0,70.7 535.0,70.8 C 539.0,70.9 541.0,71.3 545.0,71.2 C 549.0,71.1 551.0,70.2 555.0,70.3 C 559.0,70.3 561.0,71.3 565.0,71.6 C 569.0,71.8 571.0,71.5 575.0,71.5 C 579.0,71.6 581.0,71.7 585.0,71.8 C 589.0,71.9 591.0,72.0 595.0,71.9 C 599.0,71.8 601.0,71.2 605.0,71.1 C 609.0,70.9 611.0,71.0 615.0,71.1 C 619.0,71.1 621.0,71.3 625.0,71.4 C 629.0,71.5 631.0,71.7 635.0,71.7 C 639.0,71.7 641.0,71.6 645.0,71.4 C 649.0,71.3 651.0,71.1 655.0,71.1 C 659.0,71.1 661.0,71.2 665.0,71.4 C 669.0,71.6 671.0,72.1 675.0,72.0 C 679.0,71.9 681.0,71.1 685.0,70.9 C 689.0,70.7 691.0,71.2 695.0,71.0 C 699.0,70.9 701.0,70.4 705.0,70.2 C 709.0,70.0 711.0,70.1 715.0,70.0 C 719.0,70.0 721.0,70.1 725.0,70.0 C 729.0,69.9 731.0,69.6 735.0,69.7 C 739.0,69.7 741.0,70.2 745.0,70.5 C 749.0,70.8 751.0,71.0 755.0,71.0 C 759.0,71.0 761.0,70.5 765.0,70.6 C 769.0,70.7 771.0,71.4 775.0,71.6 C 779.0,71.7 781.0,71.3 785.0,71.4 C 789.0,71.4 791.0,71.6 795.0,71.9 C 799.0,72.1 801.0,72.7 805.0,72.8 C 809.0,73.0 811.0,72.7 815.0,72.8 C 819.0,72.8 821.0,72.7 825.0,73.1 C 829.0,73.5 831.0,74.0 835.0,74.8 C 839.0,75.7 841.0,76.5 845.0,77.1 C 849.0,77.8 851.0,77.7 855.0,78.1 C 859.0,78.5 861.0,78.7 865.0,79.0 C 869.0,79.4 871.0,79.6 875.0,79.8 C 879.0,80.0 881.0,80.0 885.0,80.2 C 889.0,80.3 891.0,80.2 895.0,80.5 C 899.0,80.9 901.0,81.1 905.0,81.8 C 909.0,82.5 911.0,83.1 915.0,84.1 C 919.0,85.1 921.0,85.9 925.0,87.1 C 929.0,88.2 931.0,89.4 935.0,90.0 C 939.0,90.6 941.0,90.0 945.0,90.0 C 949.0,90.0 951.0,90.0 955.0,90.0 C 959.0,90.0 961.0,90.0 965.0,90.0 C 969.0,90.0 971.0,90.0 975.0,90.0 C 979.0,90.0 981.0,90.0 985.0,90.0 C 989.0,90.0 992.0,90.0 995.0,90.0 C 998.0,90.0 999.0,88.0 1000.0,90.0 C 1001.0,92.0 1000.0,98.0 1000.0,100.0" fill="white" fill-opacity="0.6"></path>
    </clipPath>
  </defs>
  <rect class="ytp-heat-map-graph" clip-path="url(#hm_1_0)" fill="white" fill-opacity="0.2" height="100%" width="100%" x="0" y="0"></rect>
  <rect class="ytp-heat-map-hover" clip-path="url(#hm_1_0)" height="100%" x="0" y="0"></rect>
  <rect class="ytp-heat-map-play" clip-path="url(#hm_1_0)" height="100%" x="0" y="0"></rect>
</svg>

...and boiled down data:
"heatmap": [[0,1],[0.01,0.282],[0.02,0.282],[0.03,0.26],[0.04,0.23],[0.05,0.225],[0.06,0.203],[0.07,0.19],[0.08,0.198],[0.09,0.193],[0.1,0.182],[0.11,0.176],[0.12,0.171],[0.13,0.166],[0.14,0.176],[0.15,0.177],[0.16,0.179],[0.17,0.211],[0.18,0.196],[0.19,0.203],[0.2,0.199],[0.21,0.218],[0.22,0.212],[0.23,0.204],[0.24,0.213],[0.25,0.2],[0.26,0.202],[0.27,0.199],[0.28,0.194],[0.29,0.205],[0.3,0.202],[0.31,0.212],[0.32,0.205],[0.33,0.198],[0.34,0.202],[0.35,0.207],[0.36,0.208],[0.37,0.21],[0.38,0.209],[0.39,0.21],[0.4,0.223],[0.41,0.222],[0.42,0.217],[0.43,0.221],[0.44,0.229],[0.45,0.233],[0.46,0.245],[0.47,0.239],[0.48,0.256],[0.49,0.267],[0.5,0.274],[0.51,0.275],[0.52,0.291],[0.53,0.292],[0.54,0.288],[0.55,0.297],[0.56,0.284],[0.57,0.285],[0.58,0.282],[0.59,0.281],[0.6,0.289],[0.61,0.289],[0.62,0.286],[0.63,0.283],[0.64,0.286],[0.65,0.289],[0.66,0.286],[0.67,0.28],[0.68,0.291],[0.69,0.29],[0.7,0.298],[0.71,0.3],[0.72,0.3],[0.73,0.303],[0.74,0.295],[0.75,0.29],[0.76,0.294],[0.77,0.284],[0.78,0.286],[0.79,0.281],[0.8,0.272],[0.81,0.272],[0.82,0.269],[0.83,0.252],[0.84,0.229],[0.85,0.219],[0.86,0.21],[0.87,0.202],[0.88,0.198],[0.89,0.195],[0.9,0.182],[0.91,0.159],[0.92,0.129],[0.93,0.1],[0.94,0.1],[0.95,0.1],[0.96,0.1],[0.97,0.1],[0.98,0.1],[0.99,0.1]]
// or just
"heatmap": [1,0.282,0.282,0.26,0.23,0.225,0.203,0.19,0.198,0.193,0.182,0.176,0.171,0.166,0.176,0.177,0.179,0.211,0.196,0.203,0.199,0.218,0.212,0.204,0.213,0.2,0.202,0.199,0.194,0.205,0.202,0.212,0.205,0.198,0.202,0.207,0.208,0.21,0.209,0.21,0.223,0.222,0.217,0.221,0.229,0.233,0.245,0.239,0.256,0.267,0.274,0.275,0.291,0.292,0.288,0.297,0.284,0.285,0.282,0.281,0.289,0.289,0.286,0.283,0.286,0.289,0.286,0.28,0.291,0.29,0.298,0.3,0.3,0.303,0.295,0.29,0.294,0.284,0.286,0.281,0.272,0.272,0.269,0.252,0.229,0.219,0.21,0.202,0.198,0.195,0.182,0.159,0.129,0.1,0.1,0.1,0.1,0.1,0.1,0.1]

For us the data is available in the initial data under playerOverlays

You also have the decorations for given timestamps (e.g. most replayed)
Not sure how we would extract this. Might need a new field?

I think some other porn website (I don't remember what exactly is) have feature like heatmap, so it may be worth to define a new field

I think some other porn website (I don't remember what exactly is) have feature like heatmap, so it may be worth to define a new field

@Lesmiscore The feature is a one-to-one copy in the web app. It's just uncanny.
The way it uses an <svg> element that has rectangles using <clipPath> (grouped in a <defs>) as a mask is just wayyy too suspicious.

There's no need for it to be in a <defs>. <clipPath> can be on it's own.
Hell, they could just use the <path> inside on it's own. On YouTube the entire use of a <clipPath> and their multiple rectangles is unnecessary, they could just use the path directly and change it's styling dynamically... but not if you're PH and have a more static approach on things, for them it makes sense.
Not talking about the entire approach of using SVG... They could have used plain CSS clip-path: path('M 0 200 ... z'), sooo much easier. But CSS clip-path that wasn't supported when PH first implemented it.

I bet money that YouTube's front-end Dev went like *clicks on search bar* P *autocompletes* ⏎ *clicks on a video* F12 "ok, so that's what we're going with..."  Ctrl+C

YT renders that SVG from the initial data as explained in https://github.com/yt-dlp/yt-dlp/issues/3888#issuecomment-1140109638

We could extract the raw data and put it in a new field, but what would users do with it?

but what would users do with it?

It's soft data, but there's still a lot, you can do with it:

On one hand it massively helps finding the relevant or iconic parts in a video. This could be anything: Bass drop, plot twist, announcement of a date, a joke, a certain clip.
On the other hand it helps isolating the irrelevant parts of a video, like an ad or a filler scene.
In combination with the view count it helps making an estimate of the watch time.
In combination with chapters you can check if the given sections are lining up with the user watch time and project a certain relevance to them.
You can also check whether chapters were specified correctly or if e.g. the ad overlaps into the next chapter, or does not appear in the chapters at all.
You can analyze the user interaction. Some examples with music videos:

Evenly watched video, where people click on it to watch all. The music plays from start to end: Numb [Official Music Video] - Linkin Park

The music part of this music video is only from 0:30 to 2:15: Kanye West - Follow God
Clicked but basically only the first seconds are watched (maybe because people just want to dislike or interact with the comments; maybe also bots?): Kalush Orchestra - Stefania (Official Video Eurovision 2022)
 and Rick Astley - Never Gonna Give You Up (Official Music Video)

The people only watch this video for the moment around 1:00: (ORIGINAL Music Video) Stay With Me - Miki Matsubara

I understand how this data is useful when show overlayed on the video. But obv, yt-dlp can't do that. We can only extract the data and put it in a field in the infojson. To be clear, I am not against doing this. My point is just that the data does not appear to be useful in this form (https://github.com/yt-dlp/yt-dlp/issues/3888#issuecomment-1140109638). Before implementing this, we should come up with some format (1) from which the useful information can be extracted easily and (2) that can be generalized to other sites as needed.
We shouldn't repeat the same mistake on this that yt-dlc made with live chat downloads. Implementing the live chat as a subtitle has made it quite difficult to expand this to other sites. Furthermore, by making it download the raw content, it made it difficult for any third parties to extract useful info out of the live chat.
When implementing this, I just want to avoid similar mistakes. Once the feature is implemented and released, it is assumed that third parties may depends on it. Any changes after that point will be bound by backward compatibility requirements

On a related note, has anyone requested this feature to any video player (mpv/mpc/vlc)? I would like to see how they want to handle this

I agree with you, the layout should be well thought out. But I found no standards, drafts or suggestions on how to format this. Not for any file format or player.
There also seems to be no obvious standard throughout proprietary software. P***Hub calls this feature "hotspots". It seems they just yeet the data to the front-end by embedding it into HTML using a JavaScript variable called flashvars. Just a plain list of watch time counters for every 5 seconds of the video (incremented by the pings that are send every 5 seconds by their player):
"hotspots":["2126473","1509162","1415385","1263371","1130396","1063220","1032573","1012590","1049435","1042380","1047685","1097360","1098579","1046719","1120822","1071058","1024836","1004157","1010109","1007019","1009978","1014976","1088071","1141994","1142713","1115550","1004629","909680","921539","865001","830299","827111","831371","803551","788714","755455","728531","699063","665349","642931","619397","716136","663634","661081","651885","598485","556105","549820","580571","677124","697647","763739","828103","779439","780161","821711","782140","745356","702643","676545","675660","718809","755582","766731","695390","665713","637099","633781","641765","630462","618966","610812","590461","581882","556716","549597","536563","578955","554676","593583","607611","721988","627043","619569","621223","609902","594560","588217","569446","638016","552857","549362","517154","508932","492234","495604","469092","467487","468587","516710","556376","659476","727488","774049","702127","653983","588627","548224","505261","471961","445253","397921"]

Naming it heat map would make the most sense, cause it's the only scientific term for that. But standardizing the rest seems to be hard:
How to handle normalized (0-100%) as well as absolute values for the marker? Include both? If yes, should the normalized value be calculated if missing or left to be null?
If we include "start" and "end"(/"duration"), what if a site doesn't provide it? Should that be calculated or left to be null?
One of the possible formats could look like this:
"heatmap": [{
   "start": 0,          // Start of the marker in seconds. Always provided.
                        // Might be calculated with: video duration ÷ number of all markers · index of this marker
   "end": 4.321,        // End of the marker in seconds. Always provided.
                        // Might be calculated with: video duration ÷ number of all markers · (index of this marker + 1)
   "normalized": 0.81,  // Normalized heat value. Always provided.
                        // Might be calculated with: absolute value of this marker ÷ biggest absolute value of all markers
   "absolute": 2126473, // Absolute heat value. Might be not provided at all.
  // ... add more attributes for markers in the future here
}, ...],
"heatmapMeta": { // ... add more data about the heat map in the future here

I don't want to make an inappropriate remark but I have been contacted about this thread so I thought I may help some people in immediate need: my open-source YouTube operational API is able to retrieve the most replayed data from a YouTube video from its id by fetching https://A_USUAL_INSTANCE/videos?part=mostReplayed&id=VIDEO_ID (note that the official instance yt.lemnoslife.com is detected by Google as unusual).

To extract data youtube-heatmap

Was thinking about training a machine learning model for sponsorship skipping. Most people skip through it, then overshoot and go back a bit. Most videos with sponsorships have a large peak right at the end of the sponsored segment.

I don't want to make an inappropriate remark but I have been contacted about this thread so I thought I may help some people in immediate need: my open-source YouTube operational API is able to retrieve the most replayed data from a YouTube video from its id by fetching https://A_USUAL_INSTANCE/videos?part=mostReplayed&id=VIDEO_ID

Thank you, I will try not to kill the API :)

I think niklas-englert's proposal is mostly good.

end/normalized can be generated by core code automatically
Do we need normalized? It can easily be calculated by the program consuming the data (imo not needed)

Alternately, do we need absolute? What does it actually mean? Number of times the point has been watched? Some magic number Youtube came up with that means nothing to us un-normalized?

Should we have a duration similar to chapters? (imo not needed)
What's heatmapMeta supposed to be?

Note that once we add a field, it can never be removed/changed due to compatibility requirements. But any field that we skip can always be added back in future if needed

My suggestion was designed to provide a standard, able to represent heatmap data in a uniform way as well as to include all additional information that other sites provide as well as information that may come in the future.

Do we need normalized? It can easily be calculated by the program consuming the data (imo not needed)

Alternately, do we need absolute? What does it actually mean? Number of times the point has been watched? Some magic number Youtube came up with that means nothing to us un-normalized?

For YouTube (and for now) we can only provide a normalized attribute as YT only sends 0%-100% (=normalized) heatmap values. absolute should only be added if YT (or another yt-dlp supported video service) binds an actual view counter (=absolute) to the heatmap points. (CornHub and the heatmap from YouTube Analytics provide such absolute values.)

Should we have a duration similar to chapters? (imo not needed)

Well, you could drop start and end attributes... but a start and duration in milliseconds is provided in YTs data... so I personally would keep them, if YT themselves think is necessary.
At least it helps to distinguish whether the measurements were taken at a certain point in time (e.g. at exactly 12:40 only 12% of the viewers were left) or whether a time span was averaged (e.g. only 12% were watching between 12:00 and 13:20).

What's heatmapMeta supposed to be?

Can be completely omitted at the moment. I just wanted to demonstrate what could be done if more data than just the points of the heatmap are provided.

About the duration and normalized, IMO the duration is necessary. I believe we don't need the start and end, because probably the heatmap will take most of the video, even if sometimes we have some picks (mountains) in the heatmap.
The normalization is a good idea, IMHO, because it will transform the heatmap in a coordinate of time, and value.
This is what I propose:
"heatmap": [{
   "time": 0,          // the second related to the heatmap
   "normalized": 0.81,  // Normalized heat value from 0 to 1.
}, ...],

Following the approach from @Benjamin-Loison  and @WillianAgostini, I made the following gist in python.
https://gist.github.com/guifeliper/c8c79d312ea258c1ad776d2cbd919620
This gist does not use the methodology explained here, as I was not able to completely translate the logic once we have videos with chapters. What the Gist is doing is getting the HTML from a link search for a specific script that contains the ytInitialData, inside this data we can retrieve the original data from youtube.
You can also see the same code in Javascript, this is my original version. I have made a version from python that runs, but with a few errors on puppeteer, but as Python is not my "native" language and I do it only for a hobby I will let it for the experts to fix it.
https://github.com/guifeliper/yt-heatmap#readme
output example from JS:
[
  {
    timeRangeStartSeconds: 0,
    markerDurationSeconds: 11.25,
    heatMarkerIntensityScoreNormalized: 1
  },
  {
    timeRangeStartSeconds: 11.25,
    markerDurationSeconds: 11.25,
    heatMarkerIntensityScoreNormalized: 0.07961592756046977
  },
  ....
]

There is an abandoned project that creates a timebar of thumbnails and other data, with a script that rendered it in MPV: https://github.com/nordlicht/nordlicht

@aleksejrs How's that related to this issue? yt-dlp can already download storyboards (thumbnails in timebar)[^1]. This post is about heatmap (see image in OP), not storyboards
[^1]: And further processing of the images is out of scope for us

@pukkandan The program is related by having a script for MPV (I don't know if the script still works though).
How do I learn about the existence of storyboards and how to use them?

How do I learn about the existence of storyboards and how to use them?

They are listed in -F and can be selected with -f for downloading. If you have further questions, pls open a new issue

I don't know if it's a YouTube side change or if I just found out a video having this behavior but sometimes (after multiple webpage refreshes) MX5GkDRIdno ends up with two Most replayed segments (see attachments) which lead to having null returned in the following Python code snippet.
from yt_dlp import YoutubeDL
import json

with YoutubeDL() as ydl:
  info_dict = ydl.extract_info('https://www.youtube.com/watch?v=MX5GkDRIdno', download=False)
  print(json.dumps(info_dict.get('heatmap'), indent = 4))

Instead of considering ytInitialData['playerOverlays']['playerOverlayRenderer']['decoratedPlayerBarRenderer']['decoratedPlayerBarRenderer']['playerBar']['multiMarkersPlayerBarRenderer']['markersMap'][-1]['value']['heatmap']['heatmapRenderer'], we have to consider ytInitialData['frameworkUpdates']['entityBatchUpdate']['mutations'][0]['payload']['macroMarkersListEntity']['markersList']. Note that the data structure doesn't just go from an element to an array, as for instance intensityScoreNormalized is renamed to heatMarkerIntensityScoreNormalized.
I am currently managing this issue in my YouTube operational API.


/bin/sh missing

Your Windows build number:  10.0.17763.504

What you're doing and what's happening:  Opening WSL by typing "ubuntu" into windows search to open it

What's wrong / what should be happening instead: I get printed at the top my terminal -bash: /usr/bin/lesspipe: /bin/sh: bad interpreter: No such file or directory
Typing in bash into terminal gives me: bash: /usr/bin/lesspipe: /bin/sh: bad interpreter: No such file or directory
Running sudo apt update gives me errors,

Couldn't execute /usr/bin/apt-key to check /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_bionic-security_InRelease

GPG error: http://security.ubuntu.com/ubuntu bionic-security InRelease: Couldn't execute /usr/bin/apt-key to check /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_bionic-security_InRelease

Strace of the failing command, if applicable:  Running strace -o bash.strace -f bash gives
bash: /usr/bin/lesspipe: /bin/sh: bad interpreter: No such file or directory

I have reinstalled WSL multiple times now after a few errors and this is how I fixed it last time. It was working fine a few hours ago but then I restarted and it broke

Unsure what I can do to repro it. Currently, all I do is open up the Ubuntu app and I get -bash: /usr/bin/lesspipe: /bin/sh: bad interpreter: No such file or directory Printed at the top of my terminal.


install: Re-implement hostns to work around supermin

I'm experimenting with doing bootc install in a supermin VM like we do in coreos-assembler, but hit upon the fact that supermin's /sbin/init implementation incorrectly does chroot() instead of a pivot_root:
https://github.com/libguestfs/supermin/blob/5230e2c3cd07e82bd6431e871e239f7056bf25ad/init/init.c#L288
The problem here then is doing nsenter just takes us into the original rootfs, and all the real content is in /root.
Re-implement this by re-executing our own binary and doing the setns() and chroot() dance internally, because we can't really do it with external binaries.

Filed https://github.com/libguestfs/supermin/issues/19


Update actions

Reported to GitHub as CI abuse.

Ah, sorry, I've already done that and cancelled the workflow.

Thanks! I've also enabled "forbid user accounts younger than 24 hours to interract with the repo" for a couple of days.

I'm not sure how much that helps. They can just spam other 200 repos instead.


x/pkgsite: support launchpad.net in source.ModuleInfo

Add support for getting source from launchpad.net.
https://github.com/golang/pkgsite/blob/4ea5ad9/internal/source/source.go#L273
Related: #39621

@julieqiu
May I give it a try?

@amarjeetanandsingh If you're still interested, we'd love to have your help!

Thanks @jba
I will start looking into it.


(Calendar) Event Reminders Overrides Option Broken

The Reminders field does not populate correctly when I instantiate a Google::Apis::CalendarV3::Event object. This causes insert_event to fail when the :reminders option is passed, effectively preventing me from adding reminders to an event.
This is the example code I found in the API reference. There are several problems with it. I have been able to send requests successfully without reminders.
event = Google::Apis::CalendarV3::Event.new{
  summary: 'Google I/O 2015',
  location: '800 Howard St., San Francisco, CA 94103',
  description: 'A chance to hear more about Google\'s developer products.',
  start: {
    date_time: '2015-05-28T09:00:00-07:00',
    time_zone: 'America/Los_Angeles',
  },
  end: {
    date_time: '2015-05-28T17:00:00-07:00',
    time_zone: 'America/Los_Angeles',
  },
  recurrence: [
    'RRULE:FREQ=DAILY;COUNT=2'
  ],
  attendees: [
    {email<EMAIL_ADDRESS>    {email<EMAIL_ADDRESS>  ],
  reminders: {
    use_default: false,
    overrides: [
      {method' => 'email', 'minutes: 24 * 60},
      {method' => 'popup', 'minutes: 10},
    ],
  },
}

result = client.insert_event('primary', event)
puts "Event created: #{result.html_link}"

Also, there should not be a method named end.

Will have the sample updated. The reminders part likely should read:
reminders: {
use_default: false,
overrides: [
{method_prop: 'email', minutes: 24 * 60},
{method_prop: 'popup', minutes: 10},
],
},

I discovered, after reviewing the source, that :reminder_method will work as a replacement for :method. However, both the documentation and the source suggest that :method is perfectly valid when it is not.

I had the same issue and stop using reminder but change the default setting from Google calendar setting..

I have just found that my event reminders that were set as popup do not show in Google Calendar on the pc  things seem to have changed?

@sqrrrl The reminders property on Google::Apis::CalendarV3::Event does not seem to deserialize either. Think this is a Google::Apis::CalendarV3::EventReminders problem as I am failing the deserialize default_reminders on Google::Apis::CalendarV3::Events too.
{
 "kind": "calendar#events",
 "etag": "\"p33k8h1e1peit60g\"",
 "summary": "Hello Google",
 "description": "",
 "updated": "2017-04-15T04:09:06.641Z",
 "timeZone": "Asia/Seoul",
 "accessRole": "owner",
 "defaultReminders": [
  {
   "method": "popup",
   "minutes": 10
  }
 ],
 "nextSyncToken": "COiIhcHLpdMCEOiIhcHLpdMCGAU=",
 "items": [
  {
   "kind": "calendar#event",
   "etag": "\"2984458692589000\"",
   "id": "rl6jjmphq2c3uq74c9q5h866i0",
   "status": "confirmed",
   "htmlLink": "https://www.google.com/calendar/event?eid=cmw2amptcGhxMmMzdXE3NGM5cTVoODY2aTAgY2VtdTdiOTVkbHEzbDRhNWw2MHFhdmVyYWtAZw",
   "created": "2017-04-14T19:38:51.000Z",
   "updated": "2017-04-15T04:09:06.392Z",
   "summary": "Hi Linder",
   "creator": {
    "email"<EMAIL_ADDRESS>   },
   "organizer": {
    "email"<EMAIL_ADDRESS>    "displayName": "Hello Google",
    "self": true
   },
   "start": {
    "dateTime": "2017-05-09T10:00:00+09:00"
   },
   "end": {
    "dateTime": "2017-05-09T11:00:00+09:00"
   },
   "transparency": "transparent",
   "visibility": "public",
   "iCalUID"<EMAIL_ADDRESS>   "sequence": 1,
   "reminders": {
    "useDefault": false,
    "overrides": [
     {
      "method": "popup",
      "minutes": 50
     },
     {
      "method": "popup",
      "minutes": 10
     }
    ]
   }
  }
 ]
}

serializes to
#<Google::Apis::CalendarV3::Events:0x007f8a00fd7618
 @access_role="reader",
 @default_reminders=[],
 @description="",
 @etag="\"p33k8h1e1peit60g\"",
 @items=
  [#<Google::Apis::CalendarV3::Event:0x007f8a00fcf440
    @created=Fri, 14 Apr 2017 19:38:51 +0000,
    @creator=
     #<Google::Apis::CalendarV3::Event::Creator:0x007f8a00fc33e8
<EMAIL_ADDRESS>    @end=
     #<Google::Apis::CalendarV3::EventDateTime:0x007f8a00fc0a08
      @date_time=Tue, 09 May 2017 11:00:00 +0900>,
    @etag="\"2984458692784000\"",
    @html_link=
     "https://www.google.com/calendar/event?eid=cmw2amptcGhxMmMzdXE3NGM5cTVoODY2aTAgY2VtdTdiOTVkbHEzbDRhNWw2MHFhdmVyYWtAZw",
<EMAIL_ADDRESS>    @id="rl6jjmphq2c3uq74c9q5h866i0",
    @kind="calendar#event",
    @organizer=
     #<Google::Apis::CalendarV3::Event::Organizer:0x007f8a00fa73a0
      @display_name="Hello Google
",
<EMAIL_ADDRESS>      @self=true>,
    @reminders=
     #<Google::Apis::CalendarV3::Event::Reminders:0x007f8a00f9ecc8
      @use_default=true>,
    @sequence=1,
    @start=
     #<Google::Apis::CalendarV3::EventDateTime:0x007f8a00f97270
      @date_time=Tue, 09 May 2017 10:00:00 +0900>,
    @status="confirmed",
    @summary="Hi Linder",
    @transparency="transparent",
    @updated=Sat, 15 Apr 2017 04:09:06 +0000,
    @visibility="public">],
 @kind="calendar#events",
 @next_sync_token="COiIhcHLpdMCEOiIhcHLpdMCGAE=",
 @summary="Hello Google",
 @time_zone="Asia/Seoul",
 @updated=Sat, 15 Apr 2017 04:09:06 +0000>

Did some investigation here. The original report from @LaMarseillaise remains pertinent: the Ruby sample is completely wrong. (For starters, beyond the syntax issues, the Ruby client simply does not translate nested hashes in this way. It needs to be constructed object-by-object.) I will create a fix for that.
There appear to be several other reports piled on, but are either unrelated or cannot be reproduced. In particular, I tried reproducing @nicholaskim94 's deserialization issue but was unsuccessful. Executing the deserialization code used in the client:
Google::Apis::CalendarV3::Events::Representation.new(Google::Apis::CalendarV3::Events.new).from_json(input_string)

yields the correct result with the correct reminders. It is possible that something was fixed since that report.
I will get the Ruby sample fixed and close this issue.

No wonder I forgot! My post was Feb 2017...!

Yes, really sorry about that. This repo was bounced around a few teams since then, and a bunch of the issues got neglected. I'm trying to go through and fix and close things now.

I am not sure how to interface such code into my project. Or is this something you guys need to do?

Looks like I am not able to edit the sample directly. So I reported this upstream to the documentation team: https://issuetracker.google.com/issues/143895117
For reference, the following corrected sample should work:
    event = Google::Apis::CalendarV3::Event.new(
      summary: 'Google I/O 2015',
      location: '800 Howard St., San Francisco, CA 94103',
      description: 'A chance to hear more about Google\'s developer products.',
      start: Google::Apis::CalendarV3::EventDateTime.new(
        date_time: '2015-05-28T09:00:00-07:00',
        time_zone: 'America/Los_Angeles'
      ),
      end: Google::Apis::CalendarV3::EventDateTime.new(
        date_time: '2015-05-28T17:00:00-07:00',
        time_zone: 'America/Los_Angeles',
      ),
      recurrence: [
        'RRULE:FREQ=DAILY;COUNT=2'
      ],
      attendees: [
        Google::Apis::CalendarV3::EventAttendee.new(
          email<EMAIL_ADDRESS>        ),
        Google::Apis::CalendarV3::EventAttendee.new(
          email<EMAIL_ADDRESS>        )
      ],
      reminders: Google::Apis::CalendarV3::Event::Reminders.new(
        use_default: false,
        overrides: [
          Google::Apis::CalendarV3::EventReminder.new(
            reminder_method: 'email',
            minutes: 24 * 60
          ),
          Google::Apis::CalendarV3::EventReminder.new(
            reminder_method: 'popup',
            minutes: 10
          )
        ]
      )
    )

    result = client.insert_event('primary', event)
    puts "Event created: #{result.html_link}"

@ajtruckle I'm not sure what you're asking. You should be able to create a client object of type Google::Apis::CalendarV3::CalendarService, set the API key or auth credentials, and then call this or any other method as with the sample. Look at the AUTHENTICATION.md if you're not sure how to set up auth.