FFmpeg backend seems to lose some samples in select FLAC files

🐛 Describe the bug
It works correctly for most of LibriSpeech, but for some files the ffmpeg backend returns less samples than sox and soundfile.
import os
os.environ["TORCHAUDIO_USE_BACKEND_DISPATCHER"] = "1"

import torchaudio

p = "download/LibriSpeech/train-clean-100/7067/76048/7067-76048-0021.flac"
print("File:", p)
print()
for backend in ("ffmpeg", "sox", "soundfile"):
    print(f"{torchaudio.info(p, backend=backend).__dict__=}")
    print(f"{torchaudio.load(p, backend=backend)[0].shape=}")
    print()

Output:
File: download/LibriSpeech/train-clean-100/7067/76048/7067-76048-0021.flac

torchaudio.info(p, backend=backend).__dict__={'sample_rate': 16000, 'num_frames': 233472, 'num_channels': 1, 'bits_per_sample': 16, 'encoding': 'FLAC'}
torchaudio.load(p, backend=backend)[0].shape=torch.Size([1, 233472])

torchaudio.info(p, backend=backend).__dict__={'sample_rate': 16000, 'num_frames': 252480, 'num_channels': 1, 'bits_per_sample': 16, 'encoding': 'FLAC'}
torchaudio.load(p, backend=backend)[0].shape=torch.Size([1, 252480])

torchaudio.info(p, backend=backend).__dict__={'sample_rate': 16000, 'num_frames': 252480, 'num_channels': 1, 'bits_per_sample': 16, 'encoding': 'FLAC'}
torchaudio.load(p, backend=backend)[0].shape=torch.Size([1, 252480])

It's easily verified that it actually discards useful audio by plotting it:
import matplotlib.pyplot as plt

fig, axes = plt.subplots(3, 1, figsize=(12, 6), sharex=True)
for idx, backend in enumerate(("ffmpeg", "sox", "soundfile")):
    audio = torchaudio.load(p, backend=backend)[0]
    axes[idx].plot(audio[0].numpy())

You can reproduce it with some files in LibriSpeech. Here's a few that I found:

LibriSpeech/train-clean-100/7067/76048/7067-76048-0021.flac
LibriSpeech/train-clean-360/5093/29101/5093-29101-0006.flac
LibriSpeech/train-clean-360/6782/61316/6782-61316-0059.flac
LibriSpeech/train-clean-360/7867/110742/7867-110742-0014.flac
LibriSpeech/train-other-500/551/129024/551-129024-0030.flac

Versions
Collecting environment information...
PyTorch version: 2.0.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 13.3.1 (arm64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-14<IP_ADDRESS>.1)
CMake version: version 3.25.0
Libc version: N/A
Python version: 3.10.4 (main, Mar 31 2022, 03:37:37) [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-13.3.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M1 Max
Versions of relevant libraries:
[pip3] flake8==5.0.4
[pip3] k2==1.23.4.dev20230412+cpu.torch2.0.0
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.5
[pip3] torch==2.0.0
[pip3] torchaudio==2.0.0
[pip3] torchvision==0.15.0
[conda] k2                        1.23.4.dev20230412+cpu.torch2.0.0          pypi_0    pypi
[conda] numpy                     1.23.5          py310hb93e574_0
[conda] numpy-base                1.23.5          py310haf87e8b_0
[conda] pytorch                   2.0.0                  py3.10_0    pytorch
[conda] torch                     1.12.1                   pypi_0    pypi
[conda] torchaudio                2.0.0                 py310_cpu    pytorch
[conda] torchvision               0.15.0                py310_cpu    pytorch

This is what I got:
torchaudio.utils.ffmpeg_utils.get_versions()

{'libavutil': (56, 31, 100),
 'libavcodec': (58, 54, 100),
 'libavformat': (58, 29, 100),
 'libavfilter': (7, 57, 100),
 'libavdevice': (58, 8, 100)}

Since I don't see the ffmpeg version there, here's some extra info:
$ which ffmpeg
/Users/pzelasko/miniconda3/envs/lhotse-py3.10/bin/ffmpeg
$ ffmpeg -version
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
built with clang version 12.0.0
configuration: --prefix=/Users/pzelasko/miniconda3/envs/lhotse-py3.10 --cc=arm64-apple-darwin20.0.0-clang --disable-doc --enable-avresample --enable-gmp --enable-hardcoded-tables --enable-libfreetype --enable-libvpx --enable-pthreads --enable-libopus --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-static --enable-version3 --enable-zlib --enable-libmp3lame --disable-nonfree --enable-gpl --enable-gnutls --disable-openssl --enable-libopenh264 --enable-libx264
libavutil      56. 31.100 / 56. 31.100
libavcodec     58. 54.100 / 58. 54.100
libavformat    58. 29.100 / 58. 29.100
libavdevice    58.  8.100 / 58.  8.100
libavfilter     7. 57.100 /  7. 57.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  5.100 /  5.  5.100
libswresample   3.  5.100 /  3.  5.100
libpostproc    55.  5.100 / 55.  5.100

So in order to use 5.1, I would need to build both ffmpeg and torchaudio from source? If 5.1 indeed fixes the issue, is it possible to upgrade to it in the next torchaudio binary release?

So in order to use 5.1, I would need to build both ffmpeg and torchaudio from source? If 5.1 indeed fixes the issue, is it possible to upgrade to it in the next torchaudio binary release?

Yes, right now to use 5.1 one needs to build torchaudio from source.
Now that conda-forge has 5.1 for all the platforms, it seems reasonable to upgrade to 5.1.

I tested this with different versions of FFmpeg from conda-forge.
The issue happens with ffmpeg 4.1 and 4.2 but does not happen on 4.3, 4.4 and 5.1.
@pzelasko We have minor release scheduled in couple of weeks, but the window for cherry-pick has been closed.
Changing the FFmpeg version has to wait until the next major version. Meanwhile if possible, can you try FFmpeg 4.3 or 4.4?

Thanks @mthrok! I just checked ffmpeg=4.4 from conda-forge and indeed the issue is gone.


UnhandledPromiseRejectionWarning: Error: invalid_grant

So i've been having this weird issue, each time that I set up the program it works for a few days, then I start to receive this error. I am able to download individual videos one at a time by running youtube-dl itself, but can't get the scrapper to run. I have reinstalled the system (whole os) and this error starts occurring again after a few days.
The system is: Ubuntu 18.04.5 LTS
youtube-dl --version 2<IP_ADDRESS>
nodejs --version  v8.10.0
james@james:~$ sh ~/youtube-subscription-dl/yt-subs.sh 2<IP_ADDRESS> (node:3024) UnhandledPromiseRejectionWarning: Error: invalid_grant     at Gaxios.<anonymous> (/home/james/youtube-subscription-dl/node_modules/gaxios/build/src/gaxios.js:73:27)     at Generator.next (<anonymous>)     at fulfilled (/home/james/youtube-subscription-dl/node_modules/gaxios/build/src/gaxios.js:16:58)     at <anonymous>     at process._tickCallback (internal/process/next_tick.js:188:7) (node:3024) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:3024) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Weird. seems to be a known bug with google API  https://blog.timekit.io/google-oauth-invalid-grant-nightmare-and-how-to-fix-it-9f4efaf1da35
Maybe that link will help, but it seems there's nothing I can do about it, I am assuming it's hitting the API limit - if you check your Google dashboard it should show how much usage you've used, the free tier is "okay" but if you have like ~500 subscriptions it would easily hit the limit

Running the script when receiving the error doesn't even seam to hit the api request counter when it gives an error, and it only reports 5000 requests over the last 30 days so I don't think it is that. This is the third round of it breaking, and I suppose the main thing I have changed about how I use the scripts is moving from a vps to a local hyper-v machine. Off to see if virtual box treats me better.

After one week of running inside virtualbox it stopped working on the 7th day, but on the same system under a different user account i'm able to run the program after reauthenticating with the same client_secret.json .
So now my question is, how would I go about the reauthenticating process under the same user account on virtual box? I tired removing the file in .credentials and that still results in the invalid_grant error. It's just strange, since when I create a new user account it works.
Error confirming youtube API key Error while trying to retrieve access token invalid_grant

I've come across this too and belive it's to do with the key generated by google where they timeout after 7days. I didn't have this issue with a google apps acount. I havn't yet worked out how to get around this 7 days time out. I have even followed this info from google developers but it didn't work.  https://www.youtube.com/watch?v=hfWe1gPCnzc&ab_channel=GoogleDevelopers as that timed out after an hour.


VQ VAE model's reconstruction is a black image

I am using your wonderful library in my research project. There seems to be a bug in the VQ VAE mode class where the reconstruction is a blank image. Is it a known bug? Can you please help me with this issue?

You should monitor the loss. If the loss grow up dramatically, reduce the learning rate. I set the LR 0.0003

I will try @jiang-wenbin approach. Marking this closed.


Support Paged.JS for automatic hugo resume -> PDF conversion.

Feature Request
Is your feature request related to a problem? Please describe.
I would really like to maintain a resume in markdown on hugo, and then be able to generate a PDF from that when needed.
Describe the solution you'd like
The Pagedjs library seems to support this easily: https://www.pagedjs.org/ here's an example of what the button does (under the journal section of the main page: https://www.pagedjs.org/page-floats/
Describe alternatives you've considered
The alternative is uploading a PDF
Additional context
@julientaq https://github.com/julientaq is the maintainer.

Hi there!
I’m starting a Paged.js integration to Hugo asap so you’ll be able to print to PDF.
https://github.com/zzossig/hugo-theme-zzo/issues/151

Hey folks, wanna try it out for me?
Still pretty experimental, but i’d love to have your thoughts.
https://gitlab.pagedmedia.org/julientaq/pagedjs-hugo
If you follow the readme, you’ll get a print button that you can add to any page as part of the template. next step is to add a configurable option in the front matter to show it or not :)

I can try it out later this week!

I just updated it, so the print modal appears as soon as paged.js has finished.

There's no need to perform PDF generation on the frontend in this manor and we already have requests to add this to the backend part of Academic (https://github.com/sourcethemes/academic-admin ) - feel free to contribute to that repo.


bdc subcommand is missing in few places

@rl-msft - this PR is massive and complex with over 2600 commits from a wide range of people. This is a highly unusual PR and not something we can handle in the public repos. Public repos are designed for small changes to one or two files at a time.
I'm tagging the repo owners for their input and review @craigg-msft @MikeRayMSFT @rothja @MightyPen  .
Thanks.
Here's some information about working in the private repos:

Microsoft authors and contributors need to make content contributions through the private repository so the content can be staged and validated by the current validation rules.
After you review staging and fix all validation issues, you can tag an author for review or add the sign-off comment to let PR reviewers know that the updates are ready to be merged.
Read about authors working in the public repo for more information.
If you need help getting set up to work in the private repo, see How to get to the private repo. You can also take a look at the GitHub account setup page for additional details on making sure you're set up.

@rl-msft Hey, so sorry you're having problems. In my last mail I think I figured out what the issue was, but I would be happy to have a quick Teams call with you tomorrow to explain in person with screen sharing. I'm closing this one out so we can get a correct PR going. Sorry, again.


idea for theming the app

i noticed the app seemingly uses msstyles for colors, would it be possible to instead support theming everything inside the app itself? there could be default preset themes, (light mode, dark mode, etc) but it would be nice to have a separate "advanced" tab to fully customize the colors of everything

i was gonna add the idea label to this aswell but im somewhat new to creating issues on github and could not figure out how to

I highly doubt it. You'd need to somehow figure out a way to load themes for JUST the app itself, and its controls. I'd have liked to add Win10/11 dark mode support, but it felt like too much work for what could potentially be an inconsistent mess.

odd idea and im not sure if this would even work, but would it be possible to instead make the app optionally accept msstyles files inside the app directory?

I'm pretty sure it wouldn't I'm afraid. In general Microsoft does not support custom msstyles seeing as you have to patch uxtheme.dll to remove the signature requirement.

https://winclassic.net/thread/2178/loading-visual-styles-application
There is a way to theme it per application

This is too much work with undocumented interfaces that could break, for too little effort. I am not going to be using that.
The most I'd do is add support for Windows 10/11 dark mode.


chore(deps): upgrade compiler dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-compiler-dependencies-k8s-27-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3174

Questions ?
Please refer to the Backport tool documentation


New Setting "Default Language" when creating user

New Setting "Default Language" when creating user

Sending email (CA zip file) based on user's default language after user created
**How to create i18n mail letter and instructions for import CA file to browser.

create i18n file in directory src\main\distrib\data
mail.tmpl_xx_YY  (ex: tranditional chinese  mail.tmp_zh_TW )
instructions_xx_YY (ex: tranditional chinese  instructions_zh_TW)

new paramter in file src\main\java\com\gitblit\wicket\GitBlitWebApp.properties
gb.emailClientCertificateSubject = Your Gitblit client certificate for {0}

some screen shot
http://imgur.com/a/add0z
http://imgur.com/a/3DKaE

Nice.  Thanks!


[Rails] Let's add a hover tooltip over language choices in the new CMS

The new CMS (located at /member/) adds the ability to pick a language from within the CMS. Right now, these are showing as acronyms in the bottom left (e.g. EN, ES, PT...). A user might not know what these acronyms stand for.

Let's add the full language strings for each language (English, Spanish, Portuguese...)  to each language's dashboard yml file. Bonus if you actually translate the languages into each existing language (English, Español, Espanhol...); Google translate should cover that just fine!
Let's add a hover tooltip displaying the localized language string per each language, akin to how we have hover tooltips for Speakers in Stories (see screenshot below).

@rudokemper can I take this?

@nikhilbhatt sure! I assigned you, thanks!


@graphql-mesh/runtime error

package: @graphql-mesh/runtime
version: 0.2.17
Error: TypeError: Cannot read property 'get' of undefined in file /node_modules/fetchache/index.cjs.js
when i use<EMAIL_ADDRESS>the error not appeared
my .meshrc.js is
module.exports = {
  sources: [
    {
      name: "auth",
      handler: { graphql: { endpoint: "http://localhost:4003/graphql" } },
      transforms: [{ prefix: { includeRootOperations: true, value: "auth_" } }],
    },
  ],
};

``

Fixed in 0.2.18!


bottom-nightly: Add version<PHONE_NUMBER>

It's in main
Apart of some batch PRs for some personal manifests

[x] I have read the Contributing Guide.

/verify

/verify

Waiting on approval in https://github.com/sumatrapdfreader/sumatrapdf/discussions/2895#discussioncomment-3831243 yes?


Array with alternating labels

[ ] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[x] support request

I have a scenario where my model is an array [0, 255, 27].
I know that this is a Vector from my JSON-schema. I would like a repeat for the 3 values where each value have a different label.  Can't find any good examples where i could alternate information on index.
<input matInput placeholder="x" value="0">
<input matInput placeholder="y" value="255">
<input matInput placeholder="z" value="27">

here is an example that uses the field index as a label https://stackblitz.com/edit/angular-sukugx


[WIP] Modify Python scripts to run on Python 2 and 3

Some systems use Python 3 as their default interpreter (e.g. #83).
This is a collection of fixes that should get these scripts much closer to running on both Python 2 and 3. Obtained by running the six module over every .py file in the repo, and going through the proposed changes by hand.
There are probably still a few gotchas that will break in Python 3 – but this gets substantially closer to a fully working set.

I'll do a test build on OS X and Linux before merging.  Assigning to myself.

@gribozavr Thanks; I haven’t had a chance to set up a build system yet.

@alexwlchan Sorry, but the OS X build failed when it tried to run gyb:
[18/543] Generating UnicodeExtendedGraphemeClusters.cpp from UnicodeExtendedGraphemeClusters.cpp.gyb with ptr size = 8
...
Traceback (most recent call last):
  File "<src>/swift/utils/gyb", line 3, in <module>
    gyb.main()
  File "<src>/swift/utils/gyb.py", line 1061, in main
    ast = parseTemplate(args.file.name, args.file.read())
  File "<src>/swift/utils/gyb.py", line 930, in parseTemplate
    return Block(ParseContext(filename, text))
  File "<src>/swift/utils/gyb.py", line 586, in __init__
    self.children.append(Node(context))
  File "<src>/swift/utils/gyb.py", line 602, in __init__
    context.nextToken()
  File "<src>/swift/utils/gyb.py", line 519, in nextToken
    for kind in self.tokens:
  File "<src>/swift/utils/gyb.py", line 497, in tokenGenerator
    if codeStartsWithDedentKeyword(sourceLines):
  File "<src>/swift/utils/gyb.py", line 357, in codeStartsWithDedentKeyword
    in tokenize.generate_tokens(sourceLines.__iter__().__next__):
AttributeError: 'listiterator' object has no attribute '__next__'

Assigning back to you.

@gribozavr Not too surprising. I’ll get myself a build system and keep trying. I’ll ping you when it’s done.

[ ] Python 2 on OS X
[ ] Python 3 on OS X
[ ] Python 2 on Ubuntu
[ ] Python 3 on Ubuntu

@alexwlchan: As a suggestion, you could create two gyb.py scripts (seeing as that seems to be the one failing), one with python2 modification and one with python3 modifications. Then decide which one to import and run from gyb or build-script or whichever one actually runs/imports it.

@Mitame Please don't duplicate the whole script.  If we need compatibility wrappers for certain APIs, we can declare them in the script itself.

@gribozavr: I wasn't going to, it was, as i said, a suggestion.

Also, gyb.py currently fails it's own tests with these commits with python2 utils/gyb.py --test and python3 utils/gyb.py --test. Also, a possible fix might be this https://github.com/Mitame/swift/tree/python-fixeshttps://github.com/Mitame/swift/tree/python-fixes though i haven't been able to test it because none of my systems have enough RAM to finish the build... aparently 8GB+8GB swap is not enough. I also apologise for all the whitespace changes in that. My editor when kinda crazy on it, but it was only meant as a reference anyway.

@Mitame I think I can do without separate scripts here. It’s a fairly small difference, and easy to fix. The only difference is the method names:

In Python 2, to get the next item of an iterator, you’d call foo.next().
In Python 3, this was helpfully renamed to foo.__next__(); see PEP 3114.

But there’s a Python builtin next(), which can be called as next(foo), and will choose the correct method to call based on your Python version.
I should have been a bit more careful when running six.
I’m running an Ubuntu build now to see if I can hit the problem above; then I’ll drop in the fix and make sure I’ve really fixed it. Then onwards – there are probably more edge cases that I’ve missed.

I was cosidering next() though i wasn't sure whether it would return the .__next__ or .next function or it's return value when run. In my commit, i just made it detect which python version was running and decide in line which method to use to make the iterator. Again, i can't test due to lack of RAM but it might work if next() doesn't work as expected.

@Mitame For the RAM issue, please build with build-script -R in release mode, then you wouldn't be building all the debug info that causes the linker to consume a lot of RAM.  Alternatively, after you hit the RAM issue, you can go into the build directory, and run ninja -j 1 to continue building in single-core mode.  Once you get past the issue, continue with build-script.

@gribozavr That info would be nice in the Readme if it's not already there.

Traceback (most recent call last):
  File "<src>/swift/utils/gyb", line 3, in <module>
    gyb.main()
  File "<src>/swift/utils/gyb.py", line 1068, in main
    args.target.write(executeTemplate(ast, args.line_directive, **bindings))
  File "<src>/swift/utils/gyb.py", line 971, in executeTemplate
    ast.execute(executionContext)
  File "<src>/swift/utils/gyb.py", line 590, in execute
    x.execute(context)
  File "<src>/swift/utils/gyb.py", line 666, in execute
    result = eval(self.code, context.localBindings)
  File "<src>/swift/lib/Basic/UnicodeExtendedGraphemeClusters.cpp.gyb", line 22, in <module>
    from GYBUnicodeDataUtils import *
  File "<src>/swift/utils/GYBUnicodeDataUtils.py", line 532
    code_point = ('\U%(cp)08x' % { 'cp': code_point }).decode('unicode_escape')
``` Another error running the build in python3

That one can be fixed with:
code_point = (b'\U{cp}08x'.format(cp=code_point)).decode('unicode_escape')

So far, I managed to get the whole thing to work on my machine with Python 3.4.3. If someone could please check this out: https://github.com/dmshynk/swift/commit/170c67baf07fa635aa0c174481cc79745fce168d.

Sorry, forgot to stage one file: https://github.com/dmshynk/swift/commit/7816e3fdedbfeddbf16f15670af7f13c6f1d487e

@dmshynk Thanks for your commits. I’ll get those merged in ASAP. I’ve still broken Python 2 support in a few places, so I want to get that checked first before I squash and we get this merged.

@alexwlchan I'm not a Python expert but I'm pretty sure this has to do with Python 3 because if I use Python 2 I don't get the error.
Anyways when I run $ swift/utils/build-script --preset=buildbot_linux_1510 installable_package=/tmp/swift.tar.gz install_destdir=/tmp/swift-install with Python 3 as the interpreter I get the following error:
Traceback (most recent call last):
  File "/home/ryan/Source/swift/src/swift/utils/build-script", line 755, in <module>
    sys.exit(main())
  File "/home/ryan/Source/swift/src/swift/utils/build-script", line 749, in main
    return main_preset()
  File "/home/ryan/Source/swift/src/swift/utils/build-script", line 77, in main_preset
    args.preset_substitutions, args.preset_file_names, args.preset)
  File "/home/ryan/Source/swift/src/swift/utils/SwiftBuildSupport.py", line 170, in get_preset_options
    config = _load_preset_files_impl(preset_file_names, substitutions)
  File "/home/ryan/Source/swift/src/swift/utils/SwiftBuildSupport.py", line 124, in _load_preset_files_impl
    if config.read(preset_file_names) == []:
  File "/usr/lib/python3.5/configparser.py", line 697, in read
    self._read(fp, filename)
  File "/usr/lib/python3.5/configparser.py", line 1064, in _read
    lineno)
configparser.DuplicateSectionError: While reading from '/home/ryan/Source/swift/src/swift/utils/build-presets.ini' [line 179]: section 'preset: buildbot,tools=RA,stdlib=DA' already exists

I know effectively nothing about Python but I will see if I can't track down what is causing this and how to fix it.

@RLovelett So this error gets raised when the parser finds the same section more than once in the config file. In this case, the section if preset: buildbot,tools=RA,stdlib=DA, which occurs on lines 56 and 179 of utils/build-presets.ini (at commit 7d3292b).
Fix is either to rearrange the config file, or run the parser in non-strict mode. I’ll try to include a fix.

(I’m quite busy this weekend, so only have sporadic moments to dig into it. Hoping to get passing builds on Pythons 2 and 3 by the end of next week. I think I’ve squashed most of the bugs.)

Thanks for the pointer. I've come up with this patch:
From e25b9a954d2c0c34777a789b84f0565bf5528d6a Mon Sep 17 00:00:00 2001
From: Ryan Lovelett<EMAIL_ADDRESS>Date: Sat, 5 Dec 2015 15:00:50 -0500
Subject: [PATCH 12/12] Allow parsing of old style on Python 3

---
 utils/SwiftBuildSupport.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/utils/SwiftBuildSupport.py b/utils/SwiftBuildSupport.py
index 5631d3b..7048924 100644
--- a/utils/SwiftBuildSupport.py
+++ b/utils/SwiftBuildSupport.py
@@ -120,7 +120,7 @@ def check_output(args, print_command=False, verbose=False):

 def _load_preset_files_impl(preset_file_names, substitutions={}):
-    config = ConfigParser.SafeConfigParser(substitutions, allow_no_value=True)
+    config = ConfigParser.SafeConfigParser(substitutions, allow_no_value=True, strict=False)
     if config.read(preset_file_names) == []:
         print_with_argv0(
             "preset file not found (tried " + str(preset_file_names) + ")")
--
2.6.3

Unfortunately, it doesn't work on Python 2 (it fails with TypeError: __init__() got an unexpected keyword argument 'strict') and I know nothing of Python to even begin to know what the most idiomatic way of conditionally applying based on the version.

@alexwlchan I've actually found another approach which was just to stream line the section declarations. I've submitted a stand-alone patch, #248, because it should be able to work on both 2 and 3. In case, it were to be merged before this one.

I've spent a bit more time looking at this. Still failing on Python 3, at these few lines in GYBUnicodeDataUtils.py:
code_point = 0x200b
code_point = ('\U%(cp)08x' % { 'cp': code_point }).decode('unicode_escape')
as_UTF8_bytes = code_point.encode('utf8')
as_UTF8_escaped = ''.join(['\\x%(byte)02x' % { 'byte': ord(byte) } for byte in as_UTF8_bytes])

Specifically, I'm struggling to come up with an implementation that produces identical results on
Python 2 and Python 3.
I'll continue to chip away at this, but any suggestions gratefully appreciated.
I've also rebased against the latest master to keep up with the latest fixes.

@dmshynk - I can't get your proposed patch for this to work. I get this error:
>>> code_point=0x200b
>>> code_point = (b'\U{cp}08x'.format(cp=code_point)).decode('unicode_escape')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'bytes' object has no attribute 'format'

@alexwlchan, sorry for the confusion, the solution I posted initially indeed doesn't work; in https://github.com/dmshynk/swift/commit/170c67baf07fa635aa0c174481cc79745fce168d, I went with
- code_point = ('\U%{cp}08x' % { 'cp': code_point }).decode('unicode_escape')
+ code_point = ('\\U%{cp}08x' % { 'cp': code_point }).decode('unicode_escape')

which seems to work in both Python 2 and Python 3.

Rebased to resolve merge conflicts.

@alexwlchan I'm getting two test failures on OS X:
FAIL: Swift :: stdlib/Set.swift (7302 of 7672)
******************** TEST 'Swift :: stdlib/Set.swift' FAILED ********************
Script:
--
rm -rf <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp
mkdir -p <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp
<src>/swift/validation-test/stdlib/../../utils/gyb <src>/swift/validation-test/stdlib/Set.swift -o <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/main.swift
xcrun --toolchain XcodeDefault --sdk macosx  clang++ -arch x86_64 -mmacosx-version-min=10.9 -fmodules-cache-path='/var/folders/ts/nf0skrh1301drfvsp44xvrbm0000gn/T/swift-testsuite-clang-module-cacheBTSzbK' -fobjc-arc <src>/swift/validation-test/stdlib/Inputs/SlurpFastEnumeration/SlurpFastEnumeration.m -c -o <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/SlurpFastEnumeration.o
<src>/swift/validation-test/stdlib/../../utils/line-directive <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/main.swift -- xcrun --toolchain XcodeDefault --sdk macosx  <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swiftc -target x86_64-apple-macosx10.9  -module-cache-path '/var/folders/ts/nf0skrh1301drfvsp44xvrbm0000gn/T/swift-testsuite-clang-module-cacheBTSzbK' -F /Applications/Xcode7.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/../../../Developer/Library/Frameworks -Xlinker -rpath -Xlinker /Applications/Xcode7.2/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/../../../Developer/Library/Frameworks    <src>/swift/validation-test/stdlib/Inputs/DictionaryKeyValueTypes.swift <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/main.swift -I <src>/swift/validation-test/stdlib/Inputs/SlurpFastEnumeration/ -Xlinker <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/SlurpFastEnumeration.o -o <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/Set -Xfrontend -disable-access-control
<src>/swift/validation-test/stdlib/../../utils/line-directive <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/main.swift --  <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/Set.swift.tmp/Set
--
Exit Code: 1

Command Output (stdout):
--
<src>/swift/validation-test/stdlib/Inputs/DictionaryKeyValueTypes.swift:618:40: warning: closure parameter prior to parameters with default arguments will not be treated as a trailing closure
  a: NSArray, _ fe: NSFastEnumeration, _ sink: (AnyObject) -> Void,
                                       ^
<src>/swift/validation-test/stdlib/Inputs/DictionaryKeyValueTypes.swift:660:45: warning: closure parameter prior to parameters with default arguments will not be treated as a trailing closure
  d: NSDictionary, _ fe: NSFastEnumeration, _ sink: (AnyObjectTuple2) -> Void,
                                            ^
<src>/swift/validation-test/stdlib/Inputs/DictionaryKeyValueTypes.swift:980:38: warning: closure parameter prior to parameters with default arguments will not be treated as a trailing closure
  s: NSSet, _ fe: NSFastEnumeration, _ sink: (AnyObject) -> Void,
                                     ^
[ RUN      ] Set.sizeof
[       OK ] Set.sizeof
[ RUN      ] Set.COW.Smoke
[       OK ] Set.COW.Smoke
[ RUN      ] Set.COW.Fast.IndexesDontAffectUniquenessCheck
[       OK ] Set.COW.Fast.IndexesDontAffectUniquenessCheck
[ RUN      ] Set.COW.Slow.IndexesDontAffectUniquenessCheck
[       OK ] Set.COW.Slow.IndexesDontAffectUniquenessCheck
[ RUN      ] Set.COW.Fast.SubscriptWithIndexDoesNotReallocate
[       OK ] Set.COW.Fast.SubscriptWithIndexDoesNotReallocate
[ RUN      ] Set.COW.Slow.SubscriptWithIndexDoesNotReallocate
[       OK ] Set.COW.Slow.SubscriptWithIndexDoesNotReallocate
[ RUN      ] Set.COW.Fast.ContainsDoesNotReallocate
[       OK ] Set.COW.Fast.ContainsDoesNotReallocate
[ RUN      ] Set.COW.Slow.ContainsDoesNotReallocate
[       OK ] Set.COW.Slow.ContainsDoesNotReallocate
[ RUN      ] Set.COW.Fast.InsertDoesNotReallocate
[       OK ] Set.COW.Fast.InsertDoesNotReallocate
[ RUN      ] Set.COW.Slow.InsertDoesNotReallocate
[       OK ] Set.COW.Slow.InsertDoesNotReallocate
[ RUN      ] Set.COW.Fast.IndexForMemberDoesNotReallocate
[       OK ] Set.COW.Fast.IndexForMemberDoesNotReallocate
[ RUN      ] Set.COW.Slow.IndexForMemberDoesNotReallocate
[       OK ] Set.COW.Slow.IndexForMemberDoesNotReallocate
[ RUN      ] Set.COW.Fast.RemoveAtIndexDoesNotReallocate
[       OK ] Set.COW.Fast.RemoveAtIndexDoesNotReallocate
[ RUN      ] Set.COW.Slow.RemoveAtIndexDoesNotReallocate
[       OK ] Set.COW.Slow.RemoveAtIndexDoesNotReallocate
[ RUN      ] Set.COW.Fast.RemoveDoesNotReallocate
[       OK ] Set.COW.Fast.RemoveDoesNotReallocate
[ RUN      ] Set.COW.Slow.RemoveDoesNotReallocate
[       OK ] Set.COW.Slow.RemoveDoesNotReallocate
[ RUN      ] Set.COW.Fast.UnionInPlaceSmallSetDoesNotReallocate
[       OK ] Set.COW.Fast.UnionInPlaceSmallSetDoesNotReallocate
[ RUN      ] Set.COW.Fast.RemoveAllDoesNotReallocate
[       OK ] Set.COW.Fast.RemoveAllDoesNotReallocate
[ RUN      ] Set.COW.Slow.RemoveAllDoesNotReallocate
[       OK ] Set.COW.Slow.RemoveAllDoesNotReallocate
[ RUN      ] Set.COW.Fast.FirstDoesNotReallocate
[       OK ] Set.COW.Fast.FirstDoesNotReallocate
[ RUN      ] Set.COW.Fast.CountDoesNotReallocate
[       OK ] Set.COW.Fast.CountDoesNotReallocate
[ RUN      ] Set.COW.Slow.FirstDoesNotReallocate
[       OK ] Set.COW.Slow.FirstDoesNotReallocate
[ RUN      ] Set.COW.Slow.CountDoesNotReallocate
[       OK ] Set.COW.Slow.CountDoesNotReallocate
[ RUN      ] Set.COW.Fast.GenerateDoesNotReallocate
[       OK ] Set.COW.Fast.GenerateDoesNotReallocate
[ RUN      ] Set.COW.Slow.GenerateDoesNotReallocate
[       OK ] Set.COW.Slow.GenerateDoesNotReallocate
[ RUN      ] Set.COW.Fast.EqualityTestDoesNotReallocate
[       OK ] Set.COW.Fast.EqualityTestDoesNotReallocate
[ RUN      ] Set.COW.Slow.EqualityTestDoesNotReallocate
[       OK ] Set.COW.Slow.EqualityTestDoesNotReallocate
[ RUN      ] Set.deleteChainCollision
[       OK ] Set.deleteChainCollision
[ RUN      ] Set.deleteChainNoCollision
[       OK ] Set.deleteChainNoCollision
[ RUN      ] Set.deleteChainCollision2
[       OK ] Set.deleteChainCollision2
[ RUN      ] Set.deleteChainCollisionRandomized
out>>> time is<PHONE_NUMBER>
out>>> chose parameters: collisionChains=6 chainLength=0
[       OK ] Set.deleteChainCollisionRandomized
[ RUN      ] Set.BridgedFromObjC.Verbatim.SetIsCopied
[       OK ] Set.BridgedFromObjC.Verbatim.SetIsCopied
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.SetIsCopied
[       OK ] Set.BridgedFromObjC.Nonverbatim.SetIsCopied
[ RUN      ] Set.BridgedFromObjC.Verbatim.NSSetIsRetained
[       OK ] Set.BridgedFromObjC.Verbatim.NSSetIsRetained
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.NSSetIsCopied
[       OK ] Set.BridgedFromObjC.Nonverbatim.NSSetIsCopied
[ RUN      ] Set.BridgedFromObjC.Verbatim.ImmutableSetIsRetained
[       OK ] Set.BridgedFromObjC.Verbatim.ImmutableSetIsRetained
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.ImmutableSetIsCopied
[       OK ] Set.BridgedFromObjC.Nonverbatim.ImmutableSetIsCopied
[ RUN      ] Set.BridgedFromObjC.Verbatim.IndexForMember
[       OK ] Set.BridgedFromObjC.Verbatim.IndexForMember
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.IndexForMember
[       OK ] Set.BridgedFromObjC.Nonverbatim.IndexForMember
[ RUN      ] Set.BridgedFromObjC.Verbatim.Insert
[       OK ] Set.BridgedFromObjC.Verbatim.Insert
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.Insert
[       OK ] Set.BridgedFromObjC.Nonverbatim.Insert
[ RUN      ] Set.BridgedFromObjC.Verbatim.SubscriptWithIndex
[       OK ] Set.BridgedFromObjC.Verbatim.SubscriptWithIndex
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.SubscriptWithIndex
[       OK ] Set.BridgedFromObjC.Nonverbatim.SubscriptWithIndex
[ RUN      ] Set.BridgedFromObjC.Verbatim.SubscriptWithIndex_Empty
[       OK ] Set.BridgedFromObjC.Verbatim.SubscriptWithIndex_Empty
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.SubscriptWithIndex_Empty
[       OK ] Set.BridgedFromObjC.Nonverbatim.SubscriptWithIndex_Empty
[ RUN      ] Set.BridgedFromObjC.Verbatim.Contains
[       OK ] Set.BridgedFromObjC.Verbatim.Contains
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.Contains
[       OK ] Set.BridgedFromObjC.Nonverbatim.Contains
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.SubscriptWithMember
[       OK ] Set.BridgedFromObjC.Nonverbatim.SubscriptWithMember
[ RUN      ] Set.BridgedFromObjC.Verbatim.RemoveAtIndex
[       OK ] Set.BridgedFromObjC.Verbatim.RemoveAtIndex
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.RemoveAtIndex
[       OK ] Set.BridgedFromObjC.Nonverbatim.RemoveAtIndex
[ RUN      ] Set.BridgedFromObjC.Verbatim.Remove
[       OK ] Set.BridgedFromObjC.Verbatim.Remove
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.Remove
[       OK ] Set.BridgedFromObjC.Nonverbatim.Remove
[ RUN      ] Set.BridgedFromObjC.Verbatim.RemoveAll
[       OK ] Set.BridgedFromObjC.Verbatim.RemoveAll
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.RemoveAll
[       OK ] Set.BridgedFromObjC.Nonverbatim.RemoveAll
[ RUN      ] Set.BridgedFromObjC.Verbatim.Count
[       OK ] Set.BridgedFromObjC.Verbatim.Count
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.Count
[       OK ] Set.BridgedFromObjC.Nonverbatim.Count
[ RUN      ] Set.BridgedFromObjC.Verbatim.Generate
[       OK ] Set.BridgedFromObjC.Verbatim.Generate
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.Generate
[       OK ] Set.BridgedFromObjC.Nonverbatim.Generate
[ RUN      ] Set.BridgedFromObjC.Verbatim.Generate_Empty
[       OK ] Set.BridgedFromObjC.Verbatim.Generate_Empty
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.Generate_Empty
[       OK ] Set.BridgedFromObjC.Nonverbatim.Generate_Empty
[ RUN      ] Set.BridgedFromObjC.Verbatim.Generate_Huge
[       OK ] Set.BridgedFromObjC.Verbatim.Generate_Huge
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.Generate_Huge
[       OK ] Set.BridgedFromObjC.Nonverbatim.Generate_Huge
[ RUN      ] Set.BridgedFromObjC.Verbatim.EqualityTest_Empty
[       OK ] Set.BridgedFromObjC.Verbatim.EqualityTest_Empty
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.EqualityTest_Empty
[       OK ] Set.BridgedFromObjC.Nonverbatim.EqualityTest_Empty
[ RUN      ] Set.BridgedFromObjC.Verbatim.EqualityTest_Small
[       OK ] Set.BridgedFromObjC.Verbatim.EqualityTest_Small
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.EqualityTest_Small
[       OK ] Set.BridgedFromObjC.Nonverbatim.EqualityTest_Small
[ RUN      ] Set.BridgedFromObjC.Verbatim.ArrayOfSets
[       OK ] Set.BridgedFromObjC.Verbatim.ArrayOfSets
[ RUN      ] Set.BridgedFromObjC.Nonverbatim.ArrayOfSets
[       OK ] Set.BridgedFromObjC.Nonverbatim.ArrayOfSets
[ RUN      ] Set.BridgedToObjC.Verbatim.Count
[       OK ] Set.BridgedToObjC.Verbatim.Count
[ RUN      ] Set.BridgedToObjC.Verbatim.Contains
[       OK ] Set.BridgedToObjC.Verbatim.Contains
[ RUN      ] Set.BridgingRoundtrip
[       OK ] Set.BridgingRoundtrip
[ RUN      ] Set.BridgedToObjC.Verbatim.ObjectEnumerator.FastEnumeration.UseFromSwift
[       OK ] Set.BridgedToObjC.Verbatim.ObjectEnumerator.FastEnumeration.UseFromSwift
[ RUN      ] Set.BridgedToObjC.Verbatim.ObjectEnumerator.FastEnumeration.UseFromObjC
[       OK ] Set.BridgedToObjC.Verbatim.ObjectEnumerator.FastEnumeration.UseFromObjC
[ RUN      ] Set.BridgedToObjC.Verbatim.ObjectEnumerator.FastEnumeration_Empty
[       OK ] Set.BridgedToObjC.Verbatim.ObjectEnumerator.FastEnumeration_Empty
[ RUN      ] Set.BridgedToObjC.Custom.ObjectEnumerator.FastEnumeration.UseFromObjC
[       OK ] Set.BridgedToObjC.Custom.ObjectEnumerator.FastEnumeration.UseFromObjC
[ RUN      ] Set.BridgedToObjC.Custom.ObjectEnumerator.FastEnumeration.UseFromSwift
[       OK ] Set.BridgedToObjC.Custom.ObjectEnumerator.FastEnumeration.UseFromSwift
[ RUN      ] Set.BridgedToObjC.Verbatim.FastEnumeration.UseFromSwift
[       OK ] Set.BridgedToObjC.Verbatim.FastEnumeration.UseFromSwift
[ RUN      ] Set.BridgedToObjC.Verbatim.FastEnumeration.UseFromObjC
[       OK ] Set.BridgedToObjC.Verbatim.FastEnumeration.UseFromObjC
[ RUN      ] Set.BridgedToObjC.Verbatim.FastEnumeration_Empty
[       OK ] Set.BridgedToObjC.Verbatim.FastEnumeration_Empty
[ RUN      ] Set.BridgedToObjC.Custom.FastEnumeration.UseFromSwift
[       OK ] Set.BridgedToObjC.Custom.FastEnumeration.UseFromSwift
[ RUN      ] Set.BridgedToObjC.Custom.FastEnumeration.UseFromObjC
[       OK ] Set.BridgedToObjC.Custom.FastEnumeration.UseFromObjC
[ RUN      ] Set.BridgedToObjC.Custom.FastEnumeration_Empty
[       OK ] Set.BridgedToObjC.Custom.FastEnumeration_Empty
[ RUN      ] Set.BridgedToObjC.Count
[       OK ] Set.BridgedToObjC.Count
[ RUN      ] Set.BridgedToObjC.ObjectEnumerator.NextObject
[       OK ] Set.BridgedToObjC.ObjectEnumerator.NextObject
[ RUN      ] Set.BridgedToObjC.ObjectEnumerator.NextObject.Empty
[       OK ] Set.BridgedToObjC.ObjectEnumerator.NextObject.Empty
[ RUN      ] Set.BridgedToObjC.MemberTypesCustomBridged
[       OK ] Set.BridgedToObjC.MemberTypesCustomBridged
[ RUN      ] Set.BridgingRoundTrip
[       OK ] Set.BridgingRoundTrip
[ RUN      ] Set.NSSetToSetConversion
[       OK ] Set.NSSetToSetConversion
[ RUN      ] Set.SetToNSSetConversion
[       OK ] Set.SetToNSSetConversion
[ RUN      ] Set.SetUpcastEntryPoint
[       OK ] Set.SetUpcastEntryPoint
[ RUN      ] Set.SetUpcast
[       OK ] Set.SetUpcast
[ RUN      ] Set.SetUpcastBridgedEntryPoint
[       OK ] Set.SetUpcastBridgedEntryPoint
[ RUN      ] Set.SetUpcastBridged
[       OK ] Set.SetUpcastBridged
[ RUN      ] Set.SetDowncastEntryPoint
[       OK ] Set.SetDowncastEntryPoint
[ RUN      ] Set.SetDowncast
[       OK ] Set.SetDowncast
[ RUN      ] Set.SetDowncastConditionalEntryPoint
[       OK ] Set.SetDowncastConditionalEntryPoint
[ RUN      ] Set.SetDowncastConditional
[       OK ] Set.SetDowncastConditional
[ RUN      ] Set.SetBridgeFromObjectiveCEntryPoint
[       OK ] Set.SetBridgeFromObjectiveCEntryPoint
[ RUN      ] Set.SetBridgeFromObjectiveC
[       OK ] Set.SetBridgeFromObjectiveC
[ RUN      ] Set.SetBridgeFromObjectiveCConditionalEntryPoint
[       OK ] Set.SetBridgeFromObjectiveCConditionalEntryPoint
[ RUN      ] Set.SetBridgeFromObjectiveCConditional
[       OK ] Set.SetBridgeFromObjectiveCConditional
[ RUN      ] Set.init(SequenceType:)
[       OK ] Set.init(SequenceType:)
[ RUN      ] Set.init(arrayLiteral:)
[       OK ] Set.init(arrayLiteral:)
[ RUN      ] Set.isSubsetOf.Set.Set
[       OK ] Set.isSubsetOf.Set.Set

--
Command Output (stderr):
--
Traceback (most recent call last):
  File "<src>/swift/validation-test/stdlib/../../utils/line-directive", line 86, in <module>
    run()
  File "<src>/swift/validation-test/stdlib/../../utils/line-directive", line 69, in run
    line = command.stdout.readline().decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 17: ordinal not in range(128)

--

********************
FAIL: Swift :: stdlib/UnicodeTrieGenerator.gyb (7380 of 7672)
******************** TEST 'Swift :: stdlib/UnicodeTrieGenerator.gyb' FAILED ********************
Script:
--
rm -rf <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/UnicodeTrieGenerator.gyb.tmp && mkdir -p <src>/build/Ninja-ReleaseAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/stdlib/Output/UnicodeTrieGenerator.gyb.tmp && <src>/swift/validation-test/stdlib/../../utils/gyb <src>/swift/validation-test/stdlib/UnicodeTrieGenerator.gyb | FileCheck <src>/swift/validation-test/stdlib/UnicodeTrieGenerator.gyb
--
Exit Code: 2

Command Output (stderr):
--
Traceback (most recent call last):
  File "<src>/swift/validation-test/stdlib/../../utils/gyb", line 3, in <module>
    gyb.main()
  File "<src>/swift/utils/gyb.py", line 1074, in main
    ast = parseTemplate(args.file.name, args.file.read())
  File "<src>/swift/utils/gyb.py", line 943, in parseTemplate
    return Block(ParseContext(filename, text))
  File "<src>/swift/utils/gyb.py", line 599, in __init__
    self.children.append(Node(context))
  File "<src>/swift/utils/gyb.py", line 671, in __init__
    self.code = compile(source, context.filename, evalExec)
  File "<src>/swift/validation-test/stdlib/UnicodeTrieGenerator.gyb", line 56
    print 'PerfectlyCompressableProperty'
                                        ^
SyntaxError: invalid syntax
FileCheck error: '-' is empty.

--

This is great work! Thanks!! :100:
Would it make sense to split this pull request up? It would be easier to review and merge the changes if they were made in smaller chunks.

@modocache Indeed, it’s become a bit unwieldy, and I‘ve let my fork fall behind. Rather than try to tidy this up, I’m going to break it up into smaller PRs and refer back to this one.

@alexwlchan Would you mind resolving the conflicts and squashing into one commit?

@gribozavr Rebased against master and tidied up the commit history, so hopefully it’s a bit easier to review now. Pretty sure it’s still broken on Python 3 – I think there are some string subtleties that the automated conversion tools won’t pick up – but I think this time it should still be working on Python 2.
I don’t have time to do a test right now; I’ll try to do one later.

@akyrtzi How do I test tools/SourceKit/bindings/python/sourcekitd/capi.py ?

@alexwlchan I have been reviewing your patch, and I think that in order to make progress it would be better to handle it slightly differently.
For example, I don't know how to test all Python scripts (like the SourceKit Python bindings).  The easiest thing to do would be to split the pull request into multiple:

The first PR would update the Python scripts that are on the critical path to get build-script -RT passing.

Other pull requests update the rest of the Python scripts, one PR per script.

It would be really easy to review and test the first PR because all scripts are exercised during the build.  Other PRs will be reviewed in a distributed fashion by the people who wrote them or use them, so they should be able to verify that the change does not break them.
I'm sorry for the back-and-forth here, but I'm concerned about breaking scripts that I don't use or even don't know how to use.

@gribozavr That sounds sensible. I’m going to close this PR, and then reopen other PRs as necessary. I didn’t realise quite how large a change this would be when I started.

Thanks @alexwlchan!

@akyrtzi How do I test tools/SourceKit/bindings/python/sourcekitd/capi.py ?
@gribozavr This was experimental, it has no tests and no-one depends on it AFAIK. I'm not sure it still works. If someone wants to enhance it I recommend using python nose tests, like the libclang python tests: http://llvm.org/svn/llvm-project/cfe/trunk/bindings/python/tests/cindex/

Thanks, @akyrtzi, good to know!


Break out validation rules to their own dll

So that this can be used by things like choco

Probably should add the engine as well - with "strict" turned on for the validator.


Edit the codebase to that a user can input one image and have it analyzed for dark pattern.

Time: 3 days
Dependencies: main.py, run_dp_detection.
Acceptance Criteria: Bounding box produced on one image if there are dark patterns.

Edited code in the bash script so that instead of finding 15 files under the one inputted by the user, it only uses the one inputted by the user.


Correction for raw affiliation School of Management, Economics and Mathematics, King’s University College at Western University Canada, London, Canada

Correction needed for raw affiliation School of Management, Economics and Mathematics, King’s University College at Western University Canada, London, Canada
raw_affiliation_name: School of Management, Economics and Mathematics, King’s University College at Western University Canada, London, Canada
new_rors: ;02grkyz14
previous_rors: 048f5b183;02grkyz14
works_examples: W4387872219;W4390237365
contact: 8b8a943e913fa727b061bbb653f18716:4dcafc0d4010d744 @ uwo.ca

This issue was accepted and ingested by the OpenAlex team on 2024-12-21. The new affiliations should be visible within the next 7 days.


Scaladoc lists

From @charpov on September 15, 2015 16:19
I'm using lists in Scaladoc:
      /** Documentation
        *
        *  - item 1
        *  - item 2
        */

Note the two spaces between * and - item 1, which are necessary to make it a list.  My problem is that scalariform reduces these two spaces to one, thus destroying the list structure.
Is there a way to deal with that?
Copied from original issue: daniel-trinh/scalariform#78

From @charpov on November 24, 2015 21:27
No news on this?  It continues to mess up my code between sbt and IntelliJ IDEA.  Is there a way to turn off scalariform entirely for scaladoc comments?

From @daniel-trinh on November 30, 2015 8:0
It looks like its possible to modify the code to not strip out extra leading whitespace.
There's a few things that can be done:

Special case when the first character is the start of a scaladoc list to not strip leading whitespace
Add a new preference to allow no formatting of comments
Change the behavior in general to not strip out leading whitespace
Adding another preference to enable or disable trimming of whitespace in comments

The first option seems to be the "smartest", although there seem to be other reasons why someone might want to disable commenting on comments, so that might be useful to implement regardless. Not stripping out whitespace might be OK too, will need to think about this some more what the best solution is.

I'm fairly certain this was fixed in #146 . This was a bug in the handling of leading space when you enabled PlaceScaladocAsterisksBeneathSecondAsterisk. If @charpov wants to verify, that would be great - but closing for now; scaladoc comments have leading whitespace preserved in all cases now.


Add a wizard fonctionnality

This project is no longer maintained, and should really be incorporated in the official release:
Bootstrap Application Wizard
https://github.com/amoffat/bootstrap-application-wizard
Cheers

We currently have no plans to build or support such a component. Thanks though!


Grocery crud's "set relation" method ruins form layout

If you add Grocery crud's "set relation"  to Settings Controller like:
  ```
   $crud->set_relation('name','some_table','some_table_s_field');
        // render output result
        $output = $crud->render();
        $this->load->view('admin/settings/v_Settings', (array) $output);

the droplist is shown without chosen style and forms is ruined, maybe it has to do with some javascript?

If you add your own style to drop-down field you may need to create it by call back.
Did you ask me about your drop-down value populate from relation is broken or style not working?


Fix annotations support on non-webgl browsers

The annotation features used by Geonotebook are only supported by geojs's webgl renderer.  It is not uncommon for webgl to be broken on some systems and is often difficult to diagnose and fix.  To support fallback to canvas for users pulling from docker hub, this switches the default map renderer to openlayers.

LGTM +2


docs(guide/server-communication): fix links to Headers & RequestOptions

For some reason, this was fixed already. Sorry about that.


Some data types are not rendered as expected on showcase page

As a Publisher, I want to see raw data in the preview tables without any conversions so I know how exactly data looks like before downloading/working with it.
As described in https://github.com/datahq/datahub-qa/issues/131, user provided raw data is being converted while rendering on showcase page.
Acceptance criteria

[x] the preview table renders raw data in https://datahub.io/anuveyatsu/all-schema-types

Tasks

[x] analysis and investigations

[x] reproduce running showcase-js and dp-render-js libraries locally to find the source of conversions
[x] think of the best way of resolving it

[x] implement the fixes
[x] deploy frontend

FIXED, see https://datahub.io/anuveyatsu/all-schema-types


long type not supported

you could use double instead

Yes, but it would be cleaner, if the library would support it

@polyakov-v support has been added in the latest version.


Possibility to remove tag

It would be very good that there is a possibility to remove one tag (same as on InputTags with "x" button), but without an option, for a user to add new tags.

@d3xt3r2909 now you can create a popupmenu with the remove function

Youp, but with popupmenu you have one additional tap for user

@d3xt3r2909 I will insert this feature in later versions

@d3xt3r2909  new version 0.4.2.
check and leave feedback


Correctly save current buffer before reverting and add goto

Since we are switching the buffer context in order to revert, we need to
properly handle switching back so other functions operate normally.
This clears the 'error in process filter: Wrong type argument: processp, nil'
error we were seeing in:
https://github.com/trotzig/import-js/pull/61
Also, implements goto:
Make goto work with Emacs
While experimental and not necessarily bound to import-js, implementing
goto using the same underlying package finding logic is very simple and
extremely useful.
Implement the 'goto' functionality with Emacs, so you can easily
navigate to packages with a simple keystroke.

+1


feat: remove SoloRequest

This pull request is a suggestion at removing SoloRequest. Honestly, I don't think it's useful to make developers have to differentiate Request and SoloRequest.
The two elements that made a SoloRequest work were moved to Request, and SoloRequest was marked as deprecated.
I'm willing to PR the docs if this is accepted.

Hey @innocenzi many thanks for your PR and your suggestion 🤠
I am however going to have to respectfully decline this proposal. I've heard from numerous people that the SoloRequest is a very useful class for people to have, and I've found it useful myself personally too. You're free to carry on using regular requests without having to use the SoloRequest class in your application.
Additionally, I wouldn't be able to merge this PR anyway as it would involve a breaking change. ☺
Have a great day!

Hey @Sammyjo20, thanks for the answer!
I made sure that the pull request is not a breaking change—I use SoloRequest myself more than regular connector requests!
The intent of the PR is to avoid having to make the developer think what base Request class they should extend, whether a normal request or a request associated with a connector is needed. I also always thought that the term "solo" request was an odd word for something that's just a regular request. 😄
Could you perhaps take a second look at what the pull request does? It basically merges SoloRequest into Request, but keeps SoloRequest accessible to avoid the breaking change. Let me know if you need more clarification.

Hey @innocenzi appreciate the clarification of the PR. I've taken a second look, and I've decided that I would like to keep the SoloRequest in Saloon v3.
The SoloRequest class exists because the regular Request class does not contain the send or sendAsync methods. This is a deliberate design choice because for larger API integrations I would like people to use the Connector class to send requests - like the following:
$connector->send(new MyRequest);

Adding the HasConnector trait on the Request class means that the send and sendAsync methods would be available for every single request, which I'm not a fan of because that goes against Saloon v2+'s design choice and makes it difficult for the developer to choose which way they should send a request. Additionally, someone might get confused and accidentally call send on a request that doesn't contain a full request URL, resulting in an error. This is why I decided to make the SoloRequest. As explained in the documentation:

While Saloon's typical setup of a connector and requests is great, sometimes all you need is to make a single request to a service. For scenarios like these, you may create a "SoloRequest" instead of making a connector and a single request. This saves you from having to create additional classes.

The SoloRequest uses the HasConnector class because when using a SoloRequest you aren't likely going to have a Connector class. This is why the send and sendAsync methods exist only on the SoloRequest class.
(new MySoloRequest)->send();

All the best
Sam

While I don't agree with the design choice, I understand and respect the reasoning 👍
Thanks for taking a second look!


Enable multiple GA instances with different IDs

This allows usage of multiple google analytics trackers inside and outside of analytics kit.

Sorry, just saw this PR. Would you mind adding a second instance to https://github.com/twobitlabs/AnalyticsKit/blob/master/AnalyticsKit/AnalyticsKitTests/AnalyticsKitTests.m just to verify that both are getting events?

Done, hope its what you meant.

Looks good, thx!


Routing reneval segment bug

When I have Batflat in subfolder e.g domain.com/batflat i can't go into post list from tag named "batflat" in blog.
example:
domain.com/batflat/blog/tag/whatever            // work
domain.com/batflat/blog/tag/batflat                // error 404
It's about the name of the subfolder not the "batflat" name:
domain.com/cms/blog/tag/whatever            // work
domain.com/cms/blog/tag/cms                    // error 404
The error occurs in freshly installed cms
In debug mode last query is: SELECT * FROM pages WHERE slug LIKE ? AND lang = ?
as if  i was asking about a non-existent nested subpage like: domain.com/page/notexist. No blog module query.

Hi
It's a known issue. I will take a look at this.


Update Unity version for demo app and tests to 2021.3.1f1 (close #49)

Updates the version of Unity in demo app and tests to the latest LTS that is also supported by the CI scripts. This also bumps dependencies in tests and demo app.

Thanks for your pull request. Is this your first contribution to a Snowplow open source project? Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
:memo: Please visit https://docs.snowplowanalytics.com/docs/contributing/contributor-license-agreement/ to learn more and sign.
Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


IDC - Implementing the nnUNet Task024 Whole Prostate Gland MR (T2 only) Model

This is my implementation of the nnUNet Whole Prostate MR (Task024) model, trained on Promise12 challenge data.
The model is single-modality, namely T2, and segments the whole prostate gland.
Building and testing of the model dockerfile is next.

@LennyN95 we were just discussing this here, it should not be very difficult to have a setup where test data is specified in a config file, we have a utility script that pulls those files from IDC, and run a github action on the PR to build the image and run the tests. As long as the model can run on CPU, it should be quite helpful, since it will be fully automatic. As @ccosmin97 points out, there may be not enough RAM, but maybe this is worth a try? And even if we cannot use GA, this framework can be helpful to simplify regression testing of a given model. Maybe we can discuss at the next meeting.

it should not be very difficult to have a setup where test data is specified in a config file, we have a utility script that pulls those files from IDC, and run a github action on the PR to build the image and run the tests

@fedorov I'm working on a modular pipeline that automates these tasks. I can demonstrate the approach in the next meeting, feedback & suggestions are welcome!

@LennyN95 I noticed that the checks fail under the last commit, any idea which file causes the issue? is it the meta.json? maybe the "tables" formatting?

@LennyN95  Here are the test results:
Output segmentation looks fine by me.
/test
sample:
  idcv: Data Release 17.0 December 04, 2023
  data:
    - SeriesInstanceUID: <IP_ADDRESS>.4.1.145<IP_ADDRESS>.133402083824451909801205388628359333258
	    aws: s3://idc-open-data/a4763cd6-341d-4659-bb33-73775e12a0c3
	    path: test_data/
reference:
  url: https://www.dropbox.com/scl/fi/zzat1aiorbrqo4p8y5u2o/output.zip?rlkey=5fg38ey0i8986spoj7h2svond&dl=0
  input_data_url: https://www.dropbox.com/scl/fi/druwcayoohb1n5hp57u8u/input.zip?rlkey=t3ju42511h22jrycqh2246lk0&dl=0
notes:
This model takes a single-modality input, namely T2 sequence. Steps below are included to show how to create the input model case folder :
mkdir -p test_data/
s5cmd --no-sign-request --endpoint-url https://s3.amazonaws.com cp 's3://idc-open-data/a4763cd6-341d-4659-bb33-73775e12a0c3/*' test_data/

mkdir -p out_data

docker build --no-cache --build-arg MHUB_MODELS_REPO=https://github.com/ccosmin97/models.git::nnUNet_task024_promise -t dev/nnunet_task024_promise:latest

docker run -it --rm  -v /home/exouser/Documents/mhub_task024_nnunet/test_dir/test_data/:/app/data/input_data:ro \
-v /home/exouser/Documents/mhub_task024_nnunet/test_dir/out_data/:/app/data/output_data \
dev/nnunet_task024_promise:latest --debug

/test
sample:
  idc_version: 17.0
  data:
  - SeriesInstanceUID: <IP_ADDRESS>.4.1.145<IP_ADDRESS>.133402083824451909801205388628359333258
    aws_url: s3://idc-open-data/a4763cd6-341d-4659-bb33-73775e12a0c3
    path: dicom

reference:
  url: https://www.dropbox.com/scl/fi/zzat1aiorbrqo4p8y5u2o/output.zip?rlkey=5fg38ey0i8986spoj7h2svond&dl=0


import/no-unresolved with SystemJS plugins

I am using eslint-import-resolver-jspm but don't get it running with SystemJS plugins. For SystemJS plugins you have to add a ! at the end of the import.
For example I am using plugin-text and import a file like so
import myText from './test-file.hml!text';
Is there a way I can tell eslint-plugin-import to check that the file exists but don't parse the content? I played around with "import/ignore" and import/no-unresolved: [2, { ignore: ['\.html$'] }] but don't get it running 😕
If I set ignore in my rules the file will be completely ignored. That's what I want. But it doesn't check if the file exists. So I can write anything I want. If I only set "import/ignore": ['.html$'], it gives me an error because it can't find the file test-file.html!text' on the file system (it should only look for test-file.html and not for test-file.html!text)

I wonder if the jspm resolver is removing the plugin reference from the path.
Also, no-unresolved should ignore the ignore setting, IIRC. it would be better named no-parse or something, probably. 😅

Hey, I am working through a similar issue, any further progress on this one?

@CharlieGreenman I was going to recommend mentioning over at the resolver project, but: https://github.com/nfl/jspm-resolve/issues/7
So I don't understand the problem. Where is SystemJS coming into play? Is it plugged into jspm? or somewhere else?
I don't know anything about SystemJS or jspm, unfortunately, so it's unclear to me what needs to happen where to resolve this.

Closing; if this is still relevant, and if the relevant resolver can't handle this, and if no-unresolved still isn't using the resolvers, then this can be reopened.


external-dns for multiple KOPS cluster in AWS but single private zone

I am using multiple KOPS cluster --topology=private with internal load balancer.
I have external-dns installed on cluster1   and cluster2 but both are using myzone.com (private zone)
When I am exposing services using AWS interval LB , I can see A records and TXT records are created but deleted for one cluster. I guess both external-end POD in two different cluster trying to sync and delete and recreate records in infinity loop.
Could you please provide me the best way to use external-dns with multiple KOPS cluster in AWS without deleting records .
I know I can disable the dns record deletion using upsert-only but I want to use delete as it is helpful to clear services which are not available.

Can you give an example of what is happening? I don't see why there should be a problem with running external DNS on two clusters as long as you don't define the same records on both clusters.

@sam0104 - @Raffo is correct. We do this in our environment by setting a different txtPrefix and txtOwnerId per deployment of external-dns. This lets each external-dns deployment know which records it can / can't manage.


Selenoid instead of selenium grid

Is it possible to use Selenoid instead of Selenium Grid for hub?

Isn't Selenoid meant for browser testing? Does it apply for mobile testing with Appium?

Yes, this tool supports mobile testing with Appium. Selenoid works similarly to Selenium Grid with slight differences in settings.

for example our config files for testing mobile

I have to check how the configurations work for Selenoid but currently there is no support - in general I've implemented a custom Appium grid in GADS itself that has a tighter integration, Selenium Grid is just a possible option. What is the benefit of using Selenoid in the context of mobile automation? I don't know if it makes sense to try and support it :)

It seems that yes. Selenoid is the equivalent of Selenium Grid, but it’s easier to configure and as far as I know there are no memory leaks and no reboot service required, it works more stably. Also Selenoid is written in Go

It is not on my current plans to integrate with Selenoid but I will check it out - if it is not very hard to integrate I will add it at some point! GADS does not just start Appium servers but aims to be a unified solution because I am all alone and not a very good developer, so it is hard to make it very flexible to allow people to use it any way they want 😅


Remove _workspace

There is no reason to put them into git repo, 'cause they come from other projects, you don't edit it or modify it in this project, no reason to track and waste history and space, what do you think?
Ref:
http://stackoverflow.com/questions/26334220/should-i-commit-godeps-workspace-or-is-godeps-json-enough

there is a bug that i found when i compile lfs-test-server in my local environment (golang version 1.3.3 ).
for code snippet mgmt.go both line 53 and 155 should change "r *Http.Request " to  "r *request"  accoring  to the golang http module documents : "func (r *Request) BasicAuth() (username, password string, ok bool)"

@PeterDaveHello our standard practice is to fully vendor our dependencies. This has a few advantages for us: it allows build and deploy systems to not depend on third party systems,  if the referenced repositories are removed we can still build and deploy, and it allows us to see what changed in the dependencies when they're updated without having to go track them down elsewhere. It isn't a terribly big deal in this test repo to go one way or the other, that's just the general way that we work on our production code and it carries over here.
@tomzhang I think you're seeing that error because compiling this code will require go 1.4. The BasicAuth function was not present in go 1.3 or earlier. I will add a note to the build instructions about Go version requirements, thanks!


feat(react-ds): add type props in ts

Infos
Task
What is being delivered?

Add prop type in Venice Button typing

What impacts?

Typing in Button of React DS Venice

Reversal plan

Revert merge

Evidences

Before
After

DoD checklist

[x] Related code finished
[x] Documentation updated (if exists)
[x] Unit tests written and passing
[ ] Any configuration or build changes documented
[x] Project builds without errors
[x] Lint errors fixed
[ ] Test a lib's generated build inside projects !important
[ ] Update icon files (woff, woff2 and etc) from projects (if necessary).

question: Isn't it better to make the button accept any attribute coming from the html? As I understand it, the wrapper already accepts everything. 🤔

Done in caf6a1b (#609) @igorwessel


Upgrade Leder Signer to be able to sign tx besides celo legacy

Expected Behavior
can sig eip1559 and the new celo tx with feecurrecy but no gatewayfee
Current Behavior
only signs celo legacy transactions

needs someone smarter than me

@0xarthurxyz can you help figure out who has the knowledge of C and crypto needed to implement this? / who will own?

Thanks for logging this here @aaronmgdr!
Brief note for future reference that @timmoreton kindly assigned @kobigurk and took ownership to ping him on TG.

Asked for an owner (re point of contact with Kobi) on Slack.

from @marekolszewski

On the topic of ledger support,
@ericnakagawa  is working with a dev shop called Blooo right now to add support for newer transaction types to our ledger app. I would check in with him to confirm that Blooo is not adding anything that might be deprecated soon.

https://clabsco.slack.com/archives/C014SRL4URY/p1709335978942099?thread_ts=1709302433.634719&cid=C014SRL4URY


Fix NodePort e2e test

In NodePort e2e test, a test Service and its backend Pod are
created. The match label is app=agnhost. However, the test
client is also a Pod that has label app=agnhost, but netexec is
not running on the test client. As a result, the test has a 50%
chance of failing. This PR fixes the issue by using a busybox Pod
as test client.
Signed-off-by: Hongliang Liu<EMAIL_ADDRESS>

Codecov Report

Merging #3347 (25fde93) into main (3f30d84) will decrease coverage by 21.24%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main    #3347       +/-   ##
===========================================
- Coverage   60.85%   39.60%   -21.25%
===========================================
  Files         266      303       +37
  Lines       26520    38276    +11756
===========================================
- Hits        16139    15161      -978
- Misses       8591    21595    +13004
+ Partials     1790     1520      -270

Flag
Coverage Δ

integration-tests
36.26% <ø> (?)

kind-e2e-tests
?

unit-tests
41.60% <ø> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.


IOS.inde in example folder is empty

app.js file not available.
index.ios also.
updated repo.
active admin

Not Active admin,so I am closing the issue.


fix: normalize path regex

only error/warning have [ERROR IDENTIFIER]

seems like the codereport requires pygments<2.12, but idf requires 'pygments<3.0.0,>=2.13.0'

@radimkarnis PTAL

Thanks @radimkarnis. I rebased this to see if the checks pass. I'm merging it now.

thank you for taking care of it!


Not able to install in python 3.6

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-v_neh43p/supervisor/

/tmp/pip-build-v_neh43p/supervisor/

That seems to be an error installing the supervisor package, not supervisor_twiddler.  At this time, the stable releases of supervisor do not yet support Python 3.  However, it can be installed on Python 3 from its git repository.


Issue: vectorstore hence indexes and embeddings are persisting when they should not be.

Issue you'd like to raise.
langchain==0.0.250
platform = macOS
Python version 3.11
I am not sure if this is by design, I think it is not and hence reporting this as an issue.
When looping through a list of pdf files to get a summary for each I am creating an index using - VectorstoreIndexCreator().from_documents(pages).
The issue is for each subsequent file, data(documents) from previous file are also being retrieved to be passed on to GPT and end up in the summary.
Code:
from langchain.indexes import VectorstoreIndexCreator
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI
for file in files:
loader = PyPDFLoader(file['path'])
pages = loader.load_and_split()
        index = VectorstoreIndexCreator().from_documents(pages)

        retriever = index.vectorstore.as_retriever(search_type='mmr')
        retriever.search_kwargs['k'] = 10

        llm = ChatOpenAI()
        aca_qa = RetrievalQA.from_chain_type(
               llm=llm,
               retriever=retriever,
               chain_type='stuff',
               # return_source_documents=True,
        )
        result = aca_qa({'query': summary_query})

Suggestion:
Temp fix:
The resolution is to include - index = '' at the beginning of each loop cycle

I am seeing documents from previous files being retrieved for the same query.


Dead link in docs

The doc: https://googlecloudplatform.github.io/google-cloud-node/#/docs/google-cloud/0.41.2/compute/zone?method=createDisk
The dead link, "See a Disk resource": https://cloud.google.com/compute/docs/reference/v1/disks
I'm guessing it should link to here: https://cloud.google.com/compute/docs/reference/latest/disks

Hmm, https://cloud.google.com/compute/docs/reference/v1/disks works for me. It redirects to latest. We point it to the specific version in case the "latest" API changes upstream and we aren't implementing it yet.

Heh, must have been a service blip or something... the redirect is working for me, now

Phew! Thanks for looking out, though.


#121: changed set to return Future<()> in CacheLevel and its implementations

Hi.  This is an implementation of a fix that I described before. Surprisingly enough, it doesn't break any sample code.
Please let me know if any additional changes needed for this be merged.

Well, first of all thanks for taking care of implementing this on your own!
On a first quick look, I can see some code like
.onFailure {
  promise.fail($0)
}

could be rewritten as
.onFailure(promise.fail)

(not a big improvement, but makes it easier to read)
Also, I would like to see more mimic calls where possible instead of explicit handling for onSuccess and onFailure (also because there are missing handlings for onCancel here and there that may alter the expected behavior)
Thanks to your PR, I just found out that there has been a missing call to updateAccessDate in the DiskCacheLevel. This should actually make LRU behavior work, so I'll create an issue to fix it. (Thanks!)
Code documentation is missing sometimes, for example to mention that set now returns a Future to inform the caller of completion.
Tests are also missing, these I would like to have before merging, but I can take care of this in the next days.
One last thing, again I can take care, is the README.md + CHANGELOG.md updates to mention this change.
All in all, as I said very good contribution, if you want you can address the mentioned points, otherwise I'll do it in the next days!

onSuccess/onFailure calls conciseness is addressed now in subsequent commits as are README.md and CHANGELOG.md updates. I hope I will be able to use more mimic calls and write unit-tests during the week, but if not, I will work on this on the next weekend.
Thank you for the instant feedback!

Thanks to you for the contribution! Maybe we can find a way to share this branch so we can work together on it, without me having to merge it before it's done.. I guess there isn't an easy way to do so on Github without having the user as a full collaborator for the repo?

well, the only other way around this I see, is you making pull requests to my fork and then I merge those into the branch of this pull request, haha

I see, a meta pull request... all right then, I hope to find some time this week to write the tests and help here and there :)

Hi @explicitcall, I just checked the code a little bit more and realized that the increase in code complexity (for future maintenance), together with the fact that this solves a very specific edge case (DiskCacheLevel not being able to write for some reason, since the other CacheLevels always succeed) and that a client of a set call shouldn't care whether the call fails or succeed, make me less convinced that this should be merged in master.
Feel free to stop working on the PR, I'll check how complex it would be to update/add the tests, but I suspect it will be a lot of work for a - as said above - little gain.

Hi @vittoriom, I won't be able stop working on this at least in my fork as I need this implementation in my use case.
This fix not only helps to track errors, but also allows my code to be notified when the write on all cache levels have finished, which is crucial as well.
I've described my use case in #68:

My use case is fetching and uploading a lot of records (hundreds, potentially thousands) from a server API where most of them have references to each other. To establish valid references, I need to store ids of those records and as this process can span multiple API calls and the app has to be resilient, most of the ids are stored in the cache gradually as soon as they come in. This way the app can resume the process at any point if it crashed or was killed by a user.

In this scenario I also want to know when all the writes have fully persisted on disk, so that on crash or kill it will be able to start from where it stopped caching.

Hi @vittoriom, can you please clarify if its worth continuing my work on this PR?

Dear @explicitcall, as written above I can't see this feature merged in master in the short term. I'd keep the PR open anyway since I can't exclude it will be merged in future versions of Carlos.
Nevertheless I'm really grateful for the work you did!
Have a nice weekend

I'll close this but merge the idea behind in a future commit. I'm working on it, just the tests are missing :) Thanks a lot and sorry for the super long wait!


Watt Meter - No such file

Watt Meter issue, I am new though to this so let me know if this is just an issue on my end!
Macbook Pro M1 Pro 2021

Installed the most recent Python version and now I have a new issue, not sure if I will have to keep this downloaded to make it work or what I would have to do now?


Train the Trainer - Reporting

Title of ticket:
Description
We need to train Andrew Bonneau on the Dashboard before other users' get trained up
Dependencies
Are there any dependencies?
DOD

[ ] List the items that need to be complete for this ticket to be considered done
[ ]
[ ]
[ ]
[ ]

dupe - covered under #1685. closing


Evaluate Xim Again

I wasn't aware of the code and the second paper, so I may have missed some interesting parts last time I took a look at it.
https://github.com/hudon/refraction-hs
http://elaineshi.com/docs/bitcoin.pdf
http://forensics.umass.edu/pubs/bissias.wpes.2014.pdf

Last evaluation: https://github.com/nopara73/ZeroLink/issues/32#issuecomment-324995976

Xim [9] improves on its related previous work [6] in that
it uses a fee-based advertisement mechanism to pair partners
for mixing, and provides evidence of the agreement that can
be leveraged if a party aborts. Even in the simple case of a
mixing between two peers, Xim requires publishing several
Bitcoin transactions in the Bitcoin blockchain, which takes on
average at least ten minutes for each transaction.
In contrast, CoinShuffle++ requires to submit a single
transaction to the Bitcoin blockchain independently on the
number of peers. Source.

ZeroLink is like CoinShuffle++ and their authors came to the same conclusion as me. in my previous evaluation. I am short on time and I doubt I missed anything.


feat(line): hide splitLine

Brief Information
This pull request is in the type of:

[ ] bug fixing
[x] new feature
[ ] others

What does this PR do?
append a params as follow
splitLine: {
                 ...
                    showAsAxis: true,
                 ...
                }

when set the params,  the splitLine will only show when the axisLabel.formatter return !value = true
Fixed issues
close #12456

Details
Before: What was the problem?
the split line will show whatever the xAisx is show

After: How is it fixed in this PR?
the split line will show at the xAisx scale location

Usage
Are there any API changes?

[x] The API has been changed.

append params at axis.splitLine:
splitLine: {
                 ...
                    showAsAxis: true,
                 ...
                }

Related test cases or examples to use the new APIs
test/axis-interval-splitLine.html
Others
Merging options

[ ] Please squash the commits into a single one when merge.

Other information

@susiwen8 thanks a lot, and i think the 992ea8b may solution the performance problem


refactor: use maxlength attribute instead of max input component prop

Summary
The only usage for https://app.clickup.com/t/ad0fd3
Related: https://github.com/ArkEcosystem/laravel-foundation/pull/458

Checklist

[ ] I checked my UI changes against the design and there are no notable differences
[ ] I checked my UI changes for any responsiveness issues
[ ] I checked my UI changes in light AND dark mode
[ ] I checked my (code) changes for obvious issues, debug statements and commented code
[ ] I opened a corresponding card on Clickup for any remaining TODOs in my code
[ ] I added a short description on how to test this PR (if necessary)
[ ] Documentation (if necessary)
[ ] Tests (if necessary)
[ ] Ready to be merged

@crnkovic failing tests


Shimmie sites having proplems reading and showing tags

What steps will reproduce the problem?
Load any images from any shimmie-related site, like paheal.
What is the expected behavior? What do you get instead?
After a few images loaded, every other image has the tags broken, and the image stays like a thumbnail. It tends to just load 3 sets of double slashes, the resolution, and then the file size of the image/video/etc when this error happens. The image pretty much loads minutes afterwards, but that's still not OK for a program like this one. Here's a screenshot:

What version of the program are you using? On what operating system?
Grabber 5.2.2 for Windows.
Please provide any additional information below
I tried removing my blacklist of tags, to see if that worked, but nothing. I also tried keeping the search empty, to see if it breaks due to choice tags, but also nothing. Same problem keeps happening. I then, finally decided to uninstall the program, and deleted all AppData files related to Grabber, and nothing. Finally, I tried using credentials, with an account, maybe that could've helped (I probably did it wrong there, maybe not), tried and failed. The images taken from shimmie also seem to load really slowly, compared to any other site supported by Grabber. I hope there is a solution found, because I feel clueless about this issue.

Your speed issue might actually be the cause you are seeing this. rule34.paheal.net is blazing fast for me, so I indeed see the problem you describe, but only for a split second before the // mess gets replaced by the proper values.
Seems like the Regex parsing has some issues, so I'll push a fix for that. Nothing I can really do about the loading speed though, since I can't reproduce any speed problems the problems might be your connection to the site. Is it fast when browsing it online?

Yeah, the speed is back. It was probably server side, since the load speed issues were only going wrong for that site. But the tags still appear briefly broken, as you realized. I'll wait for your update. Thanks for the response to the matter!


Check that all chunks are accounted for

Some old installations can have extra chunks that are not accounted for, that is, tables that are children of the hypertable but are not registered in the metadata. Check that:

All children of the hypertable have a corresponding entry in _timescaledb_catalog.chunk with the correct hypertable.
All chunks that are not marked dropped in _timescaledb_catalog.chunk exists as children to the correct hypertable.

This query will list the hypertable and chunk that is missing:
SELECT
    (SELECT format('%I.%I', schema_name, table_name)
       FROM _timescaledb_catalog.hypertable
      WHERE id = hypertable_id) AS hypertable,
    format('%I.%I', c.schema_name, c.table_name) AS chunk
  FROM _timescaledb_catalog.chunk c LEFT JOIN information_schema.tables t
    ON table_schema = schema_name
   AND c.table_name = t.table_name
 WHERE NOT dropped AND t.table_name IS NULL;


Artists to Follow rail should be displayed for logged out users

We should show the Artists to Follow rail for logged out users, but without the "For You" tab.
Original issue for reference: https://github.com/artsy/force/issues/77

So this guy is still a lab feature, "fixing" this will make it available to everyone. Cool?


Makes history-substring-search use term specific up and down buttons.

Fixes #2089 and problems with some linux distributions and/or terminals.

@naegelejd: vi-mode appears broken with respect to the terminfo usage this PR introduced. #1533 introduced a logic error in zle-line-init and zle-line-finish that leaves things in the wrong smkx/rmkx mode so the terminfo bindings don't work. See discussion under #2735.


Android SDK: Failed to find package Cmake

Godot version
3.3.3
System information
Windows 7 32-bit 2GB RAM Intel HD Graphics
Issue description
I want to install Android SDK without Android Studio, because I can't. But it returns "Warning: Failed to find package "cmake;<IP_ADDRESS>88404""
Steps to reproduce
I have cmdline-tools: C:\Users\adm\Android-SDK\cmdline-tools\latest\bin
I use documentation to download android sdk "sdkmanager --sdk_root=<android_sdk_path> "platform-tools" "build-tools;30.0.3" "platforms;android-29" "cmdline-tools;latest" "cmake;<IP_ADDRESS>88404" "ndk;21.4.7075529".
<android_sdk_path> = "C:\Users\adm\Android-SDK"
It returns error "Warning: Failed to find package "cmake;<IP_ADDRESS>88404""
Minimal reproduction project
No response

I'm not sure if you still need to install Android SDK and NDK components manually. Last time I tried, Gradle installed them automatically for you.
Just run the SCons and Gradle steps as explained in the documentation.

You might have a too old Android SDK (sdkmanager specifically) which doesn't know about cmake;<IP_ADDRESS>88404. The package does exist in that version for me:
$ ./cmdline-tools/latest/bin/sdkmanager --list | grep cmake
  cmake;<IP_ADDRESS>88404 | 3.10.2       | CMake <IP_ADDRESS>88404            | cmake/<IP_ADDRESS>88404/
  cmake;3.18.1         | 3.18.1       | CMake 3.18.1                    | cmake/3.18.1/
  cmake;<IP_ADDRESS>88404                                                                     | 3.10.2       | CMake <IP_ADDRESS>88404
  cmake;3.18.1                                                                             | 3.18.1       | CMake 3.18.1
  cmake;3.6.4111459                                                                        | 3.6.4111459  | CMake 3.6.4111459


adding support for crd3

Have you run this locally?
Let us know when the promptsource PR is through!
https://github.com/bigscience-workshop/promptsource/pull/769

Hi @cjlovering! Sure, I'll keep you posted about the promptsource PR status

Hi @cjlovering the corresponding promptsource PR has been merged


feat(hover): POC for inline hover with rust

Hello everyone,
This PR provides a POC for supporting inline SQL in other languages (as described in https://github.com/joe-re/sql-language-server/issues/180).
The implementation is inspired by the tree sitter injections in nvim-tree-sitter where you can have syntax highlighting for inline languages.
The idea would be for the community to provide tree-sitter queries that define how certain parts of the code should be interpreted as SQL, as I did for rust in the POC.
The LSP can then remove all non-SQL code from a file and process only the SQL text.
The example allows the sqls hover function to be used on the following rust code:
fn main() {
    sqlx::query("select * FROM tags").fetch_all(&mut conn).await;
}

With the proposed architecture of using queries, the feature could easily be extended to other languages and their SQL libraries, as tree-sitter already provides parsers for most languages.
Please note that adding tree-sitter as a dependency would require the project to be built using cgo.
Running multiple language servers at the same this is supported (at least for neovim) and it should cause no problem if sqls will just return nothing for all positions that are not inside SQL strings.
What do you think about this idea?

I guess one can just use https://github.com/jmbuhr/otter.nvim


Cleanup aries_vcx/src/utils/constants.rs

Good first issue! You can learn bit of a Rust and the codebase by doing this
Task:

Remove unused constants
Cherry on top: reformat json constants from single-line to multi-line JSONs to make them easier to read
Divide into multiple files

Learning points:

rust imports
rust "features" - you will encounter that some constants are only being used when compiled with certain feature flag
refactoring rust code - you will see that IDEs tends to be pretty smart with refactoring Rust code and it's possible to move constants to other file and having (most) imports fixed by IDE

Hi @Patrik-Stas , I would like to work on this issue. Though I'm new to Aries.


Expect user entered addresses to fail

Connecting to a website provided by a user can fail for all sorts of reasons.
Now handling:

Generic socket errors
Connection refused
Connection reset
Host unreachable
SSL failure
Timeouts

Coverage increased (+0.04%) when pulling 5007feafb5dbf7e9e8bd1baa9700d5123801288f on bugfix-odibot-handle-transport-errors into 86ab71ae74622575e0e7151fd91a4545b17d3de5 on master.


Split up and refactor isCustomTask

Changes

In this change, we clean up the isCustomTask function:

Remove unused context.Context.
Remove the check that either TaskRef or TaskSpec is specified because we check that way before - see code.
Define the check that TaskRef is a CustomTask as a member function of TaskRef; and add tests for the member function.
Define the check that TaskSpec is a CustomTask as a member function of TaskSpec; and add tests for the member function.

There are no user-facing changes in this commit.
/kind cleanup
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

~[ ] Has Docs included if any changes are user facing~
[x] Has Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
~[ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)~
~[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release~

Release Notes
NONE

flake https://github.com/tektoncd/pipeline/issues/4169
/retest

/retest


Adding Expense model

added Expense model and related migration

We decided that the Expense model is not really needed


Update p4 to 2016.2.1457671

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

@mdouglass Just tried this today and I'm still running into issues. Looks like the sha256 was updated to
a18b0e0338d6331588afb12c57427ddf169dcf374e5e2ab000001bd0260efc1c
but should be
1e6165711902b65612637aa190e7ba09a8559f1e4e057e2b4587e7756c2bef59
Here is my install output:
==> Downloading http://filehost.perforce.com/perforce/r16.2/bin.macosx105x86_64/p4
######################################################################## 100.0%
==> Verifying checksum for Cask p4
==> Note: running "brew update" may fix sha256 checksum errors
Error: sha256 mismatch
Expected: a18b0e0338d6331588afb12c57427ddf169dcf374e5e2ab000001bd0260efc1c
Actual: 1e6165711902b65612637aa190e7ba09a8559f1e4e057e2b4587e7756c2bef59

Source (http://filehost.perforce.com/perforce/r16.2/bin.macosx105x86_64/SHA256SUMS):
6273c2841e8a1241ddee19fe0e04f7a4e3295492085689287ddf6384a88c7cc4 *p4ftpd
1e6165711902b65612637aa190e7ba09a8559f1e4e057e2b4587e7756c2bef59 *p4
ed046dad2478a4f67626adc5bcf3c08fa6571624fc4957b6ee74ccbd2ec771c7 *p4api.tgz

I forked the repo and started to prepare a pull request. Ran through the first two tests cases (brew cask audit/style) and it passed. Wasn't sure exactly how to go about including the cask's name and version. I checked the other logs in the repo and I couldn't find a good example?? Seems like most list the cask name and the pull request number (long time git user, new to github, how can I get the pull request number before requesting the pull?), but I could be wrong. What exacty should I put in my commit message?
If you wanted to fix this, that'd be fine. But I can see myself using this project a lot in the future and I always like to contribute where ever I can. Just need some tips on that last item of the task list really.

long time git user, new to github, how can I get the pull request number before requesting the pull?

You don’t Github does it automatically when maintainers merge as squash commits.

What exacty should I put in my commit message?

Something like “Updated {{cask_name}} to {{new_version}}”, or “{{cask_name}}: updated to {{new_version}}”.

Thank you very much @victorpopkov
I've submitted the pull request for the sha256 update (#27313).

So looked into this a bit and what Perforce is doing here is a bit annoying. They use a fixed URL for major/minor version number (http://filehost.perforce.com/perforce/r16.2/bin.macosx105x86_64/p4) but reuse that URL when they do a patch release.
When I made this PR, they were at Rev. P4/MACOSX105X86_64/2016.2/1457671 (2016/10/26). As of today, the same URL now returns a different binary that reports as Rev. P4/MACOSX105X86_64/2016.2/1468155 (2016/11/28) (as seen in @neybis PR #27313).
The best thing to do would probably be to contact Perforce and find out if there's a URL that homebrew can use that will not change for every patch release.

Ahhh, I didn't notice that with the patch releases, def a good catch although I'm sure it would have been obvious to me sooner or later :P I can see if I can get in touch with Perforce since the company I work for uses it.
If not perhaps the best solution would be to skip the sha check?? Maybe not since I just realized that a brew upgrade wouldn't pull down the patch release though since cask doesn't see the version change...

I sent a support ticket into perforce through my company as well. Received the following:

I have attached your details to an existing enhancement request for this functionality. Should this be implemented you will be notified by email at the time of release.
Your request will add weight to this enhancement request.

So it seems like multiple people making the request would be helpful.

Definitely, I'll make sure to submit something from my end, too, to try to push this through.
One thing I just thought about (which may not be supported by design), is supporting sha's at runtime. We could always download the file containing the sha, pull that out, and then check against that instead of hard coding a sha.
My biggest concern, if it were even possible to do this, is how brew cask would know an update was available... I'd imagine it would pull down the most recent patch build for the first install and it would pass even if a new patch build is released bc we pull the sha for the patch build out of the sha file. But I would guess that it wouldn't pull down the next patch release. I'd expect it wouldn't realize there is an updated until the minor version is bumped.


Update bootstrap js because bootstrap 3.3.7 is vulnerable

https://snyk.io/vuln/npm:bootstrap?lh=3.3.7&utm_source=lighthouse&utm_medium=ref&utm_campaign=audit

Seems okay. Are there other files that need updating as well, such as the kubectl and federation references?

/lgtm

/approve


AskPlantnet replyTo parentOfCandidate instead of Candidate

in the following example: https://bsky.app/profile/rangedan.bsky.social/post/3ktxhb6yohk22
the bot produces a reply to the parent post
it was originally design to reply to candidate (mention of the bot)
impact:

next simulations keeps the same candidate again ❎
disable trigger until a fix is provided ✔

first reply attempt removed


Resign keyboard search

Addresses #2074
Search already employed resign properly so this is only for bookmarks

Another keyboard bits the dust!