Force close = funds doomed in limbo

Background
I have around 500,000 sats stuck in limbo on a forever closing node + around 40,000 on another with the same issue due to a force close (by the other party I assume) after my hardware went down for 24 hours.
The channel has been "closing" for over a week now and the original funding tx was on the 28th of September.
Is there anything I can do or are my funds gone forever?
Your environment

LND 0.7.1
Ubuntu Server
Bitcoin Core RPC v0.18.0
Running within docker environment of BTCPayServer and using ZapDesktop

Steps to reproduce
I'm not sure how to 'reproduce' this issue, it's pretty straightforward, my channels were closed and my funds were thrown into limbo.
Expected behaviour
Node should have closed and returned my committed sats to me.
Actual behaviour
Sats have not been returned, stuck in limbo, all lightning wallet clients I've tried say that it is just "closing" nothing else I can do.

Can you provide logs for both channels?
Looking at the chain, the channel 5e1d9292acfc7230fbe948ca732c7e66ccaad73ad4bd5406a181c8367f5461c4 has not been closed yet. The channel fff0304be98a9f605b6e57fa9984b02d6c24e9aeedd0dfe1fa36d26813fd96cf was closed successfully and the money was swept by both parties.

How do I provide the logs?

They should be in your docker instance at the path ~/.lnd/logs/bitcoin/mainnet/lnd.log.

Thank you, I'll do it tomorrow afterwork. I have to sleep now.
Please check back here in 18 hours when I should have the logs for you.

From the logs:
2019-10-16 05:57:18.330 [INF] DISC: Update announcement for short_chan_id(659217694184833024), is premature: advertises height 599555, only height 599322 is known
It looks like your bitcoind node may not be fully caught up with the chain.

My node is caught up? BTCPayServer says it's synched.
Should I download a different node like Windows Node Launcher, set that up and then connect my LND to that and hopefully should all be synced back up?

If this is my problem, will linking my lightning node to a full normal node work to helping recover my limbo funds?

In the past I've seen it worked for some people, doesn't hurt to try, and you will need to let lnd do the  rescan which will take a while if you have many txs.  If it doesn't work, you really need the BTCPay people to help you out.

Hey @molxyz  I got my funds back by contacting the owner of the node and asking them to force close the channel. Cheers.


Use of the snap_common folder for runtime

Use the snap_common folder for the runtime folder, avoiding having to copy images and containers around on updates.
These are the tests done to make sure it works:

The first time you upgrade from a snap that had the images/containers in current, rather than common you lose those images and containers, so we should make these changes as soon as possible (before the snap goes to stable)

Once you have a docker snap that stores everything to common this is the testing I have done:

Upgrading a docker snap shuts down any running container, but this is what happens in docker in general
If you have set up your container to autorestart, it does so correctly after upgrade
After upgrade "docker ps -a" will show your previous containers correctly
After upgrade "docker images" show your previous images correctly

:heart_eyes:
This is what I originally wanted, but I wasn't aware SNAP_COMMON existed! :+1:


Unexpected colon in cradle example.

From the readme, trying the cradle: {stack:} example for hie.yaml I get an error with hie-core and yamllint.
https://github.com/mpickering/hie-bios/blame/master/README.md#L38
> hie-core
Starting hie-core
...
[1/6] Finding hie-bios cradle
hie-core: InvalidYaml (Just (YamlParseException {yamlProblem = "found unexpected ':'"
, yamlContext = "while scanning a plain scalar"
, yamlProblemMark = YamlMark {yamlIndex = 14, yamlLine = 0, yamlColumn = 14}}))

Trying yamllint
> cradle: {stack:}
found unexpected ':' while scanning a plain scalar at line 1 column 10

Thanks, fixed.


Exercise 3: Tests won't pass due to code examples missing a class attribute

In exercise 3, you expect the ul to have the class tasks but that's not displayed in the example code. That should be added.

This actually breaks every exercise form this point forward.


integrate running transformation on GPU

🚀 Feature
I am curious if we could integrate / run data loader transforms on GPU
Motivation
it may even speed up the training process
Pitch

Alternatives
see: https://github.com/PhoenixDL/rising
Additional context
@justusschock

Interesting. Just putting some of my thoughts here for discussion:
Applying transforms on GPU or loading data will also utilize some GPU memory, right? Most of the datasets in DL are usually in GBs, and I suspect that a lot of OOM errors might occur. Unless and until we only load a batch of data into memory, apply transforms, and then transfer back to the CPU.
From the README of rising repo:

Running augmentations on the GPU is only efficient if they can be executed in a batched fashion to maximize the parallelization GPUs can provide

I didn't see any benchmarks on the repo, but I can try on my system to see if there is any improved time and if it goes OOM.

So, we do have some support here. The InputTransform has the hook per_batch_transform_on_device which can be used to provide transforms that execute per batch on the GPU.
There is a small example here that shows the general idea: https://github.com/PyTorchLightning/lightning-flash#flash-transforms
But it would be nice to document this better and make sure it is tested and has a smooth integration with rising 😃

@krshrimali For medical images the pipeline usually is:

fetch dataset statistics (may require to load every sample once, important: don't cache it)
determine which transforms to use based on 1
load a single batch of images (can be 1-N samples)
augment them (can be very costly on CPU, especially resampling 3D images is very compute intensive)
transfer batch to GPU
do training with current batch

where the steps 3-6 are repeated for every batch in the dataset. This means that for most use-cases, the whole dataset is never loaded completely but only single batches.
Applying augmentations on the GPU is most efficient using batches rather than single samples whenever possible and can't easily be done inside the dataset (due to overheads with the cuda context, which would be created by every single dataloader worker if they were accessing the GPU).
With rising, the pipeline is

fetch dataset statistics (may require to load every sample once, important: don't cache it)
determine which transforms to use based on 1
load a single batch of images (can be 1-N samples)
transfer batch to GPU
augment them (should be less costly than on CPU, can usually be done without gradient computation/AD graph building and with proper memory management shouldn't result in an overhead once augmentation is finished, since the original batch ["pre augmentation"] doesn't have to be stored; An exception of this is of course test-time augmentation.)
do training with current batch

With those principles in mind, we designed rising and those principles were regarded, when designing the original data processing pipeline in flash (by @tchaton and me) which was then revised multiple times by @ethanwharris .
@ethanwharris In the past, there was also a possibility to apply per-sample transforms on device. Has this been removed? Because transformations like resizing (also costly for a 3d image) have to be applied per-sample if the samples are of different sizes and thus sometimes has to be done before even batching them :)

Hey @justusschock we still have the per_sample_transform_on_device although I would be interested to see some concrete use-cases / examples for it. That way we can start to benchmark as it's a source of a lot of complexity in the code and it might be that the boost you get from resizing on device is offset by not having each sample transformed in parrallel.
In general it would be nice if we had like an accelerating transforms guide that desicussed on_device transforms in detail and gave some concrete examples along with speedup. That would allow us to better test / support this functionality going forward.


Interview request!

Hi Aaron! I'm Sai from Console. We are a weekly open-source newsletter, and we mentioned your project last week, which got a lot of interest. So I thought it would be a great idea to chat with you and publish the interview in one of our weekly issues. So, would you be interested in doing a text-based interview? let me know what you think!

Hi Sai,
Really sorry for the very late response.
Ebi handles all my speaking requests and scheduling with my availability. I
am copying her on my reply. She will be in touch with you shortly.
Thank you,
Aaron
On Wed, Mar 8, 2023 at 11:49 AM Sai Phanindra @.***>
wrote:

Hi Aaron! I'm Sai from Console https://console.substack.com/. We are a
weekly open-source newsletter, and we mentioned your project last week,
which got a lot of interest. So I thought it would be a great idea to chat
with you and publish the interview in one of our weekly issues. So, would
you be interested in doing a text-based interview? let me know what you
think!
—
Reply to this email directly, view it on GitHub
https://github.com/tenderlove/initial-v/issues/5, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAAYNH4D37BQ5P6YBNK2R3W3DPGBANCNFSM6AAAAAAVUF562Y
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Looking forward to it!

From: Aaron Patterson @.>
Sent: Thursday, March 23, 2023 3:08 AM
To: tenderlove/initial-v @.>
Cc: Sai Phanindra @.>; Author @.>
Subject: Re: [tenderlove/initial-v] Interview request! (Issue #5)
Hi Sai,
Really sorry for the very late response.
Ebi handles all my speaking requests and scheduling with my availability. I
am copying her on my reply. She will be in touch with you shortly.
Thank you,
Aaron
On Wed, Mar 8, 2023 at 11:49 AM Sai Phanindra @.***>
wrote:

Hi Aaron! I'm Sai from Console https://console.substack.com/. We are a
weekly open-source newsletter, and we mentioned your project last week,
which got a lot of interest. So I thought it would be a great idea to chat
with you and publish the interview in one of our weekly issues. So, would
you be interested in doing a text-based interview? let me know what you
think!
—
Reply to this email directly, view it on GitHub
https://github.com/tenderlove/initial-v/issues/5, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAAYNH4D37BQ5P6YBNK2R3W3DPGBANCNFSM6AAAAAAVUF562Y
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

—
Reply to this email directly, view it on GitHubhttps://github.com/tenderlove/initial-v/issues/5#issuecomment-1480294439, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHCPNI274V5TTMN2YJCLB63W5NWNRANCNFSM6AAAAAAVUF562Y.
You are receiving this because you authored the thread.Message ID: @.***>


feat(client): create editor system theme

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Setting the default theme for the editor disabled the ability to add system theme, because it requires magic 🤷. I don't know how to word it.
Anyway, this should sort it, to test it:

visit /learn/2022/responsive-web-design/learn-html-by-building-a-cat-photo-app/step-23
set your prefered-scheme to light nothing should look weird
set your preferred-scheme to dark, nothing should look weird
log in
set your theme to default, nothing should look weird
set your theme to night, nothing should look weird
maybe I missed something, if you think there is more, please check it.

Ref #48757

this is so fragile, if theme change to anything anywhere, while the user is logged out, they will have two different states of the theme

Please add E2E tests to cover all the scenarios; that is the best way to keep this PR moving.

Created issue for it #49784

@raisedadead, the e2e test can be addressed by different person in a different PR.
If you are worried that a code can break the theme can be added, as I mentioned above, don't merge a PR that set a initial value for theme while there is redux store for the theme.
A good example for what I mean is the attempt logic in the same file, if someone set initial value 3 for attempts while we are storing it in redux, all the logic around it become useless.

Unfortunately, we can't have busy maintainers on the lookout for a PR that accidentally breaks things.

Unfortunately, you already do https://github.com/freeCodeCamp/freeCodeCamp/issues/48936.
The fragility do exist the moment, someone allowed a code that change the theme stored state, in a local scope, you are just blocking a PR that is small part of the system theme, and blocking my work that is following the PR, that fragility I have pointed out exist already.
True maintainers are busy, so to save some of my busy and productive time, because I could have finished the system PR, if this wasn't blocked for months, I will assign you to add the test, and if you can't add it the next week, I will remove the block label.
Of course, you can ask Naomi, Tom, Shaun, Niraj, Sem, Oliver and Ahmad for help.
But don't block my work and waste my productivity, in an issue that shouldn't be there in the first place.

You are clearly not aware of what this PR and its code trying to achieve, and again this isn't adding fragility or fixing something, it's enabling the changes in that will allow for easier themes changes in the future.
The test you want shouldn't be related to the work I am doing in this PR.


INSTALL ISSUES

hi~
I have installed a docker  but cd /DeepOSM-master  make dev  then  run  python bin/create_training_data.py  ,i have a error : File "bin/create_training_data.py", line 6, in
from src.training_data import download_and_serialize  ImportError: No module named src.training_data
can you give me  a detailed tutorial

What operating system are you using?

Can you please paste in all of your terminal input/output as a comment on this issue, starting with the initial clone of the repo, and cd into the directory?

ubuntu16.04  first  i had install  tensorflow

install a Docker Binary.
2.git DeepOSM CODE
3.cd DeepOSM   make dev   have some errors
ConnectionError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries exceeded with url: /tensorflow/linux/cpu/tensorflow-0.8.0-cp27-none-linux_x86_64.whl (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)
The command '/bin/sh -c pip install --ignore-installed -r /DeepOSM/requirements_cpu.txt' returned a non-zero code: 2
Makefile:16: recipe for target 'build' failed
4.when i run  "python bin/create_training_data.py"
File "create_training_data.py", line 6, in
from src.training_data import download_and_serialize
ImportError: No module named src.training_data

-----------------------i am sorry  ,I'm a green hand  please tell  me  details

@Hjy20255

you shouldn't have to install tensorflow, docker does it as part of make dev

It looks like you got a network error when docker tried to download tensorflow, which caused make dev to fail. Check your internet connection.

It won't work to call python bin/create_training_data.py unless make dev works successfully and puts you inside the docker machine

hi ~

cd  DeepOSM   make dev
Removing intermediate container<PHONE_NUMBER>9f
Successfully built eb7605e08e33
docker run -v pwd:/DeepOSM
-w /DeepOSM
-e CPLUS_INCLUDE_PATH=/usr/include/gdal
-e C_INCLUDE_PATH=/usr/include/gdal
-e AWS_ACCESS_KEY_ID=
-e AWS_SECRET_ACCESS_KEY=
-it deeposm /bin/bash
root@d9ae443e09af:/DeepOSM# python bin/create_training_data.py
error:
Traceback (most recent call last):
File "bin/create_training_data.py", line 89, in
main()
File "bin/create_training_data.py", line 85, in main
args.tile_overlap)
File "/DeepOSM/src/training_data.py", line 388, in download_and_serialize
naip_year).download_naips()
File "/DeepOSM/src/naip_images.py", line 51, in download_naips
create_cache_directories()
File "/DeepOSM/src/config.py", line 31, in create_cache_directories
shutil.rmtree(CACHE_PATH)
File "/usrb/python2.7/shutil.py", line 239, in rmtree
onerror(os.listdir, path, sys.exc_info())
File "/usrb/python2.7/shutil.py", line 237, in rmtree
names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/DeepOSM/data/generated/'
I can't download all source data.
-----------------------i am sorry ,I'm a green hand.

I also have the same problem OSError: [Errno 2] No such file or directory: '/DeepOSM/data/generated/'
Have you solved it?


Build with MSVC is broken

Attempt to use nlohmann_json_cmake_fetchcontent with Visual Studio result in this error:
CMake Error at CMakeLists.txt:14 (add_executable):
  Cannot find source file:

    build/_deps/json-src/nlohmann_json.natvis

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
  .hpp .hxx .in .txx .f .F .for .f77 .f90 .f95 .f03 .ispc

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

it's because of this section in the CMakeLists.txt:

## add debug view definition file for msvc (natvis)
if (MSVC)
    set(NLOHMANN_ADD_NATVIS TRUE)
    set(NLOHMANN_NATVIS_FILE "nlohmann_json.natvis")
.....

nlohmann_json.natvis needs to be added to this repo

This repository is not more useful. Indeed, since v3.11.3, you can use:
include(FetchContent)

FetchContent_Declare(json URL https://github.com/nlohmann/json/releases/download/v3.11.3/json.tar.xz)
FetchContent_MakeAvailable(json)

target_link_libraries(foo PRIVATE nlohmann_json::nlohmann_json)


SRC-7; Arbitrary Asset Metadata Standard

Type of change

New standard

Changes
The following changes have been made:

Introduces the SRC-7; Arbitrary Asset Metadata Standard

Notes

A separate standard will be defined for off-chain data retrieval
This PR will fail to compile until a new release of forc

Related Issues

Closes #18

Is there any reason to enforce metadata is stored by the same contract that mints the NFTs? It seems like this interface would support metadata for any AssetId, even if the contract doesn't own it.

Is there any reason to enforce metadata to be stored by the same contract that mints the NFTs? It seems like this interface would support metadata for any AssetId, even if the contract doesn't own it.

I believe the best practice would be to ensure that the contract that mints the asset implements the SRC-7 standard, hence requiring that the SRC-20 standard must also be implemented. With a sub id, anyone can verify that the asset was minted from the said contract. There could be security implications if another contract holds the metadata without a way to ensure that this data is valid.

With metadata, can there be a function called token_uri, the same as erc-1155? This will make it easy to get NFT data. Because Mint will be considered an NFT, and just after that, there will be a call to get metadata (.json file) to store that data offline.

With metadata, can there be a function called token_uri, the same as erc-1155? This will make it easy to get NFT data. Because Mint will be considered an NFT, and just after that, there will be a call to get metadata (.json file) to store that data offline.

While we intend to add an off-chain data retrival standard that focuses more on the implementation rather than an interface, to query a token's URI the "URI" key can be used.
For example, here we are using the "URI" key:
fn foo(contract_id: ContractId, asset: AssetId) {
     // Use the SRC-7 standard to query any arbitrary contract without
     // the need to know any other ABI implementations it may have
     let contract_abi = abi(SRC7, contract_id);

     // Define the key we want, in this case "URI"
     let key = String::from_ascii_str("URI");

     // Fetch the URI with the asset and key
     let metadata = contract_abi.metadata(asset, key);

     // Ensure the data we retrieved is what we expected
     assert(metadata.is_some());
     let string_data = match metadata.unwrap() {
          MetadataType::StringData(data) => data,
          MetadataType::BytesData(data) => revert(0),
          MetadataType::IntData(data) => revert(0),
     }
}

Here is some feedback regarding this standard (discussed with @dmihal earlier)

This information is going to be consumed by wallets, frontends and data  indexers
Not only the interface how to fetch data, but the data content itself must be standardised
You definitely want to have a method that fetches all data attributes once (to a some certain nice round limit like 256) attributes

When attributes you want to standardise

Website
Logo, various formats

Dark background
White background
Standard dimensions like 256x256, pick whatever Apple has decided
Note that sufficiently small logo files can be directly stored inline compressed on-chain, avoiding any issues with privacy leaks and exploits when fetching images

Social media links
- Twitter
- Discord
- Telegram
- etc.
Trading related

name; keypairs where to official trade the token
to avoid users landing on a phishing site through a Google

Security incident contact email

For more inspiration, look what EtherScan, CoinGecko and TrustWalllet token repository offers as a metadata about the tokens.
Happy to help you to come up with descriptions of key metadata attributes any asset should have.

Here is some feedback regarding this standard (discussed with @dmihal earlier)

Because on-chain is no longer expensive (it's not 2016 anymore) and metadata is very small, most of the data should be stored on-chain instead of an external source to avoid privacy and security issues (see here)
This information is going to be consumed by wallets, frontends and data  indexers: have you engaged to discussions on any of these groups what they want to see on the metadata standard?
Not only the interface how to fetch data, but the data content itself must be standardised
You definitely want to have a method that fetches all data attributes once (to a some certain nice round limit like 256) attributes

When attributes you want to standardise

Website

Logo, various formats

Dark background
White background
Standard dimensions like 256x256, pick whatever Apple has decided
Note that sufficiently small logo files can be directly stored inline compressed on-chain, avoiding any issues with privacy leaks and exploits when fetching images

Social media links

Twitter
Discord
Telegram
etc.

Trading related

name; keypairs where to official trade the token
to avoid users landing on a phishing site through a Google

Security incident contact email

For more inspiration, look what EtherScan, CoinGecko and TrustWalllet token repository offers as a metadata about the tokens.
Happy to help you to come up with descriptions of key metadata attributes any asset should have.

Yes absolutely agree that we should standardize attributes and keys along with what sort of data they return.
Our intent is to do this across multiple categories which become living standards building atop this one, allowing for additions over time as needed as more projects get built out onchain. The first example of this is the SRC-8 standard which defines the metadata and keys for any bridged assets.


Pass NULL pointer for cffi.FFI().from_buffer(empty array)

When linking to a C function from the python CUDA kernel, the data in a device array can be passed by the expression
ffi.from_buffer(d_ary)

However, an exception if raised if an empty array is passed. This should instead be passed as a NULL pointer. There is not the danger of dereferencing a NULL pointer as the size of the array must also be passed to the C function.

Thanks for the report. Do you have a minimal working reproducer for the issue please? Is the ffi.from_buffer being used in a JIT compiled region or from Python?

Here is an example of code that produces the error:
summation.cu:
extern "C" __device__ int summation(int *sum, int *values, int n) {
  *sum = 0;

  for(int i=0; i<n; i++)
    *sum += values[i];

  return 0;
}

code:
THREAD_COUNT = 1024

import numpy as np
from numba import cuda
import cffi

ffi = cffi.FFI()

summation = cuda.declare_device('summation', 'int32(CPointer(int32), int32)')

@cuda.jit(link=['/content/summation.cu'])
def kernel(values, sums) :
  i_thread = cuda.threadIdx.x

  sums[i_thread] = summation(ffi.from_buffer(values[i_thread]), values[i_thread].size)

def main(length) :
  values = np.random.randint(low=0, high=1024, size = (THREAD_COUNT, length), dtype=np.int32)
  d_values = cuda.to_device(values)
  d_sums = cuda.device_array(THREAD_COUNT, dtype=np.int32)

  kernel[1, THREAD_COUNT](d_values, d_sums)

  sums = d_sums.copy_to_host()

  print(np.all(values.sum(axis=1) == sums))

Running main(10) runs fine. However, main(0) produces the error from_buffer() unsupported on non-contiguous buffers (got array(int32, 1d, A)).

I've had a quick look at this - there appears to be a bug in or around cuda.to_device that is resulting in the type of the array appearing to be non-C-contiguous, which is why this error occurs - passing an empty array to ffi.from_buffer() is possible. I'll shortly try to make a simpler reproducer of the underlying issue.

A simpler reproducer of the underlying problem:
rom numba import cuda, typeof
import numpy as np

h_values = np.random.randint(low=0, high=2, size=(10, 0))
d_values = cuda.to_device(h_values)

print(typeof(h_values))
print(typeof(d_values))

assert typeof(h_values) == typeof(d_values)

which gives:
$ python simpler.py
array(int64, 2d, C)
array(int64, 2d, A)
Traceback (most recent call last):
  File "/home/gmarkall/numbadev/issues/9654/simpler.py", line 10, in <module>
    assert typeof(h_values) == typeof(d_values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

I see in the dummy array we copied the flags computation from NumPy 1.19, and I wonder if NumPy behaviour has changed in this respect, and we may need to adjust the logic for computing the flags accordingly.

Ah... This logic is making too broad an assumption:
https://github.com/numba/numba/blob/d2af79a790e4c7666231006f314af13beac060a3/numba/misc/dummyarray.py#L180-L182

In the commit message, I summarised the logic differently though:

All 0-size arrays are considered contiguous, even if they are multidimensional.

Looks like I made some odd mistake that went un-noticed all this time.


WIP: Add parent in create-instance

This will solve a few issues. It should make scopedSlots and slots easier to implement
Problems:

Must call $mount to create a nested component instance, which is incompatible with server-test-utils

Fixes #472

@38elements One of the issues with this PR is that it's impossible to add this functionality—https://github.com/vuejs/vue-test-utils/blob/dev/test/specs/mounting-options/slots.spec.js#L107, where the slot is rendered using the context of the component that is mounted.
On reflection, I don't think we should have this behavior, since slots do not behave like this in Vue. What do you think?

Since it is a rare use case and rarely used to write specifications,
I think it is unnecessary to support it.


Export Static Objects

Just like the initial export of all units, all static objects should be exported once.

Late activated or later spawned/created statics must be observed and also added once
Killed statics might either be removed or shown as destroyed

Statics attached to moving objects are not updated. We need to find something to detect this.


Search/filter text is invisible in dark theme - white on white

Environment:

OS: Windows 10
Extension version 1.3.0

Describe the bug:
I use the standard dark theme in VS Code. Running Edge DevTools in VS Code, text is invisible in search and filter boxes. The background is white, and the text color is white. If I select the text, I can see that the text works but is white.

Repro steps:

Set a dark theme for VS Code - I use "Dark+ (default dark)"
Add Edge DevTools, latest version
Launch edge against any web page
In Edge DevTools, type "Ctrl + F" to launch the search box
Type any query - text is invisible

The same problem exists for the "Styles filter" box, and any other text entry with a white background.
Expected behavior:
Entered text is visible.

Thank you for reporting this issue! Sorry that this is happening for you. Looks like this is part of a larger set of issues around applying themes to the extension after a switch. There are lots of components within the devtools, and when switching themes, sometimes they are not properly updated. We have just landed an architecture update that should hopefully address these kinds of issues, but it requires targeting newer versions of Edge.
As a workaround, you can try restarting the devtools and your vscode window a couple of times to hopefully get the theme state synchronized. The search bar should match the dark theme like this:

Additionally, if you would like to try out the new changes that increase our theme support, you can try targeting Edge Canary or Dev (if you have either installed) by updating your extension settings:

Targeting newer Edge versions will allow the extension to most all standard VSCode themes (Dark+, Solarized Dark, etc). These changes will roll out to stable Edge in the coming weeks as well, so hopefully this should resolve soon!

Thank you @bgoddar - the problem is as you described. While I did restart VS Code yesterday, and continued to see the same problem, when I started VS Code today the problem no longer appears. Thank you.
Closing.

Glad that worked for you!


Integrate Stellar CLI docs into Consolidated Developer Docs

Integrate CLI docs into Consolidated Developer Docs, making them searchable and easily accessible
### Tasks
- [ ] Add more examples for each command

Adding examples from CLI, and try to sync directly from README to the mdx files in dev docs


change navbar opacity in yoga template 2

Issue Type:

[ ] Bug Report
[x] Feature Request
[ ] Documentaion

i want to change the opacity so that text doesnt overlap.
Screenshots

/assign
i am a gssoc participant and want to work on this

@wre9-tesh make the PR by today.

ok


Use std::unique_ptr to manage OpenSSL memory allocations when able

Description:
This PR consolidates OpenSSL memory management by utilizing std::unique_ptr with a custom deleter to automatically handle freeing memory when exiting scope.
Related issue(s):
Fixes #173
Checklist

[X] Documented (Code comments, README, etc.)
[X] Tested (unit, integration, etc.)

@rwalworth I like the new implementation, since it hides a lot of the nastiness in just that 1 file.
Did you test this new implementation for memory leaks? It would be neat if that could be done in a unit test, not sure how possible that is

@rwalworth I like the new implementation, since it hides a lot of the nastiness in just that 1 file.
Did you test this new implementation for memory leaks? It would be neat if that could be done in a unit test, not sure how possible that is

Yes I tested for leaks here and we're still good. I don't believe google test has the capabilities to test for memory leaks as part of unit tests unfortunately, but it's pretty simply to run the whole test suite and see the memory leak info with the leaks tool.


cerebro and datastreams

HI
can I get cerebro to list "hidden" indexes produced by data streams?

@rful011 Checking the .special checkbox on the cluster overview should display all hidden indices as well. Did you try that?

I close this. @rful011 feel free to reopen it if you continue having the problem


Out of memory during import

Importing a GTFS file always ends up with
ns=0xFATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
16b2fe47a1e1 <String[14]: gtfs.stoptimes>,ops=0x24d80f184...
Abort trap: 6
I also switched to a GTFS file with only 18MB, same result.

See #45.

should be fixed with:
https://github.com/brendannee/node-gtfs/pull/56

#56 is not a permanent fix. The proper solution is to write node-gtfs to fully use streams in a non-blocking way. Splitting into chunks seems like a workaround.

@derhuerst : can you explain in more detail how using streams would solve the memory overflow issue?
My understanding is that if we use file streams instead of parse this will only reduce the amount of memory consumed by the file contents. However in our tests this did not seem to be what caused the memory overflow. That rather seems to be caused by the callbacks attached to each insert operation.
Thus the solution implemented by @gerlacdt is not actually so much the division in chunks, but using insertMany. The division in chunks is only done because the node js driver internally runs into the same issue if the chunk is too big.

Node streams are basically just event emitters that "hold" data. Streams can be connected to other streams (See .pipe()). One can also read from streams manually (data event). This alone is nothing special when dealing with large amounts of data (larger than memory). Streams have a backpressure mechanism, which keeps the amount of data "held" in memory low. If one uses streams properly (no blocking operations, no non-flowing stream usage]), Node.js scripts can deal with any amount of data. Consider the Stream Handbook for best practices.
In the build script, sync operations are being done. Also, all the data "held" in a stream is read into memory. This is why I think the Node process runs out of memory.

I think @balmoovel  is right. The out-of-memory error was not caused by loading the files completely into memory.
It was caused by the number of callbacks created in a loop.
Pseudo-Algorithm:
loop files  (agency.txt, stops.txt, stop_times.txt)
load file
get csv lines
loop lines
mongodb.insert-callback(line)
For stop_times.txt this opened 2.6 millions callbacks asynchronously! Too many for node....
So switching to "streams" is really nice-to-have and more node-like but will not solve the problem.
By the way inserting millions of entries one-by-one is always a worse solution than insertin in batch-mode (10,000 at once)!

I think @balmoovel is right. The out-of-memory error was not caused by loading the files completely into memory.
It was caused by the number of callbacks created in a loop.

I see this as a problem. For every file, it loads every (that could be loaded from disk) into memory. The data get's transformed and then written into the db. But as long as the communication with the db isn't done, it's still all in memory.
Since the script loads the data synchronously (via while loop), there's no chance for the db layer to actual write stuff to the db while it receives more and more data.

For stop_times.txt this opened 2.6 millions callbacks asynchronously! Too many for node....

AFAIK that's not a problem. (Again, the data that is kept in-memory is a problem.)

So switching to "streams" is really nice-to-have and more node-like but will not solve the problem.

As I said, the script already uses streams, but not properly. Streams would solve the problem since they limit the amount of data being read from the files. Therefore, Node only needs to keep track of a small amount of data & callbacks.

By the way inserting millions of entries one-by-one is always a worse solution than inserting in batch-mode (10,000 at once)!

With very limitied memory (try running the script on a Raspberry PI or VPS), properly implemented streams (working one-by-one) are still more suited than batch operations. But nevertheless, one can combine streams and batch operations. (;

As I have said before

However in our tests this did not seem to be what caused the memory overflow. That rather seems to be caused by the callbacks attached to each insert operation.

We have tested this, and reading the file into memory is not a problem. The problem are the millions callbacks. If you think that either our assumption is wrong or that you can provide a faster and more stable fix yourself then please do.

We have tested this, and to our understanding reading the file into memory is not a problem on systems with gigabytes of memory.

V8 has a default memory limit of 1.5GB. Also, this script should work on small VPS with just 512MB of memory.

If you think that either our assumption is wrong or that you can provide a faster and more stable fix yourself then please do.

I'm not willing to rewrite this build script, but I'd offer help doing so. I can also recommend things like promises and co for making the script more readable.
You may also want to have a look how my Berlin-specific GTFS build script looks like.

@gerlacdt I tested your pull request and it worked.
I'm down to rewrite the download script with streams - its about time it had an overhaul to be more readable.

@brendannee
thx for merging!
Only for information: After the fix we discovered that we had again problems with out-of-memory-error for node-versions < 4.4.x. (Async.queue is really memory-hungry...)
So the best would be, rewriting the script with streams. Although i don't know if this will solve all problems because as far as i understand the csv.parser.on("readable") uses already a stream.
Happy coding!

@brendannee : if you should still run into memory issues with streams have a look at the latest https://github.com/moovel/node-gtfs, it works a bit more stable using recursion.
Good luck and thanks a lot for your work!

For what it's worth, none of these solutions are working for me

#56
https://github.com/moovel/node-gtfs
--optimize_for_size --max_old_space_size=2000

http://www.stm.info/sites/default/files/gtfs/gtfs_stm.zip
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

Hi,
proposed solutions above are not working for me either. I keep getting the same error:
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

@ those who cannot import their data using https://github.com/moovel/node-gtfs : can you post a link to the GTFS data you are trying to import?

Here is the link to the Chicago Transit Agency: http://www.transitchicago.com/downloads/sch_data/google_transit.zip
I found the link here: http://www.gtfs-data-exchange.com/agency/chicago-transit-authority/
Uploaded by cta-archiver on Apr 16 2016
I excluded calendar dates, fare attributes, fare rules, shapes, frequencies, feed_info; and it threw the error when processing stoptimes file.
Here is the link to the Bay Area Rapid Transit: http://www.bart.gov/dev/schedules/google_transit.zip
Found here: http://www.gtfs-data-exchange.com/agency/bay-area-rapid-transit/
Uploaded by  bart-archiver on Apr 04 2016 02:31
I excluded calendar dates, fare attributes, fare rules, shapes, frequencies, feed_info; It threw the error when processing transfers.

I can confirm the issue - I did not find any super quick fix so far, sorry. I promise to look into it when I have some time but that can be weeks.

Here is the gtfs file I was working with
http://www.amt.qc.ca/xdata/trains/google_transit.zip

@brendannee @balmoovel
Fatal error while processing a 3-million line file.
Source: https://api.transport.nsw.gov.au/v1/publictransport/timetables/complete/gtfs
My experience is the same as @nlambert's:

For what it's worth, none of these solutions are working for me
#56
https://github.com/moovel/node-gtfs
--optimize_for_size --max_old_space_size=2000

I just pushed an update that may help handle importing very large GTFS files.
Try out your large GTFS files and let me know what errors, if any, you get.

Thanks@brendannee, it's working great!

Closing this issue - please comment if you still have memory issues.


Wildcard for content file matching not working

Hi there,
I'm using gulp-purgecss 0.12.0 and I have an issue with the following configuration:
gulp.task('purgecss', function() {
  return gulp.src(paths.build.css + 'styles-all.min.css')
    .pipe(purgecss({
      content: [paths.build.html + '*.html']
    }))
    .pipe(gulp.dest(paths.dist.css))
});

What I would expect is that all of my html files in the html build directory are parsed to generate the purged output in the purgecss destination directory. Instead, I only see any root values I've included in my css  (ie. :root { someproperty: somevalue; } )
If I change my gulp task to parse a single html file, the purgecss task works as expected:
gulp.task('purgecss', function() {
  return gulp.src(paths.build.css + 'styles-all.min.css')
    .pipe(purgecss({
      content: [paths.build.html + 'home.html']
    }))
    .pipe(gulp.dest(paths.dist.css))
});

Thank you for reporting this issue.
I will make a fix soon. In the meantime, as a workaround, you should be able to do it with:
const glob = require('glob')

gulp.task('purgecss', function() {
  return gulp.src(paths.build.css + 'styles-all.min.css')
    .pipe(purgecss({
      content: glob.sync(paths.build.html + '*.html')
    }))
    .pipe(gulp.dest(paths.dist.css))
});

It should be fix in v0.13.0.
Let me know if you find any issue

Hi Ffloriel,
I know you have closed this issue but I seem to be having the same problem described by "Occupant". Could you let me know if you see any obvious errors in my code. As described above, if I set a single file in the content section, I get results. An array of individual files also produces results, but the use of the wildcard characters results in an empty stream and ultimate failure. Any guidance will be welcome. Thanks.
gulp.task('purgecss', () => {     return gulp.src('css/build/compiled.css')         .pipe(purgecss({             content: ['../views/**/*']         }))         .pipe(rename('purged.css'))         .pipe(gulp.dest('css/build')) });


get_nist.py fails

It seems that the tools/get_nist.py script fails due to an error message returned by the NIST form. Perhaps the form parameters have been changed again?

@peterhalldestam fyi

My issue was due to a faulty cache. The script on master seems fully functional.


Find a faster way to iterate over the ruleset

Parsing + RegEx is definitely too slow even though it is solid.
Available options:

[ ] ndarray
[ ] per row tld + binary flags

Hey @oncletom,
Is there a reason why the rules.json file only has one level nested? For example, most of the speed problems are due to numerous regexes being run. The largest ruleset is .jp - and is structured like so:
{
	"jp": "asaminami.hiroshima|daiwa.hiroshima|etajima.hiroshima|fuchu.hiroshima|fukuyama.hiroshima|hatsukaichi.hiroshima"
}

Wouldn't it much faster if we .split('.') the domains, and nested the ruleset as far as needed? For example:
{
	jp: {
		hiroshima: {
			asaminami: {},
			daiwa: {},
			etajima: {},
			fuchu: {},
			fukuyama: {},
			hatsukaichi: {}
		}
	}
}```

This would allow for infinite nesting (although I don't think anything would require more than 3 levels) and would require no regexes. It most definitely would be wayyyyy faster. That's the solution that's used in a similar C++ lib: https://github.com/kejinlu/KKDomain/blob/master/KKDomain/Resources/etld.plist

Let me know what you think, I can implement these changes pretty soon.

Hi @olivierbeaulieu,
yeah I think at the beginning I was looking for a compact way of bundling the list for the browser.
But in the end RegExp are not speed efficient and simpler/bitwise operations should be preferred – browser compression will certainly do its job and reduce the filesize for us.
I guess iterating over Object structures would be 1/ faster to parse, 2/ faster to iterate and 3/ easier to maintain.
If you have time I would be happy to see your PR and merge it :-)

Hi @oncletom
If this is still something we are interested in, I'm willing to contribute some patch to implement this more efficient data-structure. I actually did something very similar once in another project (to cache the results of tldjs actually, and I can confirm it was indeed way faster!). If you give me some pointers about where to look I can give it a try!
Thanks,

@oncletom Shall we close this issue now that #97 has been merged?

@remusao totally, good reminder 🙂


Consider adding an option to hide the 🐭  in the status bar

Sometimes the status bar can get too cluttered.
Having an option to hide the 🐭  would be great! Thanks! 😀

Previous mentions of this: https://github.com/sindresorhus/atom-editorconfig/issues/110#issuecomment-255206800 and https://github.com/sindresorhus/atom-editorconfig/issues/110#issuecomment-255209559.

Thanks @battaglr!
@sindresorhus: I think it would be a good idea to introduce a config-setting for this, would you agree (i remember you had strong opinions about that)?

Why have it there in the first place then? Is it really that essential? Couldn't it just be a command in the command-palette?

That's the right question to ask, @sindresorhus. Thanks! 😉

I really like to have the indicator in the status bar, but it could be opt-in.
The command could be an additional feature.

Even in the case where it's not removed from the status bar, why does it appear when there is no active editor?

The reason I want to hide the icon is that this is one of those packages that Just Work™. I don't need it to tell (or show) me what's going on -- just apply whatever config there is. If I get a reason to not trust it I believe we have a bigger problem.

Thank you all for your questions and opinions - i am very grateful for your feedback.
I want editorconfig become a "Just Works"-package. However - it was less then unreliable (Just Don't Work :tm:) in the past and we were confronted with the results in the issue-board (for those few who made it here to leave their feedback).
The status-icon was introduced for two reasons: 1) let the people proof with one look, if the editorconfig is working and 2) use it to get some of the lost trust back by obviously showing the thing is moving forward.
I would like to have editorconfig acting in the background without any need for intervention, unfortunately this is not possible at the moment. There are a some cases where editorconfig may not work as expected. As soon as we have found to fix that i wouldn't show anything from editorconfig at all (at normal operation).
Until this is possible i tend to leave the statusbar-icon for the warning (ecfg may not work) and the error (ecfg is proven to not work) states. I agree that showing the icon in all other cases is well-meant but not necessary.
I know this does not meet all of your positions but allowing you to hide the statusbar-icon and enable it for the users is the only compromise i see to satisfy the different requirements. However we end up here, i will disable the icon for the default-states and introduce an additional editorconfig-command to invoke the notification.

You should also add an ID to the status-bar tile's top-most container element, so users have the option to hide it with CSS. E.g., add a line here like div.id = 'aec-status-bar-tile-wrap'.
I also came here to request the same thing, BTW.

Hey, since there were no further arguments or ideas added, the :mouse: will only show up in failure-states after the upcoming release.
If you want to check if EditorConfig is working invoke EditorConfig: Show State.
Thanks everybody for your contribution. 💝
P.s.: @Alhadis the id (aec-status-bar-container) is added.

Ace, thanks!


Can't Deploy Application, 'Houston, there's a problem. Network Failure'

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
Can't Deploy
Region
eu-central-1
Amplify Hosting feature
No response
Describe the bug
Trying to host a Next.js app from amplify hosting, after clicking through all the menu options (connect Github, build settings, environment variables, etc.) when I reach the final screen and click 'Save and Deploy' it stalls for a second, and then throws Houston, there's a problem. Network failure.
Someone else on my team is able to host on AWS Amplify, and I can make edits, but when it comes to hosting a new application myself, it almost always throws this error. Only once or twice have I been able to make a new application, but due to the complexities of working around an Nx repository, I've had to delete applications and try again to tweak with app-root and whatnot.
Expected behavior
Amplify should attempt to host my application, and at least reach the build step.
Reproduction steps

new app
host web app
connect git repository
configure branch, select app root, etc.
click 'save and deploy'

Build Settings
# not sure if this is applicable, however I'm using the yml specified in a workaround related to nx
# with my app name, ofc
version: 1
applications:
  - frontend:
      phases:
        preBuild:
          commands:
            - npm install
        build:
          commands:
            - npm run build:prod
      artifacts:
        baseDirectory: ../../dist/apps/mono-repro/.next
        files:
          - '**/*'
      cache:
        paths:
            - node_modules/**/*
    appRoot: apps/mono-repro

Additional information
No response

Hi @reeslabree 👋🏽 apologies for the oversight on this issue. If you run into this Houston, there's a problem. message, please contact AWS Support for further assistance.


Managing NIfTI intent fields

Comes from https://github.com/bids-standard/bids-bep016/pull/5#discussion_r320910555

To clarify: there is a nii-compliant way to write out DTI-specific derivatives. But many software libraries don't implement anything to deal with this compliant way of doing things, instead just doing... whatever.
Do we want to support the nii-compliant way of doing things? As the only way of doing things? Or one of a couple of options? This might intersect with #34 in the sense that this could be the role of some software that takes output of different processing pipelines and puts them in that format.

My personal preference would be to not interpret the NIfTI intent fields at all. BIDS metadata via JSON should completely supersede such things, as its capabilities will be a superset. Relevance moreso to #23 is that if a piece of software is known to read / write those intend fields, then any conversion to/from BIDS Derivatives should handle such appropriately.

Makes sense. Closing.


Kindly Submission - Next Steps

Respond to clarifications

https://docs.google.com/document/d/1Zcm69aYwUATJua3kZvJCLskW4EAOP5Ant601FXSjvlc/edit
GDPR Compliance


Add surfed extension

Description

Screencast

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

@pernielsentikaer thank you for your review. I have added more details and included a screenshot with the API-Key entry in the app settings.

@pernielsentikaer README > Configuration section should be better now. The API-Key is only available if Use Password is enabled. Otherwise, leave the API-Key setting blank.

It seems like there is no -l option
22:45:58.879 Failed to fetch data: Error: Command failed: /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/MacOS/surfed-cli dac -j -a -l 50
objc[10881]: Class _TtC6SQLite6Backup is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2668) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435d6c8). One of the two will be used. Which one is undefined.
objc[10881]: Class _TtC6SQLite10Connection is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee23e8) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435d7f8). One of the two will be used. Which one is undefined.
objc[10881]: Class _TtC6SQLite9Statement is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2510) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435d968). One of the two will be used. Which one is undefined.
objc[10881]: Class _TtC6SQLite13DateFunctions is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee25d0) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435ef78). One of the two will be used. Which one is undefined.
objc[10881]: Class _TtC6SQLite12TableBuilder is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2278) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435f328). One of the two will be used. Which one is undefined.
Error: Unknown option '-l'
Usage: surfed-cli [--collections] [--tags] [--all] [--json] [--password <password>] [--key <key>] <search> ...
  See 'surfed-cli --help' for more information.

    at ChildProcess.exithandler (node:child_process:422:12)
    at ChildProcess.emit (node:events:514:28)
    at ChildProcess.emit (node:domain:488:12)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
  code: 64,
  killed: false,
  signal: null,
  cmd: '/Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/MacOS/surfed-cli dac -j -a -l 50 ',
  stdout: '',
  stderr: 'objc[10881]: Class _TtC6SQLite6Backup is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2668) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435d6c8). One of the two will be used. Which one is undefined.\n' +
    'objc[10881]: Class _TtC6SQLite10Connection is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee23e8) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435d7f8). One of the two will be used. Which one is undefined.\n' +
    'objc[10881]: Class _TtC6SQLite9Statement is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2510) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435d968). One of the two will be used. Which one is undefined.\n' +
    'objc[10881]: Class _TtC6SQLite13DateFunctions is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee25d0) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435ef78). One of the two will be used. Which one is undefined.\n' +
    'objc[10881]: Class _TtC6SQLite12TableBuilder is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2278) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x10435f328). One of the two will be used. Which one is undefined.\n' +
    "Error: Unknown option '-l'\n" +
    'Usage: surfed-cli [--collections] [--tags] [--all] [--json] [--password <password>] [--key <key>] <search> ...\n' +
    "  See 'surfed-cli --help' for more information.\n"
}
22:45:59.164 Failed to fetch data: Error: Command failed: /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/MacOS/surfed-cli dace -j -a -l 50
objc[10882]: Class _TtC6SQLite6Backup is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2668) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ff96c8). One of the two will be used. Which one is undefined.
objc[10882]: Class _TtC6SQLite10Connection is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee23e8) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ff97f8). One of the two will be used. Which one is undefined.
objc[10882]: Class _TtC6SQLite9Statement is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2510) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ff9968). One of the two will be used. Which one is undefined.
objc[10882]: Class _TtC6SQLite13DateFunctions is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee25d0) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ffaf78). One of the two will be used. Which one is undefined.
objc[10882]: Class _TtC6SQLite12TableBuilder is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2278) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ffb328). One of the two will be used. Which one is undefined.
Error: Unknown option '-l'
Usage: surfed-cli [--collections] [--tags] [--all] [--json] [--password <password>] [--key <key>] <search> ...
  See 'surfed-cli --help' for more information.

    at ChildProcess.exithandler (node:child_process:422:12)
    at ChildProcess.emit (node:events:514:28)
    at ChildProcess.emit (node:domain:488:12)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
  code: 64,
  killed: false,
  signal: null,
  cmd: '/Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/MacOS/surfed-cli dace -j -a -l 50 ',
  stdout: '',
  stderr: 'objc[10882]: Class _TtC6SQLite6Backup is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2668) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ff96c8). One of the two will be used. Which one is undefined.\n' +
    'objc[10882]: Class _TtC6SQLite10Connection is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee23e8) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ff97f8). One of the two will be used. Which one is undefined.\n' +
    'objc[10882]: Class _TtC6SQLite9Statement is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2510) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ff9968). One of the two will be used. Which one is undefined.\n' +
    'objc[10882]: Class _TtC6SQLite13DateFunctions is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee25d0) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ffaf78). One of the two will be used. Which one is undefined.\n' +
    'objc[10882]: Class _TtC6SQLite12TableBuilder is implemented in both /System/Library/PrivateFrameworks/LinkServices.framework/Versions/A/LinkServices (0x280ee2278) and /Applications/Surfed.app/Contents/MacOS/surfed-cli.app/Contents/Frameworks/SharedFramework.framework/Versions/A/SharedFramework (0x105ffb328). One of the two will be used. Which one is undefined.\n' +
    "Error: Unknown option '-l'\n" +
    'Usage: surfed-cli [--collections] [--tags] [--all] [--json] [--password <password>] [--key <key>] <search> ...\n' +
    "  See 'surfed-cli --help' for more information.\n"

I'm currently on macOS Sequoia, but I guess it should work there too?

Thanks a lot for your feedback. It looks like there is a problem with the CLI in the current version that requires an app update. I can add you to TestFlight to give you access to the upcoming version. Is it possible to approve an extension and release it later? Otherwise, I'll have to ship the new app update before the Raycast extension release.

We can draft this PR until you're ready, but sure, feel free to add me to TestFlight 🙂 It's just<EMAIL_ADDRESS>😂

Thanks for the invite. I just tested it again with the version from TestFlight and it seems to retrieve data now, but I have two questions for you:

Is there a way to show the latest 50 items in the list when entering the extension?

If not, you should consider a List.EmptyView to explain what to do

Great to hear that it is now working. Have fun with Surfed! I have implemented a List.EmptyView for various scenarios.

I did not found any kind of tokens like UISearchToken. It would be great to have an equivalent API for that in a future Raycast version.

@pernielsentikaer hi, any news from the code review?

@hosy are you still around?

@pernielsentikaer the new app version is currently under review by Apple. I hope it will be approved within the next two days. I'll ping you as soon as it's approved. Thanks!

@pernielsentikaer I would like to release Surfed 3 today at 11 AM (GMT+2). Is it possible to merge the PR today?

Should be fine, drop a ping here when it's ready 🙂

@pernielsentikaer yes, it is available now. Thank you!

I have the 3.0 version from TestFlight and I'm getting this error when I search. Do you know what that might be?
10:55:09.726 Failed to fetch data: SyntaxError: Unexpected token 'S', "StorageMan"... is not valid JSON
    at JSON.parse (<anonymous>)
    at fetchURLs (/Users/pernielsentikaer/.config/raycast/extensions/surfed/index.js:69:31)

@pernielsentikaer Argh! There was a regression in the submitted build. Thanks for finding this bug. I have submitted a new build, version 3.0.1, for review.

That's better in the TestFlight version. Has this version been released yet?

The version is now available in the App Store. Thank you for your support and quick response. Best regards!


Remove peerDeps

As of npm3+, peerDeps will no longer be installed automatically. Lets remove the peerDep grunt and keep the devDep.

peerDependencies are still useful for warning if the plugin is used with an incompatible grunt version.


Microsoft documentation page not visible

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4fd30f7f-e5ce-67d5-2536-ee1d115a573d
Version Independent ID: 6979c76f-48f2-1d2a-1358-ab1b6998bfd3
Content: Quickstart: Get token & call Microsoft Graph in a console app - Microsoft identity platform
Content Source: docs/identity-platform/quickstart-v2-netcore-daemon.md
Service: active-directory
Sub-service: develop
GitHub Login: @OwenRichards1
Microsoft Alias: owenrichards

@stephenosr
Could you please elaborate the query for better understanding. So, we can assist you further.

@stephenosr
Since we have not heard back from you, we will now proceed to close this thread.
If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


When import pipeline and the alerting is not set, the import does'nt work

Describe the bug
When we import a pipeline without alerting, the import does'nt work
Expected behavior
The import should be successful even if the alerting is not set

Trello card: [#139] When import pipeline and the alerting is not set, the import does'nt work


Add Oauth2 support

This will be required in 2022-10 or so.
https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-and-exchange-online-september-2021-update/ba-p/2772210

Maybe this would work best if there were a web interface.


Roll engine daf4447a2eb2..f95bad8496c1 (1 commits)

https://github.com/flutter/engine/compare/daf4447a2eb2...f95bad8496c1
git log daf4447a2eb2..f95bad8496c1 --no-merges --oneline
f95bad849 Roll src/third_party/skia a631669e50a2..e7588323d586 (8 commits) (flutter/engine#6509)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

@rmistry @liyuqian : New roll has been created by https://autoroll.skia.org/r/flutter-engine-flutter-autoroll

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/6080138309533696

Commit queue failed; closing this roll.


We can't convert Smoothie Board to Circuit JSON

Smoothie Board: https://github.com/freerouting/freerouting/blob/master/tests/Issue145-smoothieboard.dsn
Upload to DSN Viewer: https://dsn.tscircuit.com/
This is probably a pretty hard multi-step issue. Make sure to download freerouting to make sure the snapshot resembles the DSN file.
In addition to the bounty, we will likely tip for steps along the way because doing it in one big PR may be counter-productive.
/bounty $70

@seveibar How to solve this error i have tried this reference sharp fails when installing with bun but nothing worked out.

@Anshgrover23 what version of bun are you using? Have you tried upgrading?

➜  ~ bun -v
1.1.38
@seveibar i have latest version of bun

/attempt #54

@seveibar when this issue is complete is there a final goal to reach?

/claim #54

/attempt #54


Urgent Feature: Allow manual extractor selection/override

Having the ability to manually tell youtube-dl what extractor to use while providing a link that may not be the URL the extractor expects.
For example:
youtube-dl --use-extractor 'youtube' -g http://www.blahblahblah.com/some_captcha_workaround.php?v=xxxxxxxxxxx
...would cause youtube-dl to treat the link as a youtube video url.
I need this feature to try to work on a captcha workaround by processing everything outside of youtube-dl and return the captcha-less video page's source code (it would be identical to the source that youtube-dl sees when there is no captcha).
Another cool upgraded or advanced use of this feature would be the ability feed/pipe the pure source code into youtube-dl after captcha processing or bypass.

We would have to modify a lot the regexes to make it work and it wouldn't work because we end up downloading the page youtube.com/watch?v=<id> for all the input urls.
Note that since the source code of youtube-dl is freely available, you can download it and modify the youtube extractor for testing it.
See the reasons in #4551 to know why passing the webpage source as input is not possible (among others it won't work for playlists and for youtube videos we need at least two pages).


More verbose error in VCSFromDir to indentify problem package

I recently found it quite difficult to figure out why godep save was failing, because the error message was a bit too generic:
godep: directory $GOPATH is not using any known version control system
godep: error loading dependencies

After adding some print statements and rebuilding godep, I found out that one of my libraries in my $GOPATH had been installed as a pure source download, instead of a valid repo. The problem is that this error is bubbled up from go.tools/go/vcs/vcs.go, which tries to walk up the paths to find a repo. When it returns this error message that I see, it contains a path that has already been walked up.
I've added an extended error message to VCSFromDir() to ensure that the problem package is printed and easily identifiable to the user.

bump

How about fmt.Errorf("error inspecting %q: %v", dir, err) ?

Sure. I am happy to change it, if it gets it merged

Thanks!


Translations update from Weblate

Translations update from Weblate for Kodi misc/Kore.
Current translation status:

@SyncedSynapse Is it ok to merge?


SB T2- LiveTest - Fix ReceiveTwoMessageAndComplete

Fix :  ReceiveTwoMessageAndComplete
If there is gap between .acceptSession() and we start receiving,  the session lock expire before we call first receiveMessage().
We have configured our script to expire lock in 15 seconds

/azp run java - servicebus - tests


change the default flush to cosole - tmp file

Change the default implementation.
Save the log in the tmp dir.

Right now, the only issue is that the code has not been linted, so just run black . after accepting my edits and it should be fine. 😄

Thanks a lot @sarthakpati


Added function to compare chunks

This allows for two chunks with two different pointers to still be compared for equality.
(pls merge so I don't have to update oomph's fork 🙏)

👍

bump

bump

rebump

bumpy

can we merge this? I'd like to be able to get rid of my dragonfly fork I'm currently using

@ethaniccc :V


command内调用rpc请求报错

error:  Uncaught Error: Call to a member function getTcpSetting() on null in /code/test/vendor/swoft/rpc/src/Packer/EofTrait.php:19

https://github.com/swoft-cloud/swoft-component/pull/155


ValueError: cannot convert float NaN to integer

I face below issue while running it on sample bin data.
"/fuzzy_code/cpp_fuzzy.py", line 22, in cpp_cls
res=round(fun(x0,y0))+1
ValueError: cannot convert float NaN to integer
"

Should these changes be ok (left side is default code, right side is updated code).

Also do you have libfuzzy.so built for windows?

Additionally,

nms iou thresholding is applied classwise after fuzzy classification in "class_agnostic_nms" function. In this case, there is possibility of same object being detected with multiple classes.
if fun(x0,y0) returns nan, should we assign some garbage class value  to it ?


Content Planning Q2 2022

Create a Roadmap, with Milestones to Map to the Github Project
Create a noProject board to spec, scope, and track individual projects

Acceptance Criteria
We have a page where there is a roadmap with the 'epic-level' milestones identified and given time spans
We have a project board created, with the higest priority items from protocol/launchpad Github issues (and projects by Gilecoin?) created with multiple views. One view is a table filtered by milestone, another is a table filetered by DRI/ assignee, and the other is a Kanban board ranked by priority

Done


[QUESTIOLN]: Are homekit connected cameras kept after restart?

I'm curious on how the homekit cameras work, if a user adds the camera is that connection kept after go2rtc is restarted? Does it not require any changes to the config.yaml?

Camera can be imported from Hass. So Hass storage remember HomeKit credentials.
Camera can be paired to go2rtc. Credentials will be stored in go2rtc.json file in cwd. That should be /config folder in case of Docker container. Or Hass config folder for Hass users.

I think this should be added to the readme for Docker users that not using Hass.

Also credentials could be converted to source link and used in yaml. They always will be same after pairing. But I think this is way only for geeks and developers. IP-address and port will be automatically updated on each connection via mDNS.
homekit://<IP_ADDRESS>:12345?client_id=xxx&client_private=xxx&device_id=xxx&device_public=xxx

Also credentials could be converted to source link and used in yaml. They always will be same after pairing. But I think this is way only for geeks and developers. IP-address and port will be automatically updated on each connection via mDNS.
homekit://<IP_ADDRESS>:12345?client_id=xxx&client_private=xxx&device_id=xxx&device_public=xxx

Agreed, thinking about the frigate case of supporting homekit video cameras we would either need the user to map go2rtc/config/ to their host so the config is saved, or we would need to save that URL for future use. Will be a while until that is considered / added to frigate though.


Clarify expected behaviour when connecting to Vagrant boxes - interaction required?

Describe the bug
When used from the command line, Hashicorp's vagrant is capable of ssh-ing to a newly created box without any password or user interaction being required to make the connection:
$ vagrant ssh -- echo "hello world"
hello world

But trying to do the same with pyinfra results in a No host key for [<IP_ADDRESS>]:2200 found in known_hosts, do you want to continue [y/n] prompt being shown. Is this the expected behaviour? That would seem to make pyinfra strictly less capable than vagrant.
To Reproduce
Tested on Ubuntu 20.04, using Python 3.8.10 and pyinfra 2.3 (installed with pip install pyinfra).
1. Create a Vagrantfile:

Vagrantfile contents
Vagrant.configure("2") do |config|
    config.vm.box = "generic/alpine310"

    config.vm.provider "virtualbox" do |vb|
      vb.memory = "300"
    end
end

2. Bring up the Vagrant box
$ vagrant up

3. Confirm prompt-less ssh works using vagrant ssh
$ vagrant ssh -- echo "hello world"
hello world

4. Try the same with pyinfra:
$ pyinfra --debug @vagrant exec -- echo "hello world"

Expected behavior
The command echo "hello world" should execute on the Vagrant box without any need for user interaction.
Actual behaviour
The user is required to accept a new host key:
$ pyinfra  @vagrant exec -- echo "hello world"
--> Loading config...
--> Loading inventory...
    Getting Vagrant config...

--> Connecting to hosts...
No host key for [<IP_ADDRESS>]:2200 found in known_hosts, do you want to continue [y/n]

Comments
Running pyinfra with --debug suggests it should be using the SSH options "-o UserKnownHostsFile=/dev/null" and "-o StrictHostKeyChecking=no", but for some reason it isn't doing so.
$ pyinfra --debug  @vagrant exec -- echo "hello world"
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    Getting Vagrant config...
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Loading SSH config for default
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Got Vagrant SSH info:
['Host default', 'HostName <IP_ADDRESS>', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']
    [pyinfra_cli.inventory] Checking possible group_data directory: /mnt/data/dev/vouch-demo
    [pyinfra.connectors.vagrant] Got Vagrant SSH info:
['Host default', 'HostName <IP_ADDRESS>', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']

--> Connecting to hosts...
    [pyinfra.connectors.ssh] Connecting to: @vagrant/default ({'allow_agent': False, 'look_for_keys': False, 'hostname': '<IP_ADDRESS>', '_pyinfra_ssh_forward_agent': None, '_pyinfra_ssh_config_file': None, '_pyinfra_ssh_known_hosts_file': None, '_pyinfra_ssh_strict_host_key_checking': None, '_pyinfra_ssh_paramiko_connect_kwargs': None, 'username': 'vagrant', 'port': 2200, 'timeout': 10, 'pkey': <paramiko.rsakey.RSAKey object at 0x7ffa365e0d00>})
    [pyinfra.connectors.sshuserclient.client] Loading SSH config: None
No host key for [<IP_ADDRESS>]:2200 found in known_hosts, do you want to continue [y/n] ^CExiting upon user request!

Meta
output of pyinfra --support
--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Linux
      Platform: Linux-5.13.0-48-generic-x86_64-with-glibc2.29
      Release: 5.13.0-48-generic
      Machine: x86_64
    pyinfra: v2.3
    Executable: /mnt/data/dev/vouch-demo//env/bin/pyinfra
    Python: 3.8.10 (CPython, GCC 9.4.0)

How was pyinfra installed (source/pip)?
Using pip install pyinfra.
output with -vv and --debug
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    Getting Vagrant config...
localhost: >>> vagrant status --machine-readable
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Loading SSH config for default
localhost: >>> vagrant ssh-config default
[pyinfra.connectors.util] --> Waiting for exit status...
[pyinfra.connectors.util] --> Command exit status: 0
    [pyinfra.connectors.vagrant] Got Vagrant SSH info:
['Host default', 'HostName <IP_ADDRESS>', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']
    [pyinfra_cli.inventory] Checking possible group_data directory: /mnt/data/dev/vouch-demo
    [pyinfra.connectors.vagrant] Got Vagrant SSH info:
['Host default', 'HostName <IP_ADDRESS>', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /mnt/data/dev/vouch-demo/.vagrant/machines/default/virtualbox/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']

--> Connecting to hosts...
    [pyinfra.connectors.ssh] Connecting to: @vagrant/default ({'allow_agent': False, 'look_for_keys': False, 'hostname': '<IP_ADDRESS>', '_pyinfra_ssh_forward_agent': None, '_pyinfra_ssh_config_file': None, '_pyinfra_ssh_known_hosts_file': None, '_pyinfra_ssh_strict_host_key_checking': None, '_pyinfra_ssh_paramiko_connect_kwargs': None, 'username': 'vagrant', 'port': 2200, 'timeout': 10, 'pkey': <paramiko.rsakey.RSAKey object at 0x7f49ed67a8b0>})
    [pyinfra.connectors.sshuserclient.client] Loading SSH config: None
No host key for [<IP_ADDRESS>]:2200 found in known_hosts, do you want to continue [y/n]

Some additional info on this bug/question, in case it's of use to anyone else encountering this issue.
Workaround
A workaround is to explicitly invoke the pyinfra command with the command-line arguments pyinfra --data ssh_known_hosts_file=/dev/null --data ssh_strict_host_key_checking=no.
(But pyinfra's behaviour is still odd, since those same command-line arguments are effectively contained in the output pyinfra gets from vagrant ssh-config, in pyinfra.connectors.vagrant._get_vagrant_ssh_config().)

Cause and possible fix (with patch)
When running against a Vagrant box, pyinfra has all the information it needs to make an ssh connection during execution of the function pyinfra.connectors.vagrant.make_names_data():
https://github.com/Fizzadar/pyinfra/blob/0ee6b9f509e9fdc335a23f3de27e5fa7a458a61c/pyinfra/connectors/vagrant.py#L132
_get_vagrant_ssh_config() fetches the ssh config for a Vagrant box by running vagrant ssh-config, and the config is returned as a list of lines, something like
['Host default', 'HostName <IP_ADDRESS>',
'User vagrant', 'Port 2222',
'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no',
'PasswordAuthentication no',
'IdentityFile /path/to/project_dir/.vagrant/machines/default/virtualbox/private_key',
'IdentitiesOnly yes', 'LogLevel FATAL', ''
]

For each host, all of these keys and values are put into a dictionary and passed to pyinfra.connectors.vagrant._make_name_data(), which "translates" from the key names SSH uses (like "Port", "User", and "IdentityFile") to the ones pyinfra uses (like "ssh_port", "ssh_user", and "ssh_key").
But the two SSH keys relevant to this bug (UserKnownHostsFile = /dev/null, and StrictHostKeyChecking = no) never get so translated. So by the time code in the SSH connector gets executed (e.g. pyinfra.connectors.ssh._make_paramiko_kwargs), it tries to connect with whatever SSH's default values are. (~/.ssh/known_hosts and yes, I believe.)
The fix appears to be to add those two keys to the list of keys that get translated - see the patch here: https://github.com/Fizzadar/pyinfra/compare/2.x...phlummox-dev:pyinfra:fix-vagrant-connector
However, there are currently no end-to-end Vagrant tests for testing this. It should be possible to write some – GitHub's MacOS 10.15 virtual environment includes Vagrant and VirtualBox – but I'm not really familiar enough with pyinfra's codebase to create them.

Thank you for the detailed write up @phlummox! Indeed this is a regression with the recent changes to make SSH host keys behave like OpenSSH. Would you mind PR-ing that fix and I'll get that merged in?
Added #875 to setup e2e vagrant tests 👍


Remove ACS Variables from 2010 that don't match metadata file

Addresses #243
This reverts commit ef50b5bbdda619bdf0be5884f1b5f981bb753021.
When we removed the base variable join from the script, we removed any respect for the metadata file for 2010, and just piped through the input excel file. The excel file has columns for variables not in the metadata file, causing issues for OSE. Adds an inner join on the metadata file so we can only return records with a matching metadata

This looks good but honestly would like another set of eyes @td928 @Oysters1874 @AmandaDoyle

I think the key thing that makes me a little nervous is just the dependence on the metadata file having every relevant field. But from what I remember about our conversations last time around, that's expected to be the case.

That makes me nervous as well but this whole data update was pretty wonky

The fact that OSE relies on the metadata files in the functioning of their app should provide some confidence there - they made a separate ticket because one base variable was missing for a pff variable - this would be the case for any variables included that don't have a matching metadata record


Rename data to properties in _Object and DataObject

This PR renames the data attribute within _Object and DataObject for clarity on its role in relation to the created Collection configurations. Since these map directly to the properties within the Class schema, naming it properties seems more appropriate.

@dirkkul, this will merge into introduce_typed_dict_generic_to_collection so I won't merge it until that PR is also approved. Then they will cascade into pydantic_experiment 😁

Codecov Report
Patch coverage: 93.75% and no project coverage change.

Comparison is base (d80203d) 93.80% compared to head (999606f) 93.80%.

Additional details and impacted files
@@                             Coverage Diff                             @@
##           introduce_typed_dict_generic_to_collection     #454   +/-   ##
===========================================================================
  Coverage                                       93.80%   93.80%
===========================================================================
  Files                                              84       84
  Lines                                           10839    10839
===========================================================================
  Hits                                            10168    10168
  Misses                                            671      671

Files Changed
Coverage Δ

weaviate/collection/data.py
73.92% <ø> (ø)

integration/test_collection_model.py
84.58% <76.19%> (ø)

integration/test_collection.py
99.81% <100.00%> (ø)

integration/test_collection_filter.py
87.03% <100.00%> (ø)

weaviate/collection/classes/data.py
100.00% <100.00%> (ø)

weaviate/collection/classes/internal.py
83.33% <100.00%> (ø)

weaviate/collection/grpc.py
85.23% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

can't we just merge it into pydantic_experiment?

It was tied up with the generics implementation but I've pulled that out too now


how to call aFreeRDP in another Android APP to connect to a remote App ?

I am working in an android app that get the hostname, user name and password, as well as remote program name, and use intent mechanism to invoke session activity in aFreeRDP.
however, I can't find any tech document on how to finish this invocation .
can any one help on this ?

@cloudfirst there is no real documentation on the Android client at the moment. Have a look to FreeRDP/client/Android/FreeRDPCore to see how the intent(s).
If you have further questions visit our irc channel (#freerdp on freenode) or write a mail to the mailing list but please don't add questions in the issue tracker.


Game Request: Looney Tunes Acme Arsenal

Game Name?
Looney Tunes: Acme Arsenal
Game ID?
RLYEWR
Upload the Textures dumps for the game.
arsenal.zip
I Dont' Know How Well Dynamic Input Textures fares with Motion Control Textures, bu hey, guess it'll be a first time experience!

My bad, missed a texture

My bad, missed a texture
lol.zip

I Dont' Know How Well Dynamic Input Textures fares with Motion Control Textures, bu hey, guess it'll be a first time experience!

currently rather bad, however the new version of Dynamic Input Textures will come soon.
however, it still looks manageable.

Yeah i Can imagine, the wiimote shaking texture should be easy enough, just set it to the X part of the shake control, but as for spinning the nunchuck? I'm totally lost on how to implement THAT.

Looney Tunes Acme Arsenal.zip

I will not add this graphic for right now.

This is associated with nunschuk shake X

the other textures use some kind of mask, most of the time there is an associated texture, if not i can't do anything about it.

Alright then if that's the case, that's all folks! (Get it, b-because this is a looney Tunes Game? eh whatever)

couldn't you find any mask textures?
then I assume the game uses models for the buttons.

No, sadly, that's just how they show up in game, just a white texture, what you see with the wii and nunchuck texture is what you get, thankfully they don't show up that much in game, only the tutorial


[Fix] Making TpchBenchmark tool work with S3 and VELOX_ENABLE_S3 flag.

This PR will add support for S3 file system to the TpchBenchmark code and CMakeList.txt files so that the benchmark can be used to test against S3 file system as well as the default local file system.

@majetideepak The TpchBenchmark tool no longer functions against S3. Do you know why? I looks like everything needed was merged except the registration of the S3 filesystem but adding that does not make it use S3, it instead uses the local file system.

@kgpai, @mapleFU, @pedroerp , @assignUser, @akashsha1, @Yuhta: This fixed the TpchBenchmark to correctly understand S3 filenames and to make S3 support conditional (VELOX_ENABLE_S3) when building the tool.

Could you also improve the PR description to reflect what's the exact change introduce by this PR? Thanks!

@paul-amonson a cleaner solution is to push if USE_S3FS into the S3FileSystem API so that we do not have to add these for each and every use. We did that for Parquet. See https://github.com/facebookincubator/velox/blob/main/velox/dwio/parquet/RegisterParquetReader.cpp

I am not sure how to do this. Do I move the registerS3FileSystem() method to a different source file? Currently it is in file velox/connectors/hive/storage_adapters/s3fs/S3FileSystem.cpp. This file is not compiled and would force the TpchBenchmark to require S3 support.

I am not sure how to do this. Do I move the registerS3FileSystem() method to a different source file? Currently it is in file velox/connectors/hive/storage_adapters/s3fs/S3FileSystem.cpp. This file is not compiled unless VELOX_ENABLE_S3 is defined and would force the TpchBenchmark to require S3 support.

@majetideepak I tried moving this to the FileSystem.h/.cpp files but I get link errors in other benchmark tools missing the S3 connector lib. If you have a specific solution please share. I am not sure how to satisfy this request. The other option is to always include the s3fs connector but I think @assignUser said this was not desirable.

I am not sure how to satisfy this request.

@paul-amonson I am working on a PR to improve the registration of HiveConnector related adapters (DWRF, Parquet, FileSystems). I should open one by EOD and that will fix the S3FileSystem registration.

@paul-amonson As you noticed, here is the PR for the CMake S3FileSystem changes.
https://github.com/facebookincubator/velox/pull/5469

Looks good to me, but I'll leave @majetideepak give the final stamp since he has more context on this. I can merge it then.


Opening a .img with many frames is slow

Try sprite_character_gunner_equipment_avatar_pants.NPK, switching between gn_pants0002a.img and gn_pants0002b.img. It takes half a second to a full second even in release builds.

Fixed by enabling virtualizing on the listboxes.


Kafka batch of messages too large

Is your feature request related to a problem? Please describe.
We get "KafkaJSProtocolError: The request included a message larger than the max message size the server will accept" when a batch of messages is too large to send in one go. The individual messages are small enough to be sent to Kafka.
Describe the solution you'd like
The ideal interface would be to send messages in smaller batches no matter the size size of the batch. This allows users to choose the optimal produce size instead of having to choose a suboptimal solution (like a fixed batch size based on experiments, or worse).
An implementation could request those max sizes from Kafka before trying to send, and then chop the batches down in smaller chunks if necessary. In our workaround solution, we catch the error, calculate an approximation of the size (using JSON.stringify), and then retry automatically with smaller chunks. The size that works, is then stored so that future batches don't need to go through the retry mechanism. This workaround is implemented on top of the sendBatch call, so it is completely optional.
Note that this naïve approach makes it impossible for a client to figure out which messages have been sent and which not, as opposed to the current implementation where you know nothing has been sent. One case that could be troublesome is where one of the messages is even too large to send by itself: part of the batch could be sent without a problem (everything up to this message), but this message will not make it through. It's pretty easy to first loop through the messages and throw an error if a single message is too large, though. But that requires the implementation to know the max size upfront.
Additional context
Other implementations like the client for Go and Java do automatic batching behind the scenes. The Go client uses a channel, the Java client returns a Future and continues processing asynchronously.
Automatic batching would also be nice for the javascript client. I had a look at the implementation and saw that it first groups the messages by topic and partition (which happen to be the same for the whole batch in our use case), so implementing it at that level seems preferred to my proposal above.

Async produce has a lot of challenges, what should we do if you decide to terminate the service and the background queue is full? What happens if the broker is updated accept a smaller size while we already have messages in the queue? etc. We chose to yield this responsibility back to userland, that's why you can await for producer send and be sure that by the time it resolves the messages are in Kafka.
But I like the idea of fetching the message size and validating the batch, we could do this together with the metadata refresh calls to keep the value up to date. Maybe you are lacking the necessary "tools" in the library to implement a good async producer. I am not against a built-in background producer, is just that it comes with more decisions and corner cases, but I would love to see a proposal and discuss over that.

Thanks for the quick reply!
I guess it's easy to do async in javascript with a bunch of promises and then awaiting them. But that wouldn't take advantage of the batching: each message would be sent in a separate communication towards the right kafka broker, instead of them being bundled.
I feel like you could hide the batching in a way that just sends the first message (from the first send call), and while it's still sending, queue up the messages that are being sent in the mean time. When that previous message (or batch) has finished, it can send the currently accumulated batch of messages. That would hide the complexity to the user. Of course there are still race conditions to consider when the size of a single message becomes too large due to updated settings.
I don't think that changing those size settings to a smaller value would ever be a good idea though. I think I read something about that in the Kafka docs.
I'm curious how other implementations work when those size settings are updated in flight. Interesting.

A related question with this, is there a way to actually get the maximum message size with kafkajs? I am actually trying to implement what jagij suggested and split my message into batches with a limit size of X bytes based on kafka message size configuration.

relatedly, is there a way to set max.request.size on the producer?

relatedly, is there a way to set max.request.size on the producer?

Has a solution been found yet?

relatedly, is there a way to set max.request.size on the producer?

i´m currently looking for an answer for this also

We faced the same issue. One of the key issue is the overhead that's added by the library.
I believe, please correct me if I am wrong. That the library adds a bunch of things in the headers like the clientId and the correlationId.
The limit on the kafka side includes the headers, key, and value message itself. We have been counting the bytes we are sending but we were still hitting the limit. The way we went around that is to experiment with an arbitrary byte overhead value. While that works, it is not very efficient and brittle. For example if we change the client id, I suspect that will change the headers, which will change the message size which may make it that we go over the limit again.
I see 2 solutions:

Either the lib handles the batching fully but then needs to respect a parameter or the kafka value for the maximum batch size. Handling the signal termination as suggested above.
Or it needs not to add anything to the messages and the user needs to set the correlation ids and all. Then the responsibility is on the user to build the batches of the correct size.
Or the lib provides an experience with some kind of Batch Builder object. ie that could be derived from the producer and its metadata there. something like:

const batch = producer.createBatch(topicName);

// Add a record to the batch
batch.add(message)

// check the byte size of the batch
batch.size()

I suspect this is probably the easiest to implement and the most user friendly.
If the way it works is 2 I have an issue in my code but since I don't set a cliendId in the headers, I suspect this is handled by the library.


Refactor flow control rate limit stream message processing to avoid branching

https://github.com/risingwavelabs/risingwave/pull/13057#discussion_r1372516657

Branching has minimal cost, and the refactored logic is more complex to maintain. Not planned.


Remove spaces in phone property

I checked FlutterContacts documentation, i don't see another way for delete "formatting" space.

Hm, as i remember we agreed normalization it on ui, not in datasource. We can't close it because we will have bug which sent customer.

The actual removal of spaces in the phone number occurs prior to initiating the call - c22b7e05e2ddc4814dcbb84b7a7f3cf4b1b978b1.


Manual Controller UI

We need to implement a manual controller for robots in the UI. This should consist of buttons to turn left and right and drive forward and backwards.
We could also potentially find a d-pad react component to use.

Ticket is now in draft until the it is updated to fit the task request format, or is finished