While building for tvOS Simulator, no library for this platform was found in

I got error "While building for tvOS Simulator, no library for this platform was found in '/Users/egeorgiev/Library/Developer/Xcode/DerivedData/Stremio-ehyfhtijmvvkhgacvdsezhjrdgzz/SourcePackages/artifacts/stremio-core-swift/XCFramework/StremioCore.xcframework'."
It's new tvOS project and added the library via the Package Dependencies.

Fixed! Update version to 1.2.62 and it should work for tvos-sim both x86_64 and arm64 platforms. Let me know it runs successfully

Fixed! Update version to 1.2.62 and it should work for tvos-sim both x86_64 and arm64 platforms. Let me know it runs successfully

Now I don't have any issues when I build the application but have issues when try to use it the app crashes (not just an error)
thread '' panicked at src/commonMain/rust/stremio_core_apple.rs:171:10:
RUNTIME not initialized
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
thread '' panicked at /Users/alvin/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/panicking.rs:221:5:
panic in a function that cannot unwind
It seems that it's looking for your path for rust.
I'm trying:
StremioCore.StremioApi.Login(email<EMAIL_ADDRESS>password: "adasdas") { result in                 switch result {                 case .success(let userAuthenticated):                     // Handle successful authentication                     print(userAuthenticated)                 case .failure(let error):                     // Handle login failure                     print(error)                 }             }

Could you enable RUST_BACKTRACE using setenv command? This would point me where problem is

Could you enable RUST_BACKTRACE using setenv command? This would point me where problem is

My user is not alvin. Somehow it wants to use your path not mine. It's hardcoded somewhere.

Because it's compiled in my device it shows my path. Thats how rust works even on release apparently. So please share full path

This issue caused by using Core without initializing. Please call Core.initialize()  before executing any type of API. Otherwise it will crash

This issue caused by using Core without initializing. Please call Core.initialize()  before executing any type of API. Otherwise it will crash

Could you send me some contact information, so I could reach you ?

Could i ask what topic you want to discuss?

Could i ask what topic you want to discuss?

@AlvinHV - It's not so clear how to use the library at all. I needed some information and wanted to have a quick chat with you about that and to improve the README file but never mind. I gave up on this because it's so unreasonable with all those middle layer things.  I cant event map the response of some request to the real object. I know why Stremio makes it in this way but it's not applicable for Swift well enough or something is not converted in a right way. No clear documentation how the library is included.  Adding @Ivshti  to see my feedback.

Stremio-core-swift is a fork of stremio-core-kotlin and doesn’t include stremioApi.kt in its library. It takes time for me to analyze and clarify its functionality. Additionally, having some background knowledge about Stremio’s internal APIs is necessary to use it effectively, which is beyond the scope of stremio-core-swift. Regarding mapping responses to objects, I haven’t encountered issues with the implemented APIs so far. Here is a preview of my work before joining Stremio.
Understanding and completing the implementation will require more time, so it might be best to wait for the official release. (If @Ivshti could give you access to private repo fell free to experiment with stremio-apple)

Stremio-core-swift is a fork of stremio-core-kotlin and doesn’t include stremioApi.kt in its library. It takes time for me to analyze and clarify its functionality. Additionally, having some background knowledge about Stremio’s internal APIs is necessary to use it effectively, which is beyond the scope of stremio-core-swift. Regarding mapping responses to objects, I haven’t encountered issues with the implemented APIs so far. Here is a preview of my work before joining Stremio.
Understanding and completing the implementation will require more time, so it might be best to wait for the official release. (If @Ivshti could give you access to private repo fell free to experiment with stremio-apple)

So you will create an apple tv app at some point. Is that the case ?


empty container field

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
The container field seems unavailable. Is this a bug?
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp gr51aVj-mLg --print after_move:"container: %(container)s" --no-config --no-simulate -v

[debug] Command-line config: ['gr51aVj-mLg', '--print', 'after_move:container: %(container)s', '--no-config', '--no-simulate', '-v']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[9874e82b5] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.1
[debug] Proxy map: {}
[debug] Loaded 1797 extractors
[youtube] Extracting URL: gr51aVj-mLg
[youtube] gr51aVj-mLg: Downloading webpage
[youtube] gr51aVj-mLg: Downloading android player API JSON
[youtube] gr51aVj-mLg: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] gr51aVj-mLg: Downloading 1 format(s): 18
[debug] Invoking http downloader on "https://rr5---sn-h0jeener.googlevideo.com/videoplayback?expire=1681652400&ei=UKY7ZPGkC4LW1wLY8YnYBw&ip=<IP_ADDRESS>&id=o-AINmB6lh8MHEy_ylclaVGeZEI-xwOCcEjna2ncbn7xDH&itag=18&source=youtube&requiressl=yes&mh=-9&mm=31%2C29&mn=sn-h0jeener%2Csn-h0jeln7l&ms=au%2Crdu&mv=m&mvi=5&pl=16&initcwndbps=2547500&spc=99c5CRCTA-Q3srUuh63RMD7ZGgBgHzY&vprv=1&svpuc=1&mime=video%2Fmp4&cnr=14&ratebypass=yes&dur=10.100&lmt=1636675987407441&mt=1681630389&fvip=2&fexp=24007246&c=ANDROID&txp=1218224&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Ccnr%2Cratebypass%2Cdur%2Clmt&sig=AOq0QJ8wRQIgMG-hN2dzDV0pTa1aJ6A6NsfjvuzJk7cWVEcORzZxXnECIQC-9wF8AlkYYlMfESx0wFsxbk9bj4mzZ38gGjL3keOr2w%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgC2ziAIR3XgpGoQPkUsJIpM01UJbk9wFVP33arHKORSQCIBd6IaVdVhR1G76rhqxaeoIN7ClFHIS2T30hHetdLvMM"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: Youtube-dl Annotation Test [gr51aVj-mLg].mp4
[download] Download completed
container: NA

No, because the format doesn't have a different container. What did you expect it to give?

The only example I know off the top of my head is m4a_dash. You can see the container in -F (in more info) if present.

I would have expected it to always have a value.

id, url, ext are the only fields guaranteed to have a value.

19 specific extractor modules mention 'container', though sometimes this may be a page string rather than the format metadata item, or both, as in Brightcove.

The main use of the field afaik is only to detect m4a_dash apart from other m4a so we know whether to apply fixup or not. Idk why it's even exposed to user tbh.

Indeed: https://github.com/ytdl-org/youtube-dl/commit/1394ce65b4
But the PR/issue linked from there is obviously not right.


Fix NextJs14 recommendations

This looks good, did you test that the runtime: edge still works properly?


Help with Azure Functions

Hey, thanks for the client.
I am trying to use it with azure functions, but it is not finding the client. If I enter with ssh and run pip install and prisma generate, it works, but I am not being able to do so. Is there any way to handle this issue?
Thanks!

Closing due to inactivity.
Please reopen if you have any more questions.


Consider capping uploads at a specific number of devices per IP address

This would make it much harder for people to abuse the service as a file-store.

It might be better to require something like 'Sign in with Apple' for this–it's easy for users and it helps us guarantee that anyone uploading is a genuine user.


Buck build broken for c#

Report

[x] I have searched existing issues and this is not a duplicate

Issues and Steps to Reproduce
Clone repo and run buck build of csharp targets or build-native.sh
Expected Behavior
Should run fine
Actual Behavior
Fails
Link to Code
If you have some code that maintainers can clone/test for themselves, bugs can be resolved much faster. Please paste a link here.
When applicable, use this fiddle to post a web repro.

I'm working on it.

I had to revert 3 commits that touched .buckconfig and buck file .

I think is still failing for Android? you need to check the changes of a580712b2f6833070ed4323b54a26805332eb43d

which target? buck build //csharp:yoganet#android-armv7,shared is fine for me.

iRuiXam-MBP:csharp rmarinho$ buck build //csharp:yoganet#android-x86,shared
Not using buckd because watchman isn't installed.
[-] PROCESSING BUCK FILES...FINISHED 0.0s [100%] 🐳  New buck daemon
[+] DOWNLOADING... (0.00 B/S, TOTAL: 0.00 B, 0 Artifacts)
[+] BUILDING...0.8s
BUILD FAILED: Unrecognized flavor in target //csharp:yoganet#android-x86,shared while parsing //csharp/BUCK
Here are some things you can try to get the following flavors to work::
android-x86 : Make sure you have the Android SDK/NDK installed and set up. See https://buckbuild.com/setup/install.html#locate-android-sdk

if i revert that commit it works.

hmm, I can't repro that. Do you have .buckconfig.local?
$ git rev-parse HEAD
2199a7908d49f4e1e81c32acf4407bbf9e4952ad
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean

$ rm -rf buck-out
$ buck build //csharp:yoganet#android-x86,shared
[-] PROCESSING BUCK FILES...FINISHED 0.1s [100%]
[-] DOWNLOADING... (0.00 B/S AVG, TOTAL: 0.00 B, 0 Artifacts)
[-] BUILDING...FINISHED 5.6s [100%] (10/10 JOBS, 10 UPDATED, 5 [50.0%] CACHE MISS)


Plugin crashes when multiple devices are configured with same IP

Plugin crashes when multiple devices are configured with same IP
Of course this is a silly config but it shouldn't crash the plugin

Fixed in v1.0.7 (2023-11-19)


JSON DB in Docker image does not appear to work properly

The docker image runs on SQLite beautifully, but when enabling JSON DB in Docker (MICROBIN_JSON_DB: "true"), the docker image fails to start, returning the following errors in the docker log:
2023-09-10T18:37:34 [ERROR] - Failed to create database file pasta_data/database.json: No such file or directory (os error 2)!
thread 'main' panicked at 'Failed to create database file pasta_data/database.json: No such file or directory (os error 2)!', src/util/db_json.rs:38:21
stack backtrace:
0:     0x55a0fcafe1ae -
1:     0x55a0fc8346ff -
2:     0x55a0fcad6c84 -
3:     0x55a0fcaffa0f -
4:     0x55a0fcaff60e -
5:     0x55a0fcb00680 -
6:     0x55a0fcb00164 -
7:     0x55a0fcb000c6 -
8:     0x55a0fcb000b1 -
9:     0x55a0fc70df42 -
10:     0x55a0fc76416a -
11:     0x55a0fc7644bf -
12:     0x55a0fc94e314 -
13:     0x55a0fc941435 -
14:     0x55a0fc93a743 -
15:     0x55a0fc93f24b -
16:     0x7f5743c9b18a -
17:     0x7f5743c9b245 - __libc_start_main
18:     0x55a0fc7268fa -
19:                0x0 -
I have tried to create the "pasta_data" directory manually in the mapped docker folder, but that does not appear to work.
Love the application. Keep up the good work!

I fixed it with that config option:
volumes:
  - ./microbin/data:/app/microbin_data
  - ./microbin/pasta:/app/pasta_data

But yeah, @szabodanika move it to microbin_data to avoid confusion


SincNet vs. STFT

Hi,
We are doing STT / TTS for the Russian language.
We mostly used STFT due to our ignorance in DSP and our understanding that MFCC filters used by everyone may be a bit over-engineered (I have seen no papers actually comparing them properly vs. STFT and similar features for morphologically rich languages).
So, my question is as follows. I understand, that your filters contain a magnitude less parameters than ordinary CNN layers, and in essence are just DSP inspired frequency filters.
In our experience we tried a lot of separable convolutions, and we mostly agree with this paper (i.e. the success of mobile networks shows that the convolutions are overrated, but the mix layer + shift layer can do the job).
Here STFT is imlemented as a convolution (they inherit their kernels from numpy, but I guess they are similar in essence to triangual filters from MFCC). So, my questions are:

Have you tried using STFT, but actually training the filters?
Have you tried using separable convolutions?

Best,
Alex

Hi Alex,
thank you very much for your questions. As you have pointed out in SincNet we are not using STFT, but we are doing something related to it. In fact, you can see SincNet as a way to learn the STFT itself, where instead of using "fixed fft points" we use learnable channels. If you reorder the channels according to the central frequency you obtain a representation similar to the STFT (not that in SincNet we are not taking the magnitude of the STFT, but we have the full representation that includes the phase contribution).  You can for sure learn some filters on the top of the STFT (as done in some papers cited in my work), but in SincNet we decided to avoid the step of pre-computing the STFT and starts from the lowest possible representation (i.e, the simple waveform).  We didn't tried separable convolutions, but could be worth trying.

I guess during some time in future I will report 3 experiments:

baseline stft
stft, but filters can be learned
sincnet

Would be cool to compare

@snakers4 I would be very interested to see your results. Particularly recently I was thinking how it would be possible to adapt SincNet to apply sinc operation on spectrogram instead of directly on waveform. This would be a similar to melspectrogram function of pytorch (https://pytorch.org/audio/transforms.html#melspectrogram) but the filterbanks will change depending on tunable params.
My current main issue is scaling SincNet on larger inputs (15 seconds of waveform). It takes way too much memory even to get to the max pool layer. I asked for suggestions in PyTorch forum recently (still waiting for a reply): https://discuss.pytorch.org/t/attempting-to-reduce-memory-consumption-by-fusing-conv1d-with-maxpool1d/61448

Hi,
last days I worked a bit on something related to that. I have modified the
filter-bank implementation that you find within torch.audio to make the
central frequency and the band of the filters learnable. The difference
with SincNet are the following:
1- we operate on the spectrogram rather than directly on the raw speech
samples
2- We compute the weighted average after processing the spectrum with the
triangular filters (as done for standard FBANKs).
From my very preliminary experiments, it emerges that the model is
extremely compact and better than standard FBANKs, but SincNet still
performs better.
One reason could be that with SincNet we are not performing the average,
and we thus have a  much higher temporal resolution. Also, SincNet doesn't
discard the phase. This could be helpful, but I don't think this can
explain the performance difference.
The implementation will be available in the next months under the
speechbrain project (https://speechbrain.github.io/).
Best,
Mirco
On Tue, 19 Nov 2019 at 04:31, Dilshod Tadjibaev<EMAIL_ADDRESS>wrote:

@snakers4 https://github.com/snakers4 I would be very interested to see
your results. Particularly recently I was thinking how it would be possible
to adapt SincNet to apply sinc operation on spectrogram instead of directly
on waveform. This would be a similar to melspectrogram function of pytorch (
https://pytorch.org/audio/transforms.html#melspectrogram) but the
filterbanks will change depending on tunable params.
My current main issue is scaling SincNet on larger inputs (15 seconds of
waveform). It takes way too much memory even to get to the max pool layer.
I asked for suggestions in PyTorch forum recently (still waiting for a
reply):
https://discuss.pytorch.org/t/attempting-to-reduce-memory-consumption-by-fusing-conv1d-with-maxpool1d/61448
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/74?email_source=notifications&email_token=AEA2ZVWAVVCTJWJTZ7ZIZL3QUOXABA5CNFSM4JN2G3N2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEENPKLI#issuecomment-555414829,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEA2ZVVGSQNMZGASOI275HDQUOXABANCNFSM4JN2G3NQ
.

Hi,
If I may, I can suggest you to read this paper [disclaimer : I wrote it] which compares learnable, parametrized (such as the base layer of SincNet) and fixed filterbanks (such as STFT) for speech separation.
The code for the three filterbanks is available in asteroid.filterbanks[disclaimer : I wrote it] . The SincNet filterbank is also extended to be analytic.

Have you tried using STFT, but actually training the filters?

This is exactly what the FreeFB and AnalyticFreeFB are doing in the code I linked to.

My current main issue is scaling SincNet on larger inputs (15 seconds of waveform). It takes way too much memory even to get to the max pool layer.

This is because the stride is very small, I can suggest using analytic filterbanks (from which shift invariant representation can be computed) with larger stride to solve this problem.

@mpariente, thank you for you comment and links. Do you have any examples how one can convert waveform speech signal to features? I was trying to check out your latest project code but could not figure out how to use AnalyticFreeFB. Is the code complete?

@antimora Yes the code for the filterbanks is complete but I should write how to use it somewhere, sorry.
import torch
from asteroid.filterbanks import AnalyticFreeFB, Encoder
encoder = Encoder(AnalyticFreeFB(n_filters=512,
                                 kernel_size=256,
                                 stride=128))
waveform = torch.randn(8, 1, 32000)
features = encoder(waveform)  # (batch, 512, time)

I'll soon work on the docs but don't hesitate to open an issue there.

We are finishing the current set of experiments, and dedicating a couple of weeks to run the comparison of the "first convolutions" seems alluring because I did not do it yet at all =)
Now we are just using this STFT implementation (it works on CPU faster than librosa for some reason, also to make it easier for our models to work with TTS voices, because you could just train Tacotron to output STFT and omit the vocoder for now =) ) with these params:
@attr.s(kw_only=True)
class AudioConfig(object):
    window_size = attr.ib(default=0.02)
    window_stride = attr.ib(default=0.01)
    sample_rate = attr.ib(default=16000)
    ....

self.n_fft = int(audio_config.sample_rate * (audio_config.window_size + 1e-8))
self.hop_length = int(audio_config.sample_rate * (audio_config.window_stride + 1e-8))

self.stft = STFT(self.n_fft,  # 320
                         self.hop_length,  # 160
                         self.n_fft)  # 320

This produces 161-sized feature maps.
@mpariente
Many thanks for your code and paper.
I will try to understand the deeper logic behind it.
Meanwhile a couple of basic dumb questions to do some quick and dirty experiments.

but I should write how to use it somewhere, sorry.

Or I guess I can also use this constructor you provided here, right?
Also, I just pass params like n_filters / kernel_size / stride as kwargs, right?

your filters contain a magnitude less parameters
This is because the stride is very small, I can suggest using analytic filterbanks (from which shift invariant representation can be computed) with larger stride to solve this problem.

Could you please give a very rough and dirty memory and compute comparison of the filter-banks available in your repo?
I.e. how to make this table correct?
| params | compute | memory
-- | -- | -- | --
FreeFB / Conv1D | 1 | 1 | 1
STFTFB | 1 | 1 | 1
AnalyticFreeFB | ? | ? | ?
ParamSincFB | 0.1 | ? | N
Also  maybe  could you provide some guidance on the default params I should start with for each filter-bank given our params above?

This is because the stride is very small, I can suggest using analytic filterbanks (from which shift invariant representation can be computed) with larger stride to solve this problem.

Are you referring to the stride used in SincNet by default?

Have you tried using STFT, but actually training the filters?
This is exactly what the FreeFB and AnalyticFreeFB are doing in the code I linked to.

Judging from the code, FreeFB  is just Conv1D and STFTFB is just Conv1D  initialized with STFT.
But what is AnalyticFreeFB?

@mpariente
Also forgot to ask.
If I did not miss anything in your code, even if I use STFTFB, it will be trainable?
So to do a proper ablation test, I need to freeze it separately, right?

Now we are just using this STFT implementation (it works on CPU faster than librosa for some reason, also to make it easier for our models to work with TTS voices, because you could just train Tacotron to output STFT and omit the vocoder for now =) ) with these params:
@attr.s(kw_only=True)
class AudioConfig(object):
    window_size = attr.ib(default=0.02)
    window_stride = attr.ib(default=0.01)
    sample_rate = attr.ib(default=16000)
    ....

self.n_fft = int(audio_config.sample_rate * (audio_config.window_size + 1e-8))
self.hop_length = int(audio_config.sample_rate * (audio_config.window_stride + 1e-8))

self.stft = STFT(self.n_fft,  # 320
                         self.hop_length,  # 160
                         self.n_fft)  # 320

This produces 161-sized feature maps.

The STFT implementation in asteroid is roughly equivalent. Few points where it's different :

The forward and backward basis are the same (this saves a little bit of space).
The output of the encoding is the concatenation of the real and imaginary parts instead of the magnitude and the phase. That's just a choice and these are equivalent.
The main difference is the following : I never divide by the window's envelope. I should do some proper experiments on that and write it somewhere because dividing by the window's envelope is done everywhere. Even if it enables unit-test level perfect reconstruction, in my experience, it hurts performance for speech enhancement and source separation applications by introducing amplitude modulation artifacts to the resynthesis.
np.sqrt(hanning) is used by default, it has better reconstruction that hanning (if we don't divide by the window's envelope).
I truncate the STFT filters instead of padding them so that I don't convolve with a lot of zeros in case of over-complete STFT (it should be a bit faster).

Overall, it is very similar.

If I did not miss anything in your code, even if I use STFTFB, it will be trainable?
So to do a proper ablation test, I need to freeze it separately, right?

It is not trainable, register_buffer is used, not register_parameter. A buffer is just an object which will be saved in the state_dict and be transferred to CPU or GPU along with the model but it doesn't appear in model.parameters() and is not updated by the optimzer.

@mpariente
Many thanks for your code and paper.
I will try to understand the deeper logic behind it.
Meanwhile a couple of basic dumb questions to do some quick and dirty experiments.

but I should write how to use it somewhere, sorry.

Or I guess I can also use this constructor you provided here, right?
Also, I just pass params like n_filters / kernel_size / stride as kwargs, right?

Yes exactly, make_enc_dec will provide an encoder and a decoder. To be equivalent to your piece of code, you can use:
self.stft = make_enc_dec('stft', n_filters=self.n_fft,
                         kernel_size=self.n_fft,
                         stride=self.hop_length)[0]

This will produce 322 features (the output shape is in self.stft.n_feats_out) and you can use this to take the magnitude or self.stft.post_process_inputs if you specified inp_mode='mag' in the encoder. And it will produce the same 161-dim features.

Also maybe could you provide some guidance on the default params I should start with for each filter-bank given our params above?

You can use exactly the same parameters for all filterbanks and it makes sense, that's the nice thing about it. So you can have a fair comparison between these filterbanks.

This is because the stride is very small, I can suggest using analytic filterbanks (from which shift invariant representation can be computed) with larger stride to solve this problem.

Are you referring to the stride used in SincNet by default?

Yes.

Have you tried using STFT, but actually training the filters?
This is exactly what the FreeFB and AnalyticFreeFB are doing in the code I linked to.

Judging from the code, FreeFB is just Conv1D and STFTFB is just Conv1D initialized with STFT.
But what is AnalyticFreeFB?

FreeFB will be the regular Conv1D without bias, that's right.
The STFTFB is also a Conv1D but the filters are the one from the STFT (they are fixed, not learnable)
AnalyticFreeFB is a learned analytic filterbank. I also ends up being an Conv1D but with analycity constraints.
ParamSincFB is the analytic extension of the original SincNet filters, it also is a Conv1D.

Could you please give a very rough and dirty memory and compute comparison of the filter-banks available in your repo?

Regarding the table, if the same values are used for n_filters, kernel_size and stride, the compute and memory are the same for all, only the number of parameter per filter changes.

params/filters

FreeFB / Conv1D
kernel_size

STFTFB
0

AnalyticFreeFB
kernel_size / 2

ParamSincFB
1

It is not trainable, register_buffer is used, not register_parameter. A buffer is just an object which will be saved in the state_dict and be transferred to CPU or GPU along with the model but it doesn't appear in model.parameters() and is not updated by the optimzer.

Following this logic, if I take the nvidia implementation or your implementation and substitute register_buffer with register_parameter I should get the desired effect, right?

STFTFB | 0

This is zero because of register_buffer?

This will produce 322 features (the output shape is in self.stft.n_feats_out) and you can use this to take the magnitude or self.stft.post_process_inputs if you specified inp_mode='mag' in the encoder. And it will produce the same 161-dim features.

The best practical models that we have have a constant width of 512, so it hardly would make a difference in terms of complexity.
But for some reason, in all implementations of STT that I saw the phases were always discarded. I wonder why?

Following this logic, if I take the nvidia implementation or your implementation and substitute register_buffer with register_parameter, i.e.:
        if is_trainable:
            self._filters = nn.Parameter(filters)
        else:
            self.register_buffer('_filters', filters)

Correct.

STFTFB | 0

This is zero because of register_buffer?

Correct. Note that the nvidia implementation will have weird behavior in this case for over-complete STFT (i.e. n_filters > kernel_size) because the filters are padded with a lot of zeros, and these zeros will become parameters so your effective kernel_size will be much bigger in this case. If you always use kernel_size = n_filters, it will be fine.

This will produce 322 features (the output shape is in self.stft.n_feats_out) and you can use this to take the magnitude or self.stft.post_process_inputs if you specified inp_mode='mag' in the encoder. And it will produce the same 161-dim features.

The best practical models that we have have a constant width of 512, so it hardly would make a difference in terms of complexity.
But for some reason, in all implementations of STT that I saw the phases were always discarded. I wonder why?

Because the phase has some non-local properties, it is invariant by global and 2pi rotation which are difficult to model with DNN. The paper I linked to shows that modern DNN can benefit from using the complex representation (depending on the size of the window) because the modelling capabilities are higher now.

Many thanks for your explanations.
After we complete the current batch of experiments, I guess will add the following experiments to the queue:

(2 more long experiments now in the queue)
(current baseline) nvidia stft (CPU)
(new) STFTFB fixed (CPU or GPU)
(new) STFTFB learnable (GPU)
(new) FreeFB learnable (GPU)
(new) AnalyticFreeFB learnable (GPU),
(new) ParamSincFB learnable (GPU)
Also for the best option I will try to fiddle with the number of channels - what if, for example, with SincNet we could afford 5x filters (161 * 4-5 also looks kind of similar to the networks' hidden size)?

Each experiment is best when you run it at least for ~24 hours, so it will take between 1 and 2 weeks to test all this. Also it will be very interesting to see how transferring the first convolution from CPU to GPU would affect speed and io - now my networks are a bit slower than my io, but I guess there will be some trade-offs here.

@snakers4, if you want I can also share with you the tunable filter-banks that I'm designing for the SpeechBrain project. They are extremely compact and these days I'm doing several tests to check their performance.

@mravanelli
Yeah, why not, since I will be doing a grid-search anyway
A friend of mine probably will also contribute something wavelet related @pollytur

Sure,
I would be great for me if you can share the results of your grid search
when you have something.
please contact me privately (mirco.ravanelli at gmail.com) for the fbank
code!
On Mon, 25 Nov 2019 at 10:21, Alexander Veysov<EMAIL_ADDRESS>wrote:

@mravanelli https://github.com/mravanelli
Yeah, why not, since I will be doing a grid-search anyway
A friend of mine probably will also contribute something wavelet related
@pollytur https://github.com/pollytur
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/74?email_source=notifications&email_token=AEA2ZVTIRU3KTVBDJHS3O2TQVPUPHA5CNFSM4JN2G3N2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFCX6VA#issuecomment-558202708,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEA2ZVTMRDX6A3IXAAOFCCTQVPUPHANCNFSM4JN2G3NQ
.

@mpariente
Just to confirm that I am using you fbanks correctly
from filterbanks import make_enc_dec
from filterbanks.pytorch_stft import STFT  ## nvidia stft

# in the init of the pytorch model
        if self._fbank_type != 'none':
            n_fft = int(sample_rate * (window_size + 1e-8))
            hop_length = int(sample_rate * (window_stride + 1e-8))
            fb_kwargs = {'inp_mode': 'mag',
                         'n_filters': n_fft,
                         'kernel_size': n_fft,
                         'stride': hop_length}
            if self._fbank_type == 'nvidia_stft_fixed':
                self.fbank = STFT(n_fft,
                                  hop_length,
                                  n_fft,
                                  is_trainable=False)
            elif self._fbank_type == 'nvidia_stft_trainable':
                self.fbank = STFT(n_fft,
                                  hop_length,
                                  n_fft,
                                  is_trainable=True)
            elif self._fbank_type == 'stftfb_fixed':
                self.fbank, _ = make_enc_dec('stft',
                                             **{fb_kwargs,
                                                {'is_trainable': False}})
            elif self._fbank_type == 'stftfb_trainable':
                self.fbank, _ = make_enc_dec('stft',
                                             **{fb_kwargs,
                                                {'is_trainable': True}})
            elif self._fbank_type in ['free', 'analytic_free', 'param_sinc']:
                self.fbank, _ = make_enc_dec('free',
                                             **fb_kwargs)
            else:
                raise NotImplementedError()

# in forward
        if self._fbank_type != 'none':
            if self._fbank_type in ['nvidia_stft_fixed', 'nvidia_stft_trainable']:
                x, phases = self.stft.transform(x)
            elif self._fbank_type in ['stftfb_fixed', 'stftfb_trainable',
                                      'free', 'analytic_free', 'param_sinc']:
                x = self.fbank.post_process_inputs(self.fbank(x))
            else:
                raise NotImplementedError()

Also I have 2 types of augmentations for spectrograms:
(0) An augmentation mimicking phone calls, where I just discard 50% of my frequencies;
(1) Something similar to SpecAugment (I just cut out random bands of frequencies / strips of time);
Do I understand correctly, that I can use (1) with any type of your fbanks, but I can use (0) only with STFT?

Most of it looks fine, the last filterbank definition should be
    elif self._fbank_type in ['free', 'analytic_free', 'param_sinc']:
        self.fbank, _ = make_enc_dec(self._fbank_type,
                                     **fb_kwargs)

to take into account the filterbank type right?
For the data augmentation, (1) does make sense and (0) doesn't, in the sense that discarding half the filters just amounts to dividing the number of filters by two but all the frequencies will still be covered (also holds for the STFT).

to take into account the filterbank type right?

correct, just a typo

discarding half the filters just amounts to dividing the number of filters by two but all the frequencies will still be covered (also holds for the STFT).

Hm, correct me here, but if I always discard the "upper" half, i.e. stft[81:] = 0, isn't it equivalent to actually changing the effective sampling rate from 16k (most STT applications) to 8k (phone calls)?
This was the idea behind this

Yes, you're right, I meant the learnable STFT, not the fixed one sorry. I edit the answer above to correct for this.

I have completed my tests and started the learnable frontend experiments. It is very early to draw any formiddable conclusions yet, but:

Nvidia STFT fixed+ GPU. Looks like that running STFT on GPUs provides some speed boost in my case, but I need to verify by running longer (at least ~24 hours per experiment). Looks like that I am bordering on being IO bound now, whereas in the past I was bound by my network (which is good)
Nvidia STFT trainable + GPU. Looks like when I start training my network from scratch with trainable pre-initialized STFT from nvidia (I just made these changes) with all of the other params being equal, the network explodes after a couple of batches

        self.is_trainable = is_trainable
        if self.is_trainable:
            self.forward_basis = torch.nn.Parameter(forward_basis.float())
            self.inverse_basis = torch.nn.Parameter(inverse_basis.float())
        else:
            self.register_buffer('forward_basis', forward_basis.float())
            self.register_buffer('inverse_basis', inverse_basis.float())

...

        if self.is_trainable:
            forward_transform = F.conv1d(
                input_data,
                self.forward_basis,
                stride=self.hop_length,
                padding=0)
        else:   # looks like this was written for PyTorch 0.4
            forward_transform = F.conv1d(
                input_data,
                Variable(self.forward_basis, requires_grad=False),
                stride=self.hop_length,
                padding=0)

fbank cuda:1 55.69 0.0 0.23 1.14
after fbank cuda:1 13.99 -5.26 0.00 5.74
fbank cuda: 0 55. 0.0 0.23 1.16
after fbank cuda:0 13.90 -5.30 -0.00 5.71

after one batch update
GPU-0 Epoch 1 [1/18082] Time 5.22 (5.22)        Data 2.75 (2.75)        Loss 37.69 (37.69)

fbank cuda:0 inf inf inf nan
after fbank cuda:0 nan nan nan nan
fbank cuda:1 inf inf inf nan
after fbank cuda:1 nan nan nan nan
WARNING: Working around NaNs in data

I have tried some things that have helped me in the past, so far without success:

lowering LR for the fbank

not applying the gradient clipping

decreasing the gradient clipping norm

STFT FB fixed+ GPU - seems to work as well

STFT FB trainable + GPU. Looks like when I use @mpariente 's fbanks, but I set STFT to be trainable, it does not explode / misbehave outright, I need more time to confirm. Also if I do not setup the fbank LR to be 10-100x lower than the main network LR, it also explodes. Also sometimes this happens

@mpariente
I think I will start by just comparing my current pipeline vs nvidia stft on gpu vs your stft on gpu
Also there is a small issue (?) with your implementation
I am running a multi-GPU setup with DataParallel / DistributedDataParallel
And unless I apply this hack to your code, it fails to use the buffers properly (weights are not on GPU, but tensors are)
I guess it may a problem for sinc net
class _EncDec(SubModule):
    """ Base private class for Encoder and Decoder.
    def get_filters(self):
        """ Returns filters or pinv filters depending on `is_pinv` attribute """
        if self.is_pinv:
            # return self.filters
            return self.compute_filter_pinv(self.filters)  # I changed this line
        else:
            return self.filterbank._filters

@snakers4 Thanks for the update !
Did you use any pseudo-inverse in your code?
I think it is related to this issue which was fixed. The code is a bit different. Could you try with the current version and tell me if there is still a problem please?

Did you use any pseudo-inverse in your code?

No, we are just doing STT, we just need spectrograms

I think it is related to this issue which was fixed.

Many thanks for the pointer, I will use this fix when running SincNet
First results
As for the first results, looks like running plain STFT on CPU / GPU does not make a real difference, while tuning the STFT poses the challenges noted above
I will tune for a bit more to confirm, then I guess I will try stft_fb / free filterbank / sincnet (unless they also explode) in this order
Val set 1
(actually 3 different val sets mashed together)

Val set 2

I could run the above experiment for ~5-10 additional hours, but it did not change much
Since I am using very random sampling for curriculums, the charts above are hardly different
(when the models are different is obvious even after 20-30 hours of training)
Also tried a few other things

free_fb
param_sinc

With / without this normalization (which works well with STFT)
class AdaptiveAudioNormalization(nn.Module):
    def __init__(self, sigma=20, truncate=4.0):
        super(AdaptiveAudioNormalization, self).__init__()

        filter_ = self.get_gaus_filter1d(sigma, truncate)
        self.register_buffer('filter_', filter_)
        self.reflect = torch.nn.ReflectionPad1d(sigma * int(truncate))

    def forward(self, spect):
        spect = torch.log1p(spect * 1048576)
        mean = spect.mean(dim=1, keepdim=True)
        mean = self.reflect(mean)
        mean = F.conv1d(mean, self.filter_)
        mean_mean = mean.mean(dim=-1, keepdim=True)
        spect = spect.add(-mean_mean)
        return spect

    @staticmethod
    def get_gaus_filter1d(sigma, truncate=4.0):
        sd = float(sigma)
        lw = int(truncate * sd + 0.5)
        sigma2 = sigma * sigma
        x = np.arange(-lw, lw+1)
        phi_x = np.exp(-0.5 / sigma2 * x ** 2)
        phi_x = phi_x / phi_x.sum()
        return torch.FloatTensor(phi_x.reshape(1, 1, -1))

With varying LR / optimizer

SGD with LR 1e-3 / 1e-4 / 1e-5
ADAM with LR 1e-4 / 1e-5

For any combinations, the network exploded either outright or after several hundred batches
I mostly did not change the other pipeline, so maybe the current pipeline is over-optimized for STFT and I need to start some simpler experiments
I have no other direction other than to try all of these experiments on some very eazy dataset (e.g. some small validation subset)

Thanks a lot for the detailed experiments and results.
I wonder, did you check what caused the models to explode? Do you get NaN losses or simply garbage outputs. If NaN losses, does it start with a NaN loss or a NaN gradient?
For the analytic free filterbank, when I worked at 16kHz with noisy inputs, I had to catch NaN gradients for it to work, and it ended up converging fine.
Also, for the free filterbank, it doesn't really make sense to take the magnitude, as it is not complex.. But I'm not sure it's going to change anything.

Do you get NaN losses or simply garbage outputs. If NaN losses, does it start with a NaN loss or a NaN gradient?

The symptom obviously is NaN losses, but I had no time to check what exactly is causing it.
For some setups (tunable stft) I checked - the first convolution itself exploded
fbank cuda:0 inf inf inf nan (max min mean std)
after fbank cuda:0 nan nan nan nan  (max min mean std)
WARNING: Working around NaNs in data (working around NaNs in losses)

For the analytic free filterbank, when I worked at 16kHz with noisy inputs, I had to catch NaN gradients for it to work, and it ended up converging fine.

There is gradient clipping in my pipeline, I set it to some high values (100-200), otherwise the network converges slowly (I arrived at this value after grid-searching). I work around NaNs in losses, but not explicitly in gradients.
Could you maybe share a snippet on how you do it for the gradients? Many thanks!

Also, for the free filterbank, it doesn't really make sense to take the magnitude, as it is not complex.. But I'm not sure it's going to change anything.

Yeah, basically we are taking 1/2 of the convolution, but I belive the root cause the cause is elsewhere

with noisy inputs

Our data obviously is "in the wild" and noisy.
Maybe when we will refocus on improving our models, I will run some experiments training on small and clean datasets,


simple plugin to build a changelog.md with all versions

This is not pretty but I wanted an easy way to browse through the whole history, and this fit the bill.

Current coverage is 79.70%

Merging #1158 into master will decrease coverage by -0.01% as of a2f51f8

@@            master   #1158   diff @@
======================================
  Files          162     162
  Stmts         8828    8828
  Branches       263     263
  Methods          0       0
======================================
- Hit           7037    7036     -1
  Partial          0       0
- Missed        1791    1792     +1

Review entire Coverage Diff as of a2f51f8

Powered by Codecov. Updated on successful CI builds.

👍🏾

5.0.12


Set k8s binary as an option in order to be overriden by OCP bin

While installing openstack-operator on Openshift cluster, it can be necessary to override the kubectl binary by the Openshift bin.

/ok-to-test

what I usually do is alias kubectl=oc


PHPC-454: Remove WriteResult::getInfo() method

https://jira.mongodb.org/browse/PHPC-454

Is there nothing to remove from he write_concern struct?
If no, then lgtm

Nothing at all. We simply refer to a mongoc_write_result_t, which also had no relevant info fields.


Add glossary entry for Quantity

Issue: https://github.com/kubernetes/website/issues/30885
Old PR: https://github.com/kubernetes/website/pull/39014
Hello, I am trying to solve the above given issue. From my old pr you can see that I have messed up the final part before merge which is squashing a pr. Because of this I am opening this pr to make things wasy for me. Thank you.

Hello @NitishKumar06 please merge this pr. Thank you.

/retitle Add glossary entry for Quantity
Also, please rename content/en/docs/reference/glossary/Quantity.md to content/en/docs/reference/glossary/quantity.md (all lowercase)

This file already exist. Check it out here. Now should I make change to the old file or close this pr.

I would change the existing file. You don't need to make a new PR. Sorry about the mix up.

/hold
There is a glossary entry already; we might want to keep it as-is, or to clarify it.

if you want to clarify I would like to help you in that. since I have already worked on it. I have better understanding like what to add or not. If you are okay i will start to work on that for sure.


How to disable colors?

Description
I would like to pipe the output of neofetch into another command, but cannot figure out how to disable the ASCII escape codes that litter the output. I did not see any options in man neofetch that appeared to be relevant.
Neofetch version
3.1.0
Config file
None

You mean "color bar"?
Open your configuration file in .config and use # to make comment and hide info cols. I think that this is the bar you want to hide.

No, he means the escape codes that format the text.
Neofetch currently doesn't have an option to do this and I'm not sure if I'll add this feature since Neofetch wasn't intended to be piped into other commands. If you'd like to use some info from Neofetch I'd instead suggest looking into the source code to find the file or system command that displays the information you'd like to pipe.

The most frequent thing that I use Neofetch for is filling in the "System Information" section in an issue report. So I run neofetch and then copy/paste the output. But I would much prefer to just do neofetch | pbcopy!

Maybe this would suffice for your usecase?
neofetch|sed 's/\x1B\[[0-9;]*m//g'
Using this, we would not need to modify neofetch.

Well, that would technically work, but it seems a little bit of an excessively hacky workaround to me. I could certainly put it in an alias, though. Thanks!

Actually, I take it back. Before, it looked like this:
                -/+:.
               :++++.
              /+++/.
      .:-::- .+/:-``.::-
   .:/++++++/::::/++++++/:`
 .:///////////////////////:`
 ////////////////////////`
-+++++++++++++++++++++++`
/++++++++++++++++++++++/
/sssssssssssssssssssssss.
:ssssssssssssssssssssssss-
 osssssssssssssssssssssssso/`
 `syyyyyyyyyyyyyyyyyyyyyyyy+`
  `ossssssssssssssssssssss/
    :ooooooooooooooooooo+.
     `:+oo+/:-..-:/+o+/-
<EMAIL_ADDRESS>----------------------------
OS: OS X El Capitan 10.11.6 15G1510 x86_64
Model: MacBookPro11,3
Kernel: 15.6.0
Uptime: 16 days, 14 hours, 45 minutes
Packages: 340
Shell: zsh 5.3.1
Resolution: 3008x1692, 1440x900
DE: Aqua
WM: Quartz Compositor
WM Theme: Blue
Terminal: iTerm2
Terminal Font: InputMono-Medium 14
CPU: Intel i7-4980HQ (8) @ 2.80GHz
GPU: Intel Iris Pro, NVIDIA GeForce GT 750M
Memory: 5564MiB / 16384MiB

Now, it looks like this:
                -/+:.
               :++++.
              /+++/.
      .:-::- .+/:-``.::-
   .:/++++++/::::/++++++/:`
 .:///////////////////////:`
 ////////////////////////`
-+++++++++++++++++++++++`
/++++++++++++++++++++++/
/sssssssssssssssssssssss.
:ssssssssssssssssssssssss-
 osssssssssssssssssssssssso/`
 `syyyyyyyyyyyyyyyyyyyyyyyy+`
  `ossssssssssssssssssssss/
    :ooooooooooooooooooo+.
     `:+oo+/:-..-:/+o+/-
<EMAIL_ADDRESS>----------------------------
OS: OS X El Capitan 10.11.6 15G1510 x86_64
Model: MacBookPro11,3
Kernel: 15.6.0
Uptime: 16 days, 14 hours, 44 minutes
Packages: 340
Shell: zsh 5.3.1
Resolution: 3008x1692, 1440x900
DE: Aqua
WM: Quartz Compositor
WM Theme: Blue
Terminal: iTerm2
Terminal Font: InputMono-Medium 14
CPU: Intel i7-4980HQ (8) @ 2.80GHz
GPU: Intel Iris Pro, NVIDIA GeForce GT 750M
Memory: 5572MiB / 16384MiB

So some of the escape codes are gone, but not all of them.

Those sequences are not generated by neofetch:
color() {
    case "$1" in
        [0-6]) printf "%b" "${reset}\033[3${1}m" ;;
        7 | "fg") printf "%b" "\033[37m${reset}" ;;
        *) printf "%b" "\033[38;5;${1}m" ;;
    esac
}

Do you have a wrapper in there that uses ncurses? Looks alot like curses output to me.
Updated sed line would be like neofetch|sed 's/\x1B\[[0-9;\?]*[a-zA-Z]//g'

They are generated by neofetch.
On Mon, 26 Jun 2017 at 2:38 am, Lain<EMAIL_ADDRESS>wrote:

Those sequences are not generated by neofetch
https://github.com/dylanaraps/neofetch/blob/master/neofetch#L3561:
color() {
case "$1" in
[0-6]) printf "%b" "${reset}\033[3${1}m" ;;
7 | "fg") printf "%b" "\033[37m${reset}" ;;
*) printf "%b" "\033[38;5;${1}m" ;;
esac
}
Do you have a wrapper in there that uses ncurses? Looks alot like curses
output to me.
Updated sed line would be like neofetch|sed 's/\x1B[[0-9;?]*[a-zA-Z]//g'
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dylanaraps/neofetch/issues/753#issuecomment-310913286,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGfAa1WCMyLvP_VmXjK10X8lgrpmub17ks5sHo0bgaJpZM4N-0O-
.

Ah i see.
The updated sed line should work anyways.

Judging by the output, it looks like things are no longer aligned, presumably since the alignment is done using escape codes. I can see how it would be difficult to add streaming support to Neofetch—but as a workaround, is there a way of running a command in a pseudo-tty and then copying the contents of the tty? That would really be what I want.
(But the sed command does work in stripping out the escape codes; thanks!)

You can now disable colors in stdout mode.
Run neofetch with --stdout flag enabled and you're good to go. The ASCII/Images will be disabled.

Nice! But there are still some escape codes at the bottom:
% neofetch --stdout | pbcopy

gives me
rrosborough@[redacted]
-----------------------------
OS: macOS Sierra 10.12.5 16F73 x86_64
Model: MacBookPro11,4
Kernel: 16.6.0
Uptime: 36 days, 1 hour, 35 mins
Packages: 225
Shell: zsh 5.3.1
Resolution: 1920x1200
DE: Aqua
WM: Kwm
Terminal: iTerm2
Terminal Font: InputMono-Medium 12
CPU: Intel i7-4980HQ (8) @ 2.80GHz
GPU: Intel Iris Pro
Memory: 4804MiB / 16384MiB

Try this:  neofetch --stdout --color_blocks off

It works great, thanks.
<EMAIL_ADDRESS>-----------------------------
OS: macOS Sierra 10.12.5 16F73 x86_64
Model: MacBookPro11,4
Kernel: 16.6.0
Uptime: 36 days, 1 hour, 55 mins
Packages: 226
Shell: zsh 5.3.1
Resolution: 1920x1200
DE: Aqua
WM: Kwm
Terminal: iTerm2
Terminal Font: InputMono-Medium 12
CPU: Intel i7-4980HQ (8) @ 2.80GHz
GPU: Intel Iris Pro
Memory: 5019MiB / 16384MiB

I'm slightly bothered by the extra blank lines at the bottom, but I can live with it.

@konimex I'm not sure, but shouldn't color_blocks"=off" be added to the stdout() function?

I totally forgot about that. My bad.
The color blocks have been disabled. I'm still not sure about whether we should disable line_break or not for stdout mode, but it's the cause of the extra blank lines.

Yes, I think we should disable the line breaks.


Allow comparison_plots.py to take trackers name to include in plots as parameters

Allow the comparison_plots.py script to take as parameters a list of trackers to include in the generated plots.
If no parameter is provided, all the trackers found in the trackers folder are included (default and previous behavior).
If a tracker name is provided as parameter but not found in the trackers folder, an exception is raised and the script stops.
It can be convenient in scripts (or even manually in the command line) to choose which trackers to include in the generated plots, without having to delete the folder of the "unwanted trackers" in the trackers/ folder.

Hi,
Any update on this?
Thanks


The drivers are signed by new key. This updates the public key

release: using RSA public key for the signed drivers.
We changed the algorithm to RSA to sign the drivers as it has longer validity.
So we are replacing the old unused public key with a new one.
Signed-off-by: Manish Regmi<EMAIL_ADDRESS>

please modify the commit log following below template:
release: using (the type of public key) public key for the signed drivers
Briefly explain the type of the public key, and the reason why we use this new public key to replace the old one.
Signed-off-by: Manish Regmi<EMAIL_ADDRESS>

please modify the commit log following below template:
release: using (the type of public key) public key for the signed drivers Briefly explain the type of the public key, and the reason why we use this new public key to replace the old one. Signed-off-by: Manish Regmi<EMAIL_ADDRESS>
Done


add a benchmarking mode to flutter analyze

add a benchmarking mode to flutter analyze

{
  "time": "47.029",
  "issues": 0,
  "expected": "25.0"
}

@pq @yjbanov @Hixie

LGTM!

Very cool :)


Document how run egeria from official packages efficiently

The official build of egeria is effectively the files we lay down in /Users/jonesn/src/egeria/open-metadata-distribution/open-metadata-assemblies/target
See also #2375
As we move to more dynamic loading, the target tree will diverge more from the source tree - for example in terms of where data files are put, where dynamically libraries are loaded (perhaps the biggest impact)
Some developers may already make use of the assemblies, whilst others (including me) tend to run the jars directly at the command line, and others run the code from within intellij
The simplest, easy way of running egeria components from both the IntelliJ IDE and command line, making use of the asemblies, should be clearly documented. As part of this we need to ensure as developers modify their classes, they can quickly and simple test their changes with minimal impact - for example this may need additional IntelliJ 'run' configurations.
Primarily documentation but we may identify code improvements

One reported issue has been running the server chassis jar & getting connectors dynamically loaded.
This does work

Create a run configuration as usual for the server chassis (JAR)
set 'VM options' to something like '-Dserver.port=8083 -Dloader.path=/Users/jonesn/src/egeria/open-metadata-resources/open-metadata-deployment/docker/egeria/target/assembly/server/lib/discovery-service-connectors-1.5-SNAPSHOT.jar'

I actually created 4 - with only the server port different, so on ports 8080,8081,8082, 8083. This matches our typical tutorial environment.
With this done I was able to run through the config, start,discovery notebooks without errors.
If the loader.path is omitted, the notebooks fail as expected - being unable to load classes
Note

I did not use the 'spring' capability specific in IntelliJ to the paid-for edition
I only tried a single dynamically loaded module

One 'tension' between the intellij environment & command line is that at the cli maven will generally build the right things, is coded to build the assemblies, and it's easy to then run the code directly from the assembly output
In the IDE the objective is quick turnaround.
A closer approximation is instead to change the launch configuration to run 'Users/jonesn/src/egeria/open-metadata-distribution/open-metadata-assemblies/target/egeria-1.5-SNAPSHOT-distribution/egeria-omag-1.5-SNAPSHOT/server/server-chassis-spring-1.5-SNAPSHOT.jar' and to set the libpath to '/Users/jonesn/src/egeria/open-metadata-distribution/open-metadata-assemblies/target/egeria-1.5-SNAPSHOT-distribution/egeria-omag-1.5-SNAPSHOT/server/lib' . This then is exactly how the CLI would launch the chassis
However if this is done it's not clean IntelliJ will 'know' how to update that jar, without the maven open-metadata-assemblies module being explicitly built - so there is more chance of running old code.
In fact given there now is no need for a hard maven dependency (as the loading is dynamic) how would intellij know what to build, when?
A hybrid option is to run the main jar directly from the chassis project, but point to the generic lib - this does ensure one is running the latest chassis,. but could mistakenly run older connectors.
I would be inclined, for the dev environment, to stick with options similar to the first. Explicitly add all required libraries in their 'native' locations.
A little more research on intellij needed to figure out what is definately the best approach, and document with minimal steps needed

IntelliJ also has it's own concept of artifacts, somewhat similar to maven assemblies, but there is no synchronization between the two (https://www.jetbrains.com/help/idea/working-with-artifacts.html)
this issue seems to have been discussed over many years. The best suggestion so far is to go to the bottom of the 'run/debug configuration' window, and in the last box where it says 'before launch' add an additional action to run a maven goal, specifying the assembly maven module, and 'clean package' as the goal. This should get the assembly rebuilt before each run, and should allow the launch config to point directly at the jar in the assembly, plus dynamic libraries in the assembly....

This seems to work, but extends launch time significantly (30-60s). It may be possible to optimize the assembly maven pom so that we have a special 'dev' flag that for example

only runs the omag server assembly, not all
skips source & javadoc
only deposits the files as per the assembly definition
no .gz

This could reduce the time to less than half.

This will be looked at after we've slimmed down the jars & platform.
Will keep this open as a reminder this needs clear docs for any changes

This will be looked at after we've slimmed down the jars & platform.
Will keep this open as a reminder this needs clear docs for any changes

Will persue this in #4667 as part of refactoring


Export of multiple datasources returns with exit code 0 when some fail

wizzy export datasources
Output:
{
  "id": 1,
  "message": "Datasource added"
}
✔ Datasource export successful.
Output:
Grafana API response status code = 500
No error body from Grafana API.
✘ Datasource export failed.
Output:
Grafana API response status code = 500
No error body from Grafana API.
✘ Datasource export failed.
Output:
{
  "id": 2,
  "message": "Datasource added"
}
✔ Datasource export successful.

Are you using sqlite?

Yes

So, the rootcause of the error is database locked down in sqlite. You might want to follow this thread - https://github.com/grafana/grafana/issues/7013
What we can do until grafana fixes the bug to use 'sync-requests' instead of 'requests'. Will make the change sometime today.

@sstarcher Made the export code to use sync-request. Please try it now!

This should be fixed too. Please reopen if not fixed!


Updating from 3.1.5 to 3.1.6 resulted to a multidex error when building

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformDexArchiveWithExternalLibsDexMergerForDebug'.
> java.lang.RuntimeException: java.lang.RuntimeException: com.android.builder.dexing.DexArchiveMergerException: Unable to merge dex

Enabling multidex in our project resulted to more errors. Before I try to deal with multidex, trying to figure out if someone knew how to fix the above issue.
No problem with version 3.1.5

I believe the issue arises from this line:
https://github.com/BranchMetrics/cordova-ionic-phonegap-branch-deep-linking/blob/7df49cb0ee351a6faa6eddf052464167baedf4a8/src/android/dependencies/rules.gradle#L2
so if other *.aar files exist in the project, they will be re-copied and cause duplications.

Hello. any update on this ? it s quite a big issue ....

We're getting the same issue, downgrading to 3.1.5 fixed the problem

Thanks for bringing this to our attention.  We have a ticket to dig into this issue next sprint.
One quick thing to check, is the old branch jar file and the aar file both present?   If that's the case we missed the removal of the old jar file.

@rj-david , can you give us a bit more information about your configuration? What version of Cordova are you using, gradle version, android sdk version, etc.?
We are investigating this issue, but with all of the different interdependencies, its hard to know exactly what will fix the problem in this scenario. A fix for Cordova 7 is different than a fix for Cordova 9.

This should be resolved in the latest release.


cmd/vet: false negative in vet -shadow

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
$ go version
go version go1.7 darwin/amd64

What did you do?
go tool vet -shadow finds one shadow for the hash variable, but fails to notice the password shadow.
Source:
https://github.com/mcandre/go-ios7crypt/blob/61a282b2d8003e9372cbe37449c7008ee142f1fa/cmd/ios7crypt/main.go#L29-L43

The shadow tool is very weak and has many known bugs. I'll leave this open but don't expect a fix. A likelier resolution is a complete replacement of the checker.

go vet has been rewritten since this issue was opened, and the shadow check is no longer available via a flag. It can still be installed, but manually from x/tools; see https://go-review.googlesource.com/c/go/+/154584/.
Given that the check never made it out of the "experimental" phase, and that it's no longer part of vet, and that noone plans to fix this issue in particular, I'm closing it for now.


should six.io.uilts.h5ls take as constructor argument silx.io.fabioh5.File

this will insure compatibility with the silx toolkit.
Might also take an edf file has input

Closed by #576


Email ended with 'dot' before @ sign

Hi I meet a case when my user inputing email that ended with '.' before the @ sign will passes the validation.
here the case
const validator = require('validatorjs')
const rule = {email:'email'}
const data =<EMAIL_ADDRESS>
const validation = new validator(data, rule)
validation.passes()
// return true

the problem is my backend validation (using laravel) say it false.
So, which one that actually right? is it should be pass or not?
Thanks

Ya. I believe that this should fail. I will take a closer look to confirm.

Fixed in pending 3.23 release (should be ready by EOD)


Add is_subset and is_superset

Behavior should be identical to std::collections::HashSet::is_subset/is_superset

Hi, it's a nice thought but we need a more efficient implementation, for example running the algorithm block wise.


Add concurrency limits

Introduce concurrency limits for the server configuration and enhance the configuration structure to include default values. Add tests to ensure proper functionality of the default population logic.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.56%. Comparing base (6dee113) to head (380804b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #74      +/-   ##
==========================================
+ Coverage   94.51%   94.56%   +0.05%
==========================================
  Files          30       30
  Lines        1186     1197      +11
==========================================
+ Hits         1121     1132      +11
  Misses         45       45
  Partials       20       20

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Annotation for the frame field

I have the angles for each pixel as a .npy file to be used as groundtruth as mentioned in the paper.
My question is: is this directly used in the network or is there a need to calculate frame field to be fed into the network? If so what should the format be?
I would appreciate any kind of help. Thank you.

The angles to be fed into the network are generated during training data generation process implemented as data_transforms (f.e. https://github.com/Lydorn/Polygonization-by-Frame-Field-Learning/blob/master/dataset_folds.py). You just need to adjust transformation to your own data.

@patriksabol Thank you for you input. I was able to understand that we only need the images and binary masks as input and not the angles as they will be calculated while running the network.
I was able to run the network and got the training data as '.pt' file in processed folder. Although the training stopped with error half way I was still able to get half of the data in processed folder.
I will close this issue now.


net/http: Server closeIdleConns does not close StateNew connections

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
1.9.2
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
any (linux x86_64)
What did you do?
Long Story: my web app was not closing when calling Shutdown on my http server. Google Chrome for some reason was creating a couple new connections to my server, but not making any requests on them. I'm guessing this is for performance, but I couldn't find any explanation. These extra connections stay in "StateNew" state, so, Shutdown does not close them when closing idle connections.
Short Story: Tried to "Shutdown" http server with StateNew connections.
https://play.golang.org/p/dP4vQZkKdx
What did you expect to see?
Since StateNew is an idle state, I'd expect it to be closed during Shutdown.
What did you see instead?
Shutdown must be cancelled to clear StateNew connections.

The returned error is correct, it is of type contextDeadlineExceededError and is thrown because the context timed out. See here: https://play.golang.org/p/JmX0nEqeqa
Unless I'm missing something, please close this issue.

From net/http/#Server.Shutdown:

Shutdown works by first closing all open listeners, then closing all idle connections, and then waiting indefinitely for connections to return to idle and then shut down. If the provided context expires before the shutdown is complete, Shutdown returns the context's error, otherwise it returns any error returned from closing the Server's underlying Listener(s).

As a potential solution to your problem, you might have more luck experimenting with the IdleConnTimeout setting from http.Transport. I and others would be more than happy to help if you post this as a question on the go-nuts mailling list.
Closing, since there is no bug.

Thanks for the quick response. I apologize if the problem description and example did not make the issue clear. The problem is that a new, unused connection is NOT considered "idle" by closeIdleConns and is therefor NOT closed during Shutdown. The only reason I added a context with a timer to my example was to clearly show that the idle connection was not being closed, instead, the context is timing out.  The only difference between StateNew and StateIdle is that a StateNew connection has never made a request, correct?

Since StateNew is an idle state

Go has StateNew different from StateIdle intentionally. New isn't idle. A new connection is expected to have a request sent on it soon, and browsers have certain expectations of the server not hanging up on them for new connections. But keep-alive (idle) connections are much more likely to be closed at random times, and clients expect this more.
It's intentional that Shutdown doesn't close StateNew connections.
But if Chrome is keep some warmed up TCP connections in StateNew, maybe we need to do something. Maybe close StateNew after some duration (~2-5 seconds?).
/cc @tombergan

This is somewhat duplicated by #21204

There are cases where clients will leave a connection open and not use it. For example: an HTTP/1 client might start a new connection but switch to an existing connection if one becomes idle before the dial completes. If that happens, and there are no more requests to send, the new connection will sit around unused. Clients might also "preconnect" to origins that the client predicts it will visit shortly.
@kcburge, have you tried using ReadHeaderTimeout? I think that will solve your problem. If you set ReadHeaderTimeout to 10s, then Shutdown should return within 10s after all outstanding requests complete.

From #21204, @pamburus points out that ReadHeaderTimeout is infinite by default. As a result, the default behavior of Shutdown is to block indefinitely when there are StateNew connections. This seems undesirable.

@bradfitz
It's intentional that Shutdown doesn't close StateNew connections.

Do you expect anything will break if Shutdown closes StateNew connections? The only possible breakage I can imagine is if we close a connection while a POST is in flight -- the client won't know if we processed any of the POST and thus cannot know if it's safe to retry the POST. This doesn't seem very significant. It might be safe to close StateNew connections immediately in Shutdown.

I'm fine closing them if they're a few seconds old.
My concern was closing a request that was just being sent.

I'm not an expert in writing http servers, but, closing the State New connections either "now" (immediately when Shutdown is called, in closeIdleConns), or at some time later (through ReadHeaderTimeout or during Context cancellation, one of which is required by the current implementation), only differs as to WHEN the connection is closed: either way, the connection could have a POST in flight, or the client could be about to submit a request, or....  So, this makes the discussion about those things moot: either way, the connection is going to be closed. That's why I asked why make the server wait if it knows nothing has happened on the socket. Thanks for looking at this.


Rewrite "Job Search I: The McCall Search Model"

Has all of our big changes (NLsolve, Expectations, Structs, etc.) @jlperla I think the rewrite you sent me will come into play for McCall 2.

Add a test to the final cell based on stop_times

Thanks @Nosferican, added that test cell.


[Bug]: Searching by spatial does not include collecting events that only have a Geographic Area but no georeference

Steps to reproduce the bug
1. When I assign a CO to a CE that has only a geographic area (e.g. it's a county-level record so a georeference is meaningless)
2. Then I go to Filter COs
3. And I select "Geographic Area" to Filter
4. And I select "Spatial"
5. Then that CO/CE does not get returned until I add a georeference, even if the GeoArea assigned has a polygon/shapefile.

Request blocker as this is preventing us from properly downloading records for an upcoming paper.

Screenshot
No response
Expected behavior
Should return all COs/CEs with a GeoArea that spatially matches AND those that have a georeference that matches.
Additional Screenshots
No response
Environment
Production
Sandbox Used
No response
Version
0.44.2
Browser Used
Firefox

@tmcelrath Need a filter -> JSON example please?

Not showing up:
https://sfg.taxonworks.org/tasks/collection_objects/filter?per=500&otu_id[]=164018&geographic_area_id[]=33446&geographic_area_mode=true&extend[]=dwc_occurrence&extend[]=repository&extend[]=current_repository&extend[]=collecting_event&extend[]=taxon_determinations&extend[]=identifiers&exclude[]=object_labels&page=1

Should be showing up under first result but isn't: https://sfg.taxonworks.org/tasks/accessions/comprehensive?collection_object_id=788501

Notes for resolution, none are 100% statisfactory

Add `expand_buffer= option to let users clip in other near-similar shapes; only present when target has shape
Add match_name=true option to let users do lateral/down expansions of the result as well

Evolution beyond this will be piecemeal until  is to move to a single unified dataset (geoboundaries?) with optional user Gazeteer.
https://github.com/SpeciesFileGroup/taxonworks/issues/1954
https://github.com/SpeciesFileGroup/taxonworks/issues/3627

More notes

[ ] Refactor to make modes generalized and addative
[ ] We are missing extensions in spatial the include non-spatial descendants

Adding some extra push here because this is an issue I am running into as well for my Arkansas Plecoptera paper. Many records are left out when I filter out COs due to this issue!

We spent the afternoon running through various exercises.  One of the take-homes is that if GBIF can return the result set that you expect then we can replicate it in TaxonWorks.
The entry-point that got us a long way was searching on CollectingEvent cached_adm1 fields for 'Arkansas' in either the Project Vocabulary or DwcOccurrence filter then pivoting those results to specimen records etc.


Documentation for ReadOnly is inaccurate regarding the number of copies per process

Original issue: https://charm.cs.illinois.edu/redmine/issues/1571

"They are broadcast to every PE by the Charm++ runtime, and can be accessed in the same way as C++ ``global'' variables on any PE. "
Readonly data is broadcast to every Cmi Node (process), only one copy exists per process. Not per PE. This applies to simple types and complicated types.  We only pup it once.
This inaccuracy could lead to at least two different kinds of trouble

Assume that one needs a different mechanism to get one copy per process readonly data, thereby wasting effort reinventing the wheel.
Falsely assume your readonly data has an independent copy on each PE, then modifying it (in violation of the readonly label), thus leading to ugly race conditions and correctness failures when all the PEs of a single node are in fact modifying the same data.

Original date: 2017-05-24 03:47:39

~~https://charm.cs.illinois.edu/gerrit/#/c/2549/~~ https://github.com/UIUC-PPL/charm/commit/a9cab4678a88b9054475ade4cecc7e4c74119586


Check for active_configuration before using it.

Using usbproxy.py with an older Yubikey results in this error when the target system attempts to attach the device via the proxy.
INFO    | logging        | [17:55:07] USBControlRequest(direction=1, type=0, recipient=1, number=6, value=8704, index=0, length=71, data=bytearray(b''), device=USBProxyDevice(name='generic device', device_class=0, device_subclass=0, protocol_revision_number=0, max_packet_size_ep0=64, vendor_id=24843, product_id=18003, manufacturer_string='Facedancer', product_string='Generic USB Device', serial_number_string='S/N 3420E', supported_languages=(<LanguageIDs.ENGLISH_US: 1033>,), device_revision=0, usb_spec_version=512, device_speed=None, requestable_descriptors={<DescriptorTypes.DEVICE: 1>: <function USBBaseDevice.__post_init__.<locals>.<lambda> at 0x7efffe1984a0>, <DescriptorTypes.CONFIGURATION: 2>: <bound method USBBaseDevice.get_configuration_descriptor of ...>, <DescriptorTypes.STRING: 3>: <bound method USBBaseDevice.get_string_descriptor of ...>}, configurations={}, backend=<facedancer.backends.greatdancer.GreatDancerApp object at 0x7efffd0c1590>))
INFO    | logging        | [17:55:07] <: b'\x05\x01\t\x06\xa1\x01\x05\x07\x19\xe0)\xe7\x15\x00%\x01u\x01\x95\x08\x81\x02\x95\x01u\x08\x81\x01\x95\x05u\x01\x05\x08\x19\x01)\x05\x91\x02\x95\x01u\x03\x91\x01\x95\x06u\x08\x15\x00%e\x05\x07\x19\x00)e\x81\x00\t\x03u\x08\x95\x08\xb1\x02\xc0'
TRACE   | greatdancer    | EP0/IN: <- b'\x05\x01\t\x06\xa1\x01\x05\x07\x19\xe0)\xe7\x15\x00%\x01u\x01\x95\x08\x81\x02\x95\x01u\x08\x81\x01\x95\x05u\x01\x05\x08\x19\x01)\x05\x91\x02\x95\x01u\x03\x91\x01\x95\x06u\x08\x15\x00%e\x05\x07\x19\x00)e\x81\x00\t\x03u\x08\x95\x08\xb1\x02\xc0'
INFO    | greatdancer    | Disconnecting from host.
Traceback (most recent call last):
  File "/home/user/hack/usb/facedancer/examples/usbproxy.py", line 29, in <module>
    main(proxy)
  File "/home/user/hack/usb/facedancer/facedancer/devices/__init__.py", line 44, in default_main
    device.emulate(*coroutines)
  File "/home/user/hack/usb/facedancer/facedancer/device.py", line 234, in emulate
    self.run_with(*coroutines)
  File "/home/user/hack/usb/facedancer/facedancer/device.py", line 219, in run_with
    asyncio.run(inner())
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/user/hack/usb/facedancer/facedancer/device.py", line 217, in inner
    await asyncio.gather(self.run(), *coroutines)
  File "/home/user/hack/usb/facedancer/facedancer/device.py", line 206, in run
    self.backend.service_irqs()
  File "/home/user/hack/usb/facedancer/facedancer/backends/greatdancer.py", line 792, in service_irqs
    self._handle_transfer_events()
  File "/home/user/hack/usb/facedancer/facedancer/backends/greatdancer.py", line 496, in _handle_transfer_events
    self._handle_transfer_readiness()
  File "/home/user/hack/usb/facedancer/facedancer/backends/greatdancer.py", line 642, in _handle_transfer_readiness
    self.connected_device.handle_buffer_available(endpoint.number)
  File "/home/user/hack/usb/facedancer/facedancer/device.py", line 383, in handle_buffer_available
    endpoint = self.get_endpoint(ep_num, USBDirection.IN)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/hack/usb/facedancer/facedancer/device.py", line 344, in get_endpoint
    endpoint = self.configuration.get_endpoint(endpoint_number, direction)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/hack/usb/facedancer/facedancer/configuration.py", line 151, in get_endpoint
    for interface in self.active_interfaces.values():
                     ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'USBConfiguration' object has no attribute 'active_interfaces'. Did you mean: 'get_interfaces'?

Proxying system is Linux and target system is macOS.
The issue seems to be that the USBConfiguration class assumes active_interfaces always exists. However that not always the case it seems. Adding an explicit check for it resolves the issue for me and the proxied device works as expected.

There is indeed a bug here, but this isn't the right way to fix it.
The underlying problem here is that I broke USBProxy when I added alternate interface support in PR #122. :cry:
The active_interfaces attribute needs to be initialized to default values when the configuration is applied with a SET_CONFIGURATION request, and also needs to be updated when alternate interface settings are applied with a SET_INTERFACE request. I did that in the standard request handlers on USBDevice, but those aren't used by the proxy code.
So we need some changes to USBProxy to do this. I've just opened a draft PR #137, which implements the SET_CONFIGURATION part of this and may work for you.
But it doesn't yet deal with intercepting SET_INTERFACE requests and updating the active_interfaces, and actually that opens a bigger can of worms, because on backends with a fixed number of hardware endpoints, we're sometimes going to need to reallocate those on a SET_INTERFACE request, because that action changes the set of available endpoints that the host can use.
PR #122 didn't address that, and addressing it properly is going to require new handling in all of those backends. :sob:


Switch between View/Complete Notes for Pre-session

Any context for feature
When we add notifications to our ALUM app, we need to somehow detect whether the user has completed the notes or not so as to stop notifying them. This is also important to know to display the View pre-session notes vs Complete pre-session notes on the session details page. Solution to this is we store the status of a note as saved or not in our mongoDB object for the session such that the data persists.
Requirements for feature

[x] When user saves the pre-session notes (PATCH notes), we need to mark the note as modified so that should be done by adding a sessions schema field for each type of note.
[x] Note this field needs to be added to the schema and the GET sessions route
[ ] Based on this field, you need to change between "View" or "Complete" pre-session notes so the NotesService and the GET sessions response handling would need to updated according to this.

Resources / tips for development
Jenny worked on setting up the pre/post notes on frontend and Christen worked on the backend if any help is needed.

frontend change not yet made so reopening this issue

tracked by pr #103


An improved mod customization (custom stickers, custom bf freeplay, and custom results screen)

What feature do you want to get added on the P-Slice engine?
One thing I want you to include in the p-slice hotfixes is customization to certain things about v-slice
that is, the transition stickers, the freeplay bf (or almost the entire menu), and the results screen
This more than anything to perhaps give more essence to the engine, and more than anything to make it more customizable, since I tried to test the customized stickers in the p-slice and the engine crashed several times
Maybe in future updates they will add more customization and maybe make an improved modfolder

Custom stickers is definitely something I want to add in 1.1
But I still want to balance any features from v-slice with existing Psych mods


wrong

File "D:\anaconda3\envs\treelearn\lib\site-packages\spconv\pytorch_init_.py", line 7, in
from spconv.pytorch.core import SparseConvTensor
File "D:\anaconda3\envs\treelearn\lib\site-packages\spconv\pytorch\core.py", line 21, in
from spconv.tools import CUDAKernelTimer
File "D:\anaconda3\envs\treelearn\lib\site-packages\spconv\tools.py", line 16, in
from spconv.cppconstants import CPU_ONLY_BUILD
File "D:\anaconda3\envs\treelearn\lib\site-packages\spconv\cppconstants.py", line 15, in
import spconv.core_cc as _ext
ImportError: DLL load failed while importing core_cc: The specified module could not be found
How to solve this error?

It seems like you are trying to install the cpu only version of spconv? Our code is only written for supporting the cuda version. Please make sure that you have a cuda-capable gpu on your system and that the nvidia drivers and cuda are correctly installed.

python tools/pipeline/pipeline.py --config configs/pipeline/pipeline.yaml
I'm running this code, spconv-102 exists in the environment, reinstalled the environment and also this error is reported，cuda version is 10.2

Please excuse the long waiting time for our answer. This seems to be an issue with installing spconv. Please reach out to the maintainers of spconv for support with this (https://github.com/traveller59/spconv). In general, we can only provide support running TreeLearn on Linux systems and not windows (as you seem to be using), as we only ever tested it on linux systems.


some lines are too long!

I still haven't figured out a day to deal with tables and code example blobs. So I'll refrain from making .jshintrc-tset yell at long lines for now.
@jsaleigh
Reviewers: you can click on display rich diff on Reference.md to see that I have done minimal modification to the actual appearance of the doc

:+1: yay for cleanup!


First try at catalog support

Description
First try at catalog support.

Adds catalog or database for profile to set the initial catalog.

catalog is an alias of database

Use catalog parameter to connect to the cluster when databricks-sql-connector>=2.0

Thanks! merging.


Temp folder change

hello
how can we change save folder from temp folder to other folder?

Hi, you can pass a custom path as the second parameter to the \StanDaniels\ImageGenerator\Image::create method.


Support new version of joi.

Hi guys.
@marshallswain @EdwardDrapkin @adamvr @eddyystop @claustres
As you know joi divorced with hapi and a new version of joi is not compatible with the current version of validate-joi.
Do you have plans to migrate to the latest version of joi? if you are busy I can do it. ;)

That would be wonderful to make the switch. We can do it as a new major version number.  If you have the time to make the PR that would be very much appreciated.


Gizmo Manager Use Default Utility Layer

Please read the comments here
The purpose of this PR is to ensure GizmoManager uses the UtilityLayerRenderer.DefaultUtilityLayer by default.
If an alternate UtilityLayer is desired, it can be input in GizmoManagers constructor.
This is critical for applications who want to render utilityLayer outside of the main render loop as seen in this doc.

LGTM! @CedricGuillemet ?

Let me test it ....

Good for me!


Login tracking added

Fixes #20
How many users are logged in can be tracked now.

Database is now working

great work @rkishan516  :)
Things, I noticed.

You need to add is IP column in users_key.
Logout not working it's not updating the status.

And don't insert in table always,  insert when new IP is detected for that user then. If user login or logout with the same IP then just update with a new token and new time stamp and insert of new IP found.

Working Fine but when New Ip is detected we are updating the last ip

Also check here user login from database. Check current session token is valid with user_key with this ip https://github.com/NJACKWinterOfCode/iitpConnect/blob/2ab02a6b940feb5725d995927dba494cd45d5bb5/libraries/classes/User.php#L120

Oops wrong button


Not possible to add Next.js API routes due to shopify-auth setup

Issue summary
It is not possible to follow Next.js documentation for creating API routes in your pages/api directory. It's caused by the default setup and documentation how koa-shopify-auth is being used. As its documented right now, every single GET request to your server will be forced to authenticate, which forces any SSR request to fail, because the server is trying to verify its own API calls.
Expected behavior
The app template should support usage of Next.js API routes feature.
Actual behavior
requests to /api are being redirected to /auth.
Reduced test case
https://nextjs.org/docs#api-routes
Add the simplest of pages to pages/api/index.js that just returns 200.
Add an API call in pages/index.js using getInitialProps to that newly added /api route.

@tolgap Were you able to figure out how to convert the server.js middleware into the new API routes? You can't host using now.sh if we are using a custom server.js file so I'm working on migrating auth and graphQLproxy to api routes, but I'm having some problems.

@junaid33
Yes I did fix it. The problem was that cookies (Shopify's and your own app's) were not being passed into the fetch API calls. So I created a small wrapper around fetch that just does that:
// lib/http.js

import qs from "qs";
import _ from "lodash";
import fetch from "isomorphic-fetch";
import { getCookies } from "cookies-next";

const joinCookies = ctx => {
  return _(getCookies(ctx))
    .map((value, key) => {
      return [key, value].join("=");
    })
    .join(";");
};

export const get = async (ctx, path, params) => {
  const res = await fetch(
    [
      process.env.HOST,
      path,
      qs.stringify(params, { addQueryPrefix: true })
    ].join(""),
    {
      method: "GET",
      headers: {
        Cookie: joinCookies(ctx)
      }
    }
  );

  return await res.json();
};

export const post = async (ctx, path, params) => {
  return await fetch([process.env.HOST, path].join(""), {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Cookie: joinCookies(ctx)
    },
    body: JSON.stringify(params)
  });
};

export const patch = async (ctx, path, params) => {
  return await fetch([process.env.HOST, path].join(""), {
    method: "PATCH",
    headers: {
      "Content-Type": "application/json",
      Cookie: joinCookies(ctx)
    },
    body: JSON.stringify(params)
  });
}

export default {
  get,
  post,
  patch,
  put: patch
};

This small http lib accepts the current Next.js context as the first parameter, and figures out the cookies from that context.
Usage:
// pages/post/[id].js

class Post extends React.Component {
  static async getInitialProps(ctx) {
    const { id } = ctx.query;
    const { data: { post } } = await http.get(ctx, `/api/posts/${id}`);

    return { post };
  }
}

disclaimer: I am 100% sure this can be done in a better way "Next"-style way, but I am still figuring out Next.js. So if you do find a better way to automatically pass cookies into these calls, please let me know in this issue 👍 .

@tolgap Oh maybe I misunderstood. API routes worked fine for me with the current set-up. I think Next.js provides res.cookie to the api routes as well. My issue is converting the functions in server.js into API routes so the app can be hosted on now V2.
For example:
// server/server.js

  server.use(
    graphQLProxy({
      version: ApiVersion.October19
    })
  );

If I try to convert it into a API route, it does not work:
// pages/api/gqlproxy.js

import Koa from 'koa';
import { graphQLProxy } from '@shopify/koa-shopify-graphql-proxy';

const server = new Koa();
const handler = (req, res) => {
     graphQLProxy({shop: "testshop", password:'testpass',});
};

export default server.use(handler);

Any idea on how to migrate with middleware into a serverless api route?

Seeing as I figured this out (by making sure I'm using cookies when performing the API calls in getInitialProps), I'm closing this issue.

Having the same issue

Hello,
I created a package to remove the custom server and managed to deploy an app shopify with only nextjs on Vercel.
The package: https://github.com/bluebeel/nextjs-shopify-auth
The demo app: https://github.com/bluebeel/nextjs-shopify
It only lacks the webhooks management if I have time or if someone wants to contribute.
Cheers :)
cc: @Steven4294


Fixed bug with blank space on the end of query

Fixes #108

i got an error:

undefined method `strip' for nil:NilClass

@polosaty fixed in #112 and available in the new release https://github.com/flant/loghouse/releases/tag/v0.2.3


refactor: cElementTree is deprecated

defusedxml.cElementTree is deprecated
splunk_add_on_ucc_framework/__init__.py:29
  /Users/arys/src/addonfactory-ucc-generator/splunk_add_on_ucc_framework/__init__.py:29: DeprecationWarning: defusedxml.cElementTree is deprecated, import from defusedxml.ElementTree instead.
    from defusedxml import cElementTree as defused_et

Codecov Report

Merging #339 (6bd987e) into main (b7e0596) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #339   +/-   ##
=======================================
  Coverage   79.48%   79.48%
=======================================
  Files          28       28
  Lines        1867     1867
=======================================
  Hits         1484     1484
  Misses        383      383

Impacted Files
Coverage Δ

splunk_add_on_ucc_framework/__init__.py
77.04% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7e0596...6bd987e. Read the comment docs.

:tada: This PR is included in version 5.9.0 :tada:
The release is available on:

v5.9.0
GitHub release

Your semantic-release bot :package::rocket:


Makefile fails to create objects in proper directory

The Makefile script fails to create the objects in the src directory. Therefore, it fails to compile the exec

https://stackoverflow.com/questions/4102469/makefile-to-put-object-files-from-source-files-different-directories-into-a-sing
https://stackoverflow.com/questions/17795557/makefile-executable-and-object-files-in-different-directories
https://stackoverflow.com/questions/15494915/how-to-write-a-makefile-to-generate-object-files-and-executable-in-different-dir

Needed to add the -o $@ flags while compiling the objects themselves.
Fixed in e1b70f292ba1f171d1f2c07ea7a2996bf9b5383d