Combined upset plot

Hi again,
I was wondering how the upset plot data is laid out for individual assignment. It looks like you converted singlets to "1" and anything labelled as a doublet to "0" for the upset. However, I am confused as to how you decided the "Final Individual Assignment" label. What did you do in the case of one tool assigning a different label than another tool? For example if demuxlet assigns a nuclei to donor 0 for example, souporcell assigned it to unassigned, vireo assigned it to donor 1, and freemuxlet to doublet. How would your upset plot show that? thanks!

Hi @changostraw,
This will depend on how you have run the Combine_Results.R script. You can use multiple different methods for combining the results with --method so the way it calls the combined annotation for each cell will be dependent on what you selected for that. In addition, the way that it combines the different methods results depends on the inputs for each as well:

If you have mapped reference donors to each cluster/cell, then it will use those annotations and call based on the --method you used
If you only had reference annotations for some of the methods but did not map to individuals for others, it will use the overlap of the clusters of cells with the annotated IDs to standardize the individual annotation across the different software. The software used as the reference for this can be selected with --ref. This defaults to vireo (if provided), then demuxlet (if provided), then freemuxlet (if provided), then souporcell (if provided). The required proportion overlapping between the groups can be updated with --pct_agreement but defaults to 0.7. Therefore, any cluster that does not overlap > 0.7 with a cluster of the selected reference, will be annotated as unassigned for that software.
If you have no SNP genotypes and, therefore, no reference IDs, the IDs will be standardized across the different softwares using the same ordering in the previous bullet

Let me know if that helps clarify or if you need some additional information.
Cheers,
Drew

Okay thanks that makes sense. I used majority singlet as the method and was looking at the wrong file for the consensus Majority Singlet Droplet Type call. I am still getting donor 1, donor0 etc despite having done the genotype step for Freemuxlet and Souporcell (I didn't add genotype to the vireo with 1000G results). The individual assignments for Freemuxlet and Souporcell with 1000G didn't get picked up it seems. I wanted to combine results from Demuxlet, Freemuxlet, Souporcell using raw donor GT, imputed donor GT and common variants from 1000G, Vireo using all three vcf files as well. However, the combine function doesn't seem to be able to call from 3 different souporcell and vireo results in the same file. Is there a way to do this? If not I can do them separately and combine them in R after. However, then I would need to figure out how to remake the upset_df dataframe with my combined 'combined_results_w_combined_assignments.tsv' if that makes sense. I have tried to do that and have had no success.
I also have a list of singlet donor assigned barcodes from physical cholesterol modified oligo tagging (hastag oligos) that I would like to add to the upset plot. So I added this information to the combined_results_w_combined_assignments.tsv. and went through the Combine_Results.R code and tried running your code on my combined results but I am new to bioinformatics and R and wasn't able to figure out how you converted the combined_results_w_combined_assignments.tsv file into a structure suitable for the upset plot with 1's and 0's and the consensus assignment. If I could recreate that dataframe within R, I could add the extra souporcell and vireo columns and the hastag oligo information and make an integrated upset plot.
Sorry for all the questions! and in one post. If you prefer I can post each question separately.  Thanks so much for this pipeline and all the documentation. It makes it so much easier for someone new to bioinformatics!

Hi @changostraw,
Really great points. First, I would say that you should indicate the -ref as either freemuxlet or souporcell so that you get the donor IDs you are looking for distributed across all the methods.
Regarding your question about multiple different results from souporcell, vireo or any other method, this is not something I've implemented but I can definitely see how it would be very useful.  I'll have to think a bit about how to do this. At first thought, a user could provide a list of results for a given parameter (ie a list of souporcell results) but if there are only genotype matching files for some, I will have to consider how that will be entered and how I would need to code it to allow that.
For now, I can help you sort out some code to make an upset plot with all your data. Could you send me either the file (you can email it to me if you prefer at<EMAIL_ADDRESS>or just send through a head of the file so I can see the structure and hopefully put together some code for the upset plot? I agree that working out code for upset plots can be rather mind boggling - especially for complicated cases like this.
I'm glad you're asking questions because it will help me optimize the tool and documentation and will help other users who may have the same questions and can then find the solutions here.

Thanks for the offer @drneavin ! I actually think I figured out most of it. But I have a few more questions.
So what I did was create two combined reports with Demuxafy so that I could have two sets of souporcell and vireo results and then combined them in R. I then added the assignments from the hastag oligos to the table. I also calculated both "MajoritySinglet" and "AnyDoublet" using the most tools as possible and added that information to the table. So my final table so far is:
CMOs (hastag oligos)
Freemuxlet 1000G
Souporcell 1000G
Vireo 1000G
Demuxlet rawGT
Souporcell rawGT
Vireo rawGT
Scds
(I will eventually add Demuxlet, Souporcell, Vireo with imputed donorGT as well)
The MajoritySinglet and the AnyDoublet calculations were made with:
Freemuxlet 1000G
Souporcell 1000G
Vireo 1000G
Demuxlet rawGT
Scds
I wanted to see if demultiplexing by genotype is as good as demultiplexing based on hashtag oligos, and if so, what method is best. So I calculated 1. CMO assignment vs MajoritySinglet assignment and 2. CMO assignment vs AnyDoublet (here I called it NoDoublet) and made an upset plot for each - attached here. Basically if the CMO and GT assignment matched it returned the agreed assignment (309,318,325,366 or Multiplet), if they didn't match it returned "Disagree", if there was no CMO assignment = "No_CMO" and no GT assignment = "no_GT". I think it worked pretty well and indicates, as you suggest in your paper, that MajoritySinglet seems to be the most accurate (at least vs hashtag oligos). However, I am not sure I am correctly interpreting the MajoritySinglet and AnyDoublet methods.
I assumed MajoritySinglet means that the majority of the tools selected agreed on the assignment (ie >50% of them). I assumed the AnyDoublet would mean that if one tool called a doublet it would be assigned doublet. However, I have noticed that in my AnyDoublet column there are doublet assignments even when none of the tools assigned it a doublet. For example if Demuxlet is "unassigned", Scds is "singlet" and Vireo, Souporcell, Freemuxlet are "singlet" and agree on assignment, AnyDoublet calls this a doublet. Is this correct? If so, is there a way to ignore "unassigned" in the calculations? (Unfortunately, right now I have an issue with the rawGT of one of the samples in my pool, so that is why Demuxlet rawGT has so many unassigned.) However, in general I am wondering if it wouldn't be better to treat unassigned as NA in the combining methods. Does that make sense?
Thanks!

Hi @changostraw ,
Very glad you were able to work this out and also very glad to hear that the oligos and the Demuxafy results seem to have pretty good agreement. We have compared antibody hashtag to genetic-based demultiplexing and find that the genetic data is slightly better because there isn't oligo/antibody jumping for genetic information and because there are a good number of cells that don't have enough antibody reads to be called for a given hashtag. It looks like that is probably what has happened for a small subset of your data as well since the demultiplexing and doublet assignments seem to be relatively consistent for those that either disagree or are missing CMO. It's great to hear that the oligos that you're working with have similar profiles. It seems like you have very clean data from what you've shared.
Regarding the MajoritySinglet and AnyDoublet Demuxafy calls, your assumptions on how the calls should be made are accurate. Can you confirm a few things?

Does this issue with miscalling doublets occur just with the AnyDoublet calls (ie not with the MarjoritySinglet)?
For these droplets that are being called as 'doublets' but have an 'unassigned' for demuxlet, can you tell me what the Demuxlet_DropletType and Demuxlet_Individual_Assignment are?
Can you confirm what singularity image version you're using? I think it's >= 1.0.1 based on the images you sent through but just want to double check.

I previously decided to call 'unassigned' as 'doublets' but more recently have changed the coding so that (at least with the MajoritySinglet method) droplets that are called as singlets but have a combination of individual assignments (or unassigned) from different demultiplexing methods have been called as singlets + unassigned. I'm wondering if I didn't quite carry this coding through for the AnyDoublet classifications.

Hi Drew, sorry for the delay. answers to your questions:

The calling unassigned doublets looks to be isolated to AnyDoublet outputs. I couldn't find any occurences in MajoritySinglet
For Demuxlet unassigned droplets in the MajoritySinglet output the Demuxlet_DropletType and Demuxlet_Individual_Assignment are both "unassigned" and the MajoritySinglet is always "singlet" if the other tools say singlet. For Demuxlet unassigned droplets in the AnyDoublet output the Demuxlet_DropletType and Demuxlet_Individual_Assignment are both "unassigned" and the AnyDoublet columns are both "doublet".
Here is the information on the image version:
Author: = Drew Neavin
Image_version: = 2.0.1
maintainer: Anaconda, Inc
org.label-schema.build-arch: amd64
org.label-schema.build-date: Friday_20_January_2023_0:58:22_UTC
org.label-schema.schema-version: 1.0
org.label-schema.usage.singularity.deffile.bootstrap: docker
org.label-schema.usage.singularity.deffile.from: continuumio/miniconda3
org.label-schema.usage.singularity.version: 3.10.0-dirty
org.opencontainers.image.created: 2022-12-23T15:47:51.857Z
org.opencontainers.image.description: Repository of Docker images created by Continuum Analytics
org.opencontainers.image.licenses:
org.opencontainers.image.revision: e50c6323cd703e0c2df7aa56eb18e05757df2402
org.opencontainers.image.source: https://github.com/ContinuumIO/docker-images
org.opencontainers.image.title: docker-images
org.opencontainers.image.url: https://github.com/ContinuumIO/docker-images
org.opencontainers.image.version: 22.11.1
Thanks!

Hi @changostraw,
I think I had opted for 'doublet' in the droplet type columns because unassigned annotations couldn't be effectively used anyway and so would be remove. However, this is something I can change in future versions since you've pointed out that it would be beneficial. However, I don't think I'll be able to make these changes in the short term (ie next month) as my plate is full with some other projects.
In the meantime, let me know if you'd like some help coming up with some code to update those rows to 'unassigned' in the droplet type column to help with your assessments.
Cheers,
Drew

Thanks @drneavin! I think I have a workaround.
There are some issues with some of my pools for freemuxlet where they are all unassigned.  However, the genotype correlation was pretty obvious. Am I correct in the assumption that there is a cutoff where the genotype correlations have to be => ~ 0.75? If so, is there anyway to lower it? I have some lower value correlations that are still obvious (0.71 for one cluster vs 0.4 for every other cluster). Thanks!

Hi @changostraw,
There is a flag to indicate what correlation is considered high enough for inclusion as a called cluster of cells for a given donor. You can specify it with. There is a different flag for each software so for souporcell, it’s --souporcell_correlation_limit. By default it’s set to 0.7 but can be set to whatever threshold you want.
I would personally be hesitant about the annotating the cluster with the 0.4 correlation unless you think there is a reason for such low correlation (small number of cells, poor genotype data). If there isn’t a clear reason for that low correlation, it suggests to me that the identified cluster likely contains cells from another donor or doublets  or that there is high ambient rna in the data (which you can check with souporcell and vireo as well as other methods like DecontX and CellBender). if you have the genotype, it might be best to use genotype informed methods like vireo or demuxlet depending on your experimental design and the expected reason for the poor correlation between the snp genotypes and inferred snp genotypes from the cells.

thanks @drneavin, sorry I wasn't clear. I have 21 pools of 4 samples each. After running the genotype correlation for freemuxlet many of the freemuxlet clusters are clearly correlated with one genotype, where one cluster was correlated in the low .7's and the other clusters only reaching 0.4's for that genotype. However, when I do the combined output these freemuxlet clusters are not assigned to a genotype, they are instead called a "doublet". It seems the threshold for this is ~0.75. Where freemuxlet clusters that have a correlation value of >0.75 get assigned to the genotype in the combined output, but freemuxlet clusters that have a correlation value of < 0.75 are assigned as "doublet".
For example for one pool freemuxlet results genotype correlations are:
Cluster 1_GT13359     2_GT13375     3_GT13377     4_GT14352
CLUST0  0.5037766821022964	0.5083906675311902	0.4979855623303216	0.7286447697287014
CLUST1  0.5004675995379368	0.7669771215639491	0.4997817624567396	0.4926245174611682
CLUST2  0.7652384719802461	0.5048069449959758	0.49648566942231986     0.4919653643516229
CLUST3  0.5008796427485248	0.5098617043175034	0.7496989304110875	0.4934534238166346
In the output files freemuxlet cluster 2 and cluster 1 are assigned to genotypes 1_GT13359 and 2_GT13375, but freemuxlet cluster 3 and cluster 0 are assigned as "doublets".
Hopefully that is more clear!
thanks!

Hi @changostraw, you can set this in the Assign_Indiv_by_Geno.R but you can also select the threshold in the Combine_Results.R script. For example, the parameter for souporcell would be -k SOUPORCELL_CORRELATION_LIMIT. The default is 0.7 so you could increase it to 0.75.
Cheers!
-Drew


Error creating machine: Maximum number of retries (5) exceeded

I can get Kitematic working with Windows 10 build 10547 (the last one at the momoent) I can get the VM 'default' working and Running but no way Kitematic connect to it. When I try the configuration, this error comes up:
"Error creating machine: Maximum number of retries (5) exceeded You will want to check the provider to make sure the machine and associated resources were properly removed."

I try with DockerToolbox 1.8.2c and then try to update the Virtualbox to latest testbuilds and stable version but no luck.
Any idea about this problem?

Ok, Finally I achieve a solution to my problem and I want it to post it here. Maybe this could help other person with the same problem. I've been facing different issues but the most important one was this: Somehow the Virtualbox installation and the VMWare Workstation installation was making the Virtualbox network configuration go wrong.
So, first I uninstalled the VMWare Workstation. Then I realize the latest Windows 10 build is causing that the Virtualbox Host Only adapter, is not checking the "Virtualbox NDIS6 Bridged Networking Driver" so the default machine cannot start properly.
With these two tips, I got my kitematic working!

Glad this is now working as expected for you - Thanks for the feedback/how-to


Ensure onPicked only calls if completeSelection been called.

Many thanks to your work, but i'v found the code below
void updateSelection(List<MediaViewModel> selectedMediaList) {
    if (widget.mediaCount == MediaCount.multiple) {
      setState(() {
        _selectedMedia = selectedMediaList;
      });
    } else if (selectedMediaList.length == 1) {
      widget.onDone(selectedMediaList);
    }
  }

When user selected a photo, it will auto call onPicked callback, its weird, user should to confirm their choice.
Can you update the logic to ensure the onPicked callback only calls if completeSelection called.

Many thanks to your work, but i'v found the code below
void updateSelection(List<MediaViewModel> selectedMediaList) {
    if (widget.mediaCount == MediaCount.multiple) {
      setState(() {
        _selectedMedia = selectedMediaList;
      });
    } else if (selectedMediaList.length == 1) {
      widget.onDone(selectedMediaList);
    }
  }

When user selected a photo, it will auto call onPicked callback, its weird, user should to confirm their choice.
Can you update the logic to ensure the onPicked callback only calls if completeSelection called.

@laeo , Thank you so much for the appreciation.
Please let me be clear about the issue. Are you talking about single selection mode?
Usually users just select the picture and it's done.
Maybe you have a requirement where user can change the mode from single to multiple. I would like to ask isn't multiple picking mode solving your issue?

Many thanks to your work, but i'v found the code below
void updateSelection(List<MediaViewModel> selectedMediaList) {
    if (widget.mediaCount == MediaCount.multiple) {
      setState(() {
        _selectedMedia = selectedMediaList;
      });
    } else if (selectedMediaList.length == 1) {
      widget.onDone(selectedMediaList);
    }
  }

When user selected a photo, it will auto call onPicked callback, its weird, user should to confirm their choice.
Can you update the logic to ensure the onPicked callback only calls if completeSelection called.

@laeo , Thank you so much for the appreciation. Please let me be clear about the issue. Are you talking about single selection mode? Usually users just select the picture and it's done. Maybe you have a requirement where user can change the mode from single to multiple. I would like to ask isn't multiple picking mode solving your issue?

Yes, i'm talking about single selection mode.
I think it may better to use completeSelection callback to control when the selection is done. Force call onDone() when user just selected a photo is not a good idea.
Maybe provide completeSelection to onPicking() callback is more better way.
MediaPicker(
  onPicking: (List<Media> selected, VoidCallback completeSelection) {
    if (selected.length == 1) completeSelection();
  },
);


Bump version for fqn checker fix

We needed to fix hidden files checking in FQN checker tool.

@dmdashenkov PTAL


Compatibility matrix page - broken markdown link in "Version 17 compatibility" section

The last row of the table within "Version 17 compatibility" section contains a cell value that looks like:
[19.7](https://support.microsoft.com/help/5014490|20.1|
It seems like there is a missing ')' symbol at the end of the link, which also ruins the next cell (20.1).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4c8868c4-fb29-4b1f-e8ba-7560642481fc
Version Independent ID: e838d8a4-07b9-02a8-f0b5-756220b3e64d
Content: Business Central compatibility matrix - Business Central
Content Source: dev-itpro/upgrade/upgrade-v14-v15-compatibility.md
Service: dynamics365-business-central
GitHub Login: @jswymer
Microsoft Alias: jswymer

We've fixed the article and the update will be published soon. Thanks for the feedback!


Upload photo fails when image is uploaded from a folder other than camera.

When i try to upload a pic from a folder other than camera, say downloads or whatsapp image upload fails. Not able to figure out whats wrong. Camera folder images gets uploaded as usual.

Tested it multiple times...photo upload from whatsapp folder or download folder fails.

In another test, When i take one pic from downloads folder and other pic from camera folder, the upload is successful. Strange, but ..

Can you provide some recording of this issue?
Folder permissions on Android are a mystery to me, I have to tell you that.

Can you provide some recording of this issue?
File try to add one..

@viv3kk any luck with the recording of the issue?

any luck with the recording of the issue?
https://www.youtube.com/watch?v=EmBcPuvOVug&feature=youtu.be

Hmmm I see.
Do you have this app in a public repo, so I can install it and see if I get the same behavior on my phone?
I'm asking this, because I have to make sure that it's not something related to other permissions that you might have set in your app, or if it's related to your device.

no i dont have a public repo..

@viv3kk is this also related to your rooted phone? If so, can I close this issue?

Closing issue due to inactivity.


No close button when using Navigator in full-screen dialog route

Steps to Reproduce
I want to use a navigator in a modal route, but when I do it like in the minimal example code below, there is no close button shown on the app bar:

import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      // theme: ThemeData.dark(),
      home: Scaffold(
        appBar: AppBar(),
        body: Builder(
          builder: (context) {
            return SafeArea(
              child: Column(
                children: <Widget>[
                  InkWell(
                    onTap: () => _pushModalWidget(context),
                    child: Text('Modal widget'),
                  ),
                  InkWell(
                    onTap: () => _pushModalNavigator(context),
                    child: Text('Modal navigator'),
                  ),
                ],
                crossAxisAlignment: CrossAxisAlignment.center,
              ),
            );
          },
        ),
      ),
    );
  }

  void _pushModalWidget(BuildContext context) {
    Navigator.of(context).push(
      MaterialPageRoute(
        fullscreenDialog: true,
        builder: (context) => Scaffold(
              appBar: AppBar(),
              body: Text('Modal widget'),
            ),
      ),
    );
  }

  void _pushModalNavigator(BuildContext context) {
    Navigator.of(context).push(
      MaterialPageRoute(
        fullscreenDialog: true,
        builder: (context) => Navigator(
              onGenerateRoute: (routeSettings) {
                return MaterialPageRoute(
                  fullscreenDialog: true,
                  builder: (context) {
                    return Scaffold(
                      appBar: AppBar(),
                      body: Text('Modal navigator'),
                    );
                  },
                );
              },
            ),
      ),
    );
  }
}

Analyzing flutter_playground...
No issues found! (ran in 1.8s)

flutter doctor -v
[✓] Flutter (Channel stable, v1.5.4-hotfix.2, on Mac OS X 10.14.4 18E226, locale en-DE)
    • Flutter version 1.5.4-hotfix.2 at /Users/timm-l1-ay/Downloads/flutter
    • Framework revision 7a4c33425d (3 weeks ago), 2019-04-29 11:05:24 -0700
    • Engine revision 52c7a1e849
    • Dart version 2.3.0 (build 2.3.0-dev.0.5 a1668566e5)

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/setup/#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, set ANDROID_HOME to that location.
      You may also want to add it to your PATH environment variable.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.2.1, Build version 10E1001
    • ios-deploy 1.9.4
    • CocoaPods version 1.6.1

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/setup/#android-setup for detailed instructions).

[!] IntelliJ IDEA Ultimate Edition (version 2018.2.6)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.dev/intellij-setup/#installing-the-plugins

[✓] VS Code
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.0.2

[✓] Connected device (1 available)
    • iPhone Xs • 44C9099F-E7CD-42A7-A76D-4AFFE8100B39 • ios •
      com.apple.CoreSimulator.SimRuntime.iOS-12-2 (simulator)

! Doctor found issues in 3 categories.

Haven't had a chance to look at this carefully however it looks like Scaffold whose app bar isn't showing a back button isn't showing a back button because there's no route to pop. In this case you don't want the Scaffold's Navigator to pop its route, but the enclosing Navigator to do so. Maybe add a BackButton to the AppBar that DTRT.

@HansMuller You're right. I didn't think of the fact, that the Scaffold brings its own Navigator.
I changed the _pushModalNavigator to render a close button in the app bar, but for some reason I don't understand yet, the pop() triggers by the close button seems to dismiss all pages, resulting in a black screen:
  void _pushModalNavigator(BuildContext context) {
    Navigator.of(context).push(
      MaterialPageRoute(
        fullscreenDialog: true,
        builder: (context) => Navigator(
              onGenerateRoute: (routeSettings) {
                return MaterialPageRoute(
                  fullscreenDialog: true,
                  builder: (context) {
                    return Scaffold(
                      appBar: AppBar(
                        leading: InkWell(
                          child: Icon(Icons.close),
                          onTap: () {
                            Navigator.of(context).pop();
                          },
                        ),
                      ),
                      body: InkWell(
                        child: Text('Modal navigator'),
                        onTap: () {
                          Navigator.of(context).push(
                            MaterialPageRoute(
                              builder: (context) => Scaffold(
                                    appBar: AppBar(),
                                    body: Text('Modal navigator page 2'),
                                  ),
                            ),
                          );
                        },
                      ),
                    );
                  },
                );
              },
            ),
      ),
    );
  }

Not sure what is going in here, as the same Navigator.of(context) usage works just fine when pushing a second page in the modal dialog.

Your

Navigator.of(context).pop();
Is getting the Navigator starting on this line:
builder: (context) => Navigator(

One way to fix this is to change line 9 to:

builder: (context2) {

Hi @tp
Looks you managed to have a workaround for this
Do you think this issue still valid?
Thank you

Without additional information we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.

Use the root navigator to push a fullscreenDialog page and then pop from it too:
Navigator.of(context, rootNavigator: true)..pop();


Add simple linear script

I wonder if you could look at the perturbation field initialization.

Sorry for the delay. It starts to be in a better shape. I fixed the tests.
Now doc/examples/run_linear_simple.py need to be turned into a proper unittest (in a file tests/test_linear.py). See how it is done in tests/test_slow.py. The test in tests/test_linear.py should be as short as possible (in particular smallest possible size). It's better if the test actually tests some behavior (as with the assert in https://github.com/exabl/snek5000-cbox/blob/main/tests/test_slow.py#L74).

It would be also nice to have doc/examples/run_linear_simple.py running smaller and shorter simulations. It's really too long now.

It would be also nice to have doc/examples/run_linear_simple.py running smaller and shorter simulations. It's really too long now.

I'll do it. I also added the scripts for linear simulation to submit the job on the cluster.

The linear test is too long. I don't know how to skip it with running slow.

It's nearly good :-)
When I try python run_linear_simple.py, I get:
Starting time loop ...

     DT/DTCFL/DTFS/DTINIT   0.120E-01   0.120E-01   0.000E+00   0.500E+00
Step      1, t= 1.1993193E-02, DT= 1.1993193E-02, C=  0.500 0.0000E+00 0.0000E+00
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 0 on node meige8pcpa79 exited on signal 11 (Segmentation fault).

It's nearly good :-)
When I try python run_linear_simple.py, I get:
Starting time loop ...

     DT/DTCFL/DTFS/DTINIT   0.120E-01   0.120E-01   0.000E+00   0.500E+00
Step      1, t= 1.1993193E-02, DT= 1.1993193E-02, C=  0.500 0.0000E+00 0.0000E+00
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 0 on node meige8pcpa79 exited on signal 11 (Segmentation fault).

I don't get this error. Could you please try resources={"nproc": 2} to see if it is related to the available processors?

Should we do something with this PR? I guess we could remove this branch? What do you think @akhoubani ?

Otherwise we need to rebase on master.

Yes, it would be nice to rebase it. I wanted to add scripts for linear sims on the cluster.

@paugier This can be merged.


TextInput dataDetectorTypes PhoneNumber Issue When use multiple numbers on same page

When I use multiple TextInput on the same page with datadetectorTypes and if that is phoneNumber. And when I click on any number then ActionSheet will appear with Three options "Copy", "Send Message", "Add To Contacts" then all others numbers colour is getting changed on the same page.
You can check the below stack link
https://stackoverflow.com/questions/49111058/textinput-datadetectortypes-as-phonenumber-issue
Environment
(Run react-native info in your terminal and paste its contents here.)
Expected Behavior
Number color should not change.
(Write what you thought would happen.)
Actual Behavior
(Write what happened. Include screenshots if needed.)
Steps to Reproduce
1)Use Multiple TextInput with dataDetctorType as phoneNumber
2)After tapping Action sheet will appear.
3)And if you see the other number on the same page there will also change to grey that should not happen.
(Link to Snack, or steps to reproduce.)

this will reproduce the issue:
https://repl.it/@AnandNeema/PleasantBarrenTransversal
https://stackoverflow.com/questions/49111058/textinput-datadetectortypes-as-phonenumber-issue

https://repl.it/@AnandNeema/PleasantBarrenTransversal Go to this scan it using Expo and Long Press on Any number then color of other is also changing. That should not happen specially on ipad if you see .


test: admin.signOut() throws error when signed out

admin.signOut() throws TypeError: Cannot read property 'session' of null when a user is not signed in.  This commit contains documentation and failing unit/integration test updates in order to catch the condition where a user has attempted to sign out without being signed in and, rather than throwing the current error, throw an UnauthenticatedError: Not signed in error, instead.
closes hoodiehq/camp#12

@gr2m Thanks for your guidance.  Corrections made.

LGTM ✨

Great work, thanks @capellini 👏 LGTM

released as v4.1.1


Add SPDM 1.2 CHUNK support.

Missing 1.2 feature, after https://github.com/DMTF/libspdm/pull/477
Since this is optional, recommend to define a new LIBSPDM_ENABLE_CAPABILITY_CHUNK

1/10/2022 - add when we have use case.

I'm starting to look at chunking commands now. I'm interested on your thoughts @steven-bellock @jyao1 on how chunking should be exposed to users of the requester library.
Should requests that allow for chunking automatically fallback to chunk mode on large response errors?
For example, libspdm requester code issues a get_measurements request. If an error with error code "large response" is returned, should the library issue chunk_get requests behind the scenes, without the consumer of libspdm requester knowing? or should the consumer of libspdm requester code be required to issue the chunk_get request?

Good question.
Yes. I do hope it can be done in in common function, without known by each individual command.
For example, get_measurement just knows it gets a response, but it does not know it is a large response or not.

Ok, from the previous response, I will try to hide the chunking from any consumers of libspdm as much as possible. Now another question.
What level of error handling/restarting of a request should we support?
For example, suppose the requester is performing get_measurement and needs a large response.  As the libspdm requester code is processing the chunk_get requests, one of the chunks fails with responder busy. Normally, the requester code retries the request on busy, but I think this would complicate the code significantly.
I was thinking of adding the large message handling into the error handling code path deep in the requester under libspdm_handle_error_response_main. And the retry loop on busy errors is at the highest level of the requester. So handling error busy while handling on error large message may not be clean, unless there are multiple places for retries.
Is it sufficient for the libspdm requester code to fail the entire get_measurements request if any chunks fail and return error back to the consumer of libspdm requester, or does libspdm need to be able to restart at only the chunk that fails and keep trying, without the consumer of the requester knowing about the failure?
Perhaps we can discuss this in the task force meeting.

Sure. It seems good topic.

Another question. If there is a large response stored in the responder waiting for CHUNK_GET, is it an invalid usage model to receive a CHUNK_SEND request before the CHUNK_GETs are all requested? I suppose a bad requester could never ask for the large response after the large response error then issues a CHUNK_SEND and responder should be able to handle this situation.

The specification is silent on if, after the Requester has received the LargeResponse error from the Responder, the Requester has to initiate CHUNK_GET to get the response, or if it can ignore the response and send some other request. For libspdm I think it's fine for the Responder to return UnexpectedRequest if the Requester fails to get all the chunks via CHUNK_GET.


Update: Links properly set

Update: docs, meta, tests
Update: links no longer using blog
Update: CODE_OF_CONDUCT included

Resolves: #24, #25

Fixed both in the last 2 commits :)


Unable to Chat With Villagers

Describe the bug
Unable to chat with villagers when I type in chat. Using the default prefix "$". Followed OpenAI instructions and input my API key. Didn't work. Nothing appears in chat when I try to type something. Not even my own message.
Logs
latest.log
I noticed in the logs there seems to be some issues with the mod. I'm not savvy enough to know exactly what's going on though.
Version information

Game version: 1.21
Mod loader: Fabric
OS version: Windows 10
Mod Version: 1.0

Additional context
Add any other context about the problem here.

I noticed that the error occurred when the mod was trying to generate the personality for a new villager. Please check that there are any candidates of personality.
For instance, consider replacing "personalities": [] in saysth/saysth-config.json with
"personalities": [
    "decisive",
    "confident",
    "motivational",
    "analytical",
    "introspective",
    "creative",
    "empathetic",
    "supportive",
    "nurturing",
    "spontaneous",
    "curious",
    "energetic",
    "diplomatic",
    "harmonious",
    "patient",
    "imaginative",
    "visionary",
    "resourceful",
    "practical",
    "down-to-earth",
    "reliable",
    "charismatic",
    "expressive",
    "entertaining",
    "knowledgeable",
    "curious",
    "detail-oriented",
    "ambitious",
    "inspirational",
    "forward-thinking"
]

and add a comma after that.
See also: https://saysth.ranzeplay.space/docs/config/main

So the final config would be like this:
{
  "personalities": [
    "decisive",
    "confident",
    "motivational",
    "analytical",
    "introspective",
    "creative",
    "empathetic",
    "supportive",
    "nurturing",
    "spontaneous",
    "curious",
    "energetic",
    "diplomatic",
    "harmonious",
    "patient",
    "imaginative",
    "visionary",
    "resourceful",
    "practical",
    "down-to-earth",
    "reliable",
    "charismatic",
    "expressive",
    "entertaining",
    "knowledgeable",
    "curious",
    "detail-oriented",
    "ambitious",
    "inspirational",
    "forward-thinking"
  ],
  "nameCandidates": [],
  "conclusionMessageLimit": 10,
  "villagerChatPrefix": "$",
  "useExistingVillagerName": true,
  "showTimeConsumption": false,
  "timeoutSeconds": 10,
  "apiConfigPlatform": "cloudflare"
}

Also don't forget to fill in the name list :)

Names are generating when I use the prefix... But I am not getting a response even when directly addressing them.
latest.log

I see. It is my fault that I didn't consider all the exceptions. I'll upload a new version to identify the real issue more easily.
Thank you for your feedback and support!

And thank you for the timely responses! I'm quite pleasantly surprised that you got back to me so quickly.

Tried.... Now it's saying I don't have an API key when I clearly do.
latest.log

I redid the api config as it showed and now this comes up.
latest.log
I changed "authCredentials" to "Authorization": Bearer
Then followed that up with my key.

Okay, try to change "authCredentials": "xxx-xxx-xxx", to "authCredentials": "Bearer xxx-xxx-xxx",

Apologies for the delay. I went to bed.
Now it says "The model gpt-4 does not exist or you do not have access to it."

I should have access to this. I even pay for ChatGPT for more advanced versions.

I actually fixed that problem myself! But now I've run into yet ANOTHER problem.
latest.log

I'm working on it, seems like compatibility issue.

Try this mod file built just now

Oh my goodness. It just now told me you responded even though it shows that you responded yesterday. That's odd. I'm so sorry.
It doesn't appear to have worked. I am now getting this. The names won't even load now. It seems to be another compatibility error.
latest.log

That's too strange. I'll continue to working on it tonight.

I reproduced your issue with this archive, which was mentioned here:

Could you please download a zip archive from here? You may check that it is an artifact of my GitHub Actions workflow. It is safe since all the details are public in this repo. Use saysth-fabric-1.0.jar inside to replace the original mod file, do what you did before again, and then upload the log file. Thank you :)

But I could not reproduce the issue in this archive, everything seems fine.

Try this mod file built just now

It's still not working.
latest.log
Is it a compatibility error regarding another mod I have installed...?

It could be. If want to step further, please send me the modpack you are using.

Here you are.
1.21 Fabric.zip

Highly possible that there are some compatibility issues.

Ok, I think I've figured it out.
You should not put all 3 jars in the zip archive to mods folder. Instead, remove the following:

saysth-fabric-1.0-dev-shadow.jar
saysth-fabric-1.0-sources.jar

Still not working.
latest.log

I repacked the modpack, which works on my computer. If still not working on your side, then I may ave no idea.
Link: 1.21.Fabric.zip

Well the last time I did it. I didn't see any real problem in the log. Names came up again, but the villagers just didn't chat back.

Noooooo

Don't worry. I'm just now starting up that last pack you sent me.

  That was a hell of a lot of issues to go through. Did no one else have these issues?

No one sent me the feedback before.
I'll update the mod on Modrinth now.

By the way, you may better change your OpenAI API key now. I'm afraid that the public file might leak your key.

Oh wow. Well, I'm glad to have been able to work with you to fix it! The other guy that made a Villager AI mod never responded to my issues so that's when I came here and you responded damn near instantly lol.

I'm glad that I can help you! Although it cost you so much time.
Sorry for not solving it faster.

Maaaaan you're fine. I'm an admin IRL, so I understand stuff happens.

Okay, so we can finally close the issue happily?

Yes we can!


cram for mm10

Hi, I know cram file supported have been added (https://github.com/KCCG/mity/issues/6) but when I am running mity on cram files for mm10 and getting the following error.
mity call --reference mm10 --prefix CH --out-folder-path CH.cram  --region chrMT:1-16299 --normalise
mity version 0.4.0
Calling mitochondrial variants Running
FreeBayes in sensitive mode
FreeBayes failed: b'ERROR(freebayes): Could not open input BAM files\nBamMultiReader::Open: unable to open all files: \t\n\tunable to open file: CH.cram\n\n'

Hi,
I've never tried this combination (of mm10 and cram), but let's start with the command. can you try:
mity call --reference mm10 --prefix CH --out-folder-path ./CH_results/ --region chrMT:1-16299 --normalise CH.cram

thanks

Thanks, it did work, eventually.


will files in pytorch directory be updated to use manyblock?

will files in pytorch directory be updated to use manyblock? If so, how much longer do you think it will take? If not, then I will have to do it myself. Is there anything different I would have to do in terms of giving weights, conditional input, or the output tensor?

fixed simply by giving 4 as arguement for function wavenet_infer in file wavenet_infer.cu


User guide: add guidance on BQ permissions not included

Document key BQ permissions that users will likely need that are not included, eg. jobs.create

Hi @dmedora could you add more details on this issue? I didn't get what info you suggest adding to the user guide.

While the user can see the TD->BQ mapping to see which TD permissions are not accounted for in BQ, we do not show the other direction. So the idea was just to mention this somewhere, that there are some important BQ permissions that don't exist in the TD world that they will need to add themselves.

Yes, it's a good point. We need to add this to the user guide.


Syntax error at the end of input when using onConflict and empty values

Hello! It seems like internally kysely strips undefined or empty values and that breaks the SQL syntax.
Using this (simulating an upSert):
let data = await kysely.insertInto('table')
          .values({
            unique: 'value
            value1: input.value1,
            value2: input.value2,
          })
          .onConflict(oc => {
              return oc.column('unique')
              .doUpdateSet({
                value1: input.value1,
                value2: input.value2,
              })
          })
          .executeTakeFirst();

If value1 and value2 are undefined, will generate this SQL:
insert into "table" ("table") values ($1) on conflict on column 'unique' do update set
The do update set is incomplete and will trigger the error in the subject. The workaround is simple enough, just making sure that at least one of the empty values is an empty string will generate the right SQL however I think this should be handled by kysely
This works:
let data = await kysely.insertInto('table')
          .values({
            unique: 'value
            value1: input.value1,
            value2: input.value2,
          })
          .onConflict(oc => {
              return oc.column('unique')
              .doUpdateSet({
                value1: input.value1 || '', //See empty value as default
                value2: input.value2,
              })
          })
          .executeTakeFirst();

The LogEvent shows parameters as an empty array

An empty default value is not an acceptable fix. There's nothing a query builder can do here. An ORM could do some opinionated shit, but we can't. Just make sure you write valid queries.

@koskimas I disagree since you are allowing undefined values in the doUpdateSet object, but then stripping them as if they were not allowed in the final query which causes this issue.
I understand what you are saying, that you cannot assume what I meant by the undefined, but then why not forcing the types to not allow it? Otherwise you are indeed making an opinion, which is that you allow undefined values but then don't use them.
In the end, it is your library, you do you, but I was not expecting an undefined value to be stripped and the query is not logged in this case so it took longer to debug. In my use case adding an empty value so the update query has at least one value works and that is enough of a workaround. I just submitted since I thought it could be improved.
Cheers

If we didn't strip undefined values, what would we use?

@koskimas When did I bitch about it? You are reading my tone incorrectly.  All I did was brought up one particular behaviour which I did not believe was either right or was trying to understand how it works.
A simple explanation would have sufficed. It took me longer to understand this behaviour as the log option "query" does not log this particular scenario so it just throws the exception and you cannot even see the query that is missing all the values. The only way to see the full query is to check the LogEvent which has the SQL property.
Sometimes, you can have a full table with all optional columns (like mine) and you just want to pass an entire object without having to validate and sanitise every single property. I could do it, of course, but I just felt like the library can know right before making the on update set that the syntax will be wrong (parameters array is empty) and perhaps throw a more clear warning to the user or an exception we can catch and do our own alternative.
Don't get me wrong, this is a great library and you guys did great work. Just was not expecting all of you to be so defensive about a question. I will stop this thread now. Thanks and good luck.

I'm in the process of porting queries from TypeORM to Kysely and encountered this as well. This could ideally be prevented with typing, but if this is impossible, then I think it'd be better for Kysely itself to throw an error rather than sending the malformed query to the DB. It would be more helpful in understanding what went wrong and avoid the network request.


[release/1.7 backport] alias log package to github.com/containerd/log v0.1.0

This is a partial backport of https://github.com/containerd/containerd/pull/9093, only backporting the first commit (for now)

I also had to update some other dependencies, which I did in separate commits.
vendor: github.com/sirupsen/logrus v1.9.3
full diff: https://github.com/sirupsen/logrus/compare/v1.9.0...v1.9.3
v1.9.3

Fix a potential denial of service vulnerability in logrus.Writer()
that could be triggered by logging text longer than 64kb without newlines
Fix panic in Writer

v1.9.2
Reverts "Fix a potential denial of service vulnerability in logrus.Writer()",
which introduced a regression.
v1.9.1

Fix data race in hooks.test package
Add instructions to use different log levels for local and syslog
Fix a potential denial of service vulnerability in logrus.Writer()
that could be triggered by logging text longer than 64kb without newlines
Use text when shows the logrus output

vendor: github.com/stretchr/testify v1.8.4
full diff: https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4
alias log package to github.com/containerd/log v0.1.0
This "soft" deprecates the package, but keeps the local uses of the package,
which can make backporting this to release-branches easier (we can
still move all uses in those branches as well though).

⚠ Let me know if you think I should include the second commit as well (which marks the package as deprecated); I wasn't 100% sure to include that in a patch release.

@thaJeztah let's avoid the deprecation commit for now

^^ after discussing, I included the deprecation for this (1.7) so that it can act as stepping-stone towards v2.0, but we're not backporting it to the 1.6 branch

@AkihiroSuda @cpuguy83 PTAL


Stage editor has some glitches

Describe your bug here.
So, when using the stage editor, I've encountered multiple glitches
1. Unable to add more than one sprite - After adding one sprite I am unable to add another, the only working option is adding a "solid color" sprite but other than that I can only add one sprite. I actually noticed that someone else had this issue too prior to posting this so I know I'm not the only person experiencing this issue.
2. Save stage button does not work - The save button to save your stage quite literally doesn't work, no matter what i do i can not save my stage, you can click on it, but it doesn't do anything. this renders the stage editor basically useless, I'm unsure if this is an issue made by the Psych devs or a mistake made by me, but if it is just an error with the engine then that is insane because the stage editor is a long-awaited feature for about 3 years.
I hope the dev team continues working on the engine and fixes the issues!
~Hollon
Command Prompt/Terminal logs (if existing)
N/A

Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows
Did you edit anything in this build? If so, mention or summarize your changes.
No scripts or code has been modified or changed in any way, no files have been moved or messed with (Unchanged)

I have this bug too. I can't save stage or add more than one objects to stage. There was no errors in 1.0 Prerelease.

same error here. it happens whenever u have an object selected for some reason. i just deselect the object by switching the compiled assets to something else. hope this helps


Fix ncoords and ndims to support Cartesian2D and Cartesian3D

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.25%. Comparing base (56dc19a) to head (a8339d0).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #118      +/-   ##
==========================================
- Coverage   95.29%   95.25%   -0.04%
==========================================
  Files          27       27
  Lines        1190     1181       -9
==========================================
- Hits         1134     1125       -9
  Misses         56       56

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Google analytics and other javascript files

Hi, I have my-mind running on my server and I see noscript blocking google analytics and other javascript files being loaded from different domains.
Now my problem is that I am not using google analytics, so how is this project using google analytics?
This means that my website is providing data from my visitors to third party and be it only the IP address of my visitors. Now the problem with that is that since it is not my google analytics, I don't have a data processing contract with google regarding those IP addresses (which qualify as personal data under the GDPR).
Transmitting personal data to third parties without consent is illegal under the GDPR. Having a third party receive personal data without a data processing agreement is illegal under the GDPR.
GDPR = https://en.wikipedia.org/wiki/General_Data_Protection_Regulation
And if it is possible to load javascript files locally, that would also help avoid leaking of personal data.

Hi @pozzo-balbi,
thanks for your comment. Your observation is correct; as you can see, the source code includes the Google Tag Manager. This is simply because I wanted to measure traffic for the publicly hosted site at https://my-mind.github.io/.
I assume that the correct solution here would be to remove the GTM code from the code at https://github.com/ondras/my-mind/ and add it specifically for the fork at https://github.com/my-mind/my-mind.github.io.
I am, unfortunately, somewhat out of time right now, leaving for a 3-week vacation in two days. If you wish, feel free to submit this change via PR. I would be able to do that myself once I return, probably in September.


[14x-comb2023] Support ROOT 6.30 and C++20

Exact port of https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/pull/873.
This helps do debug some things in the Higgs combination.

The CI failures are not related to this PR.


Error While running

Hello, I installed this project via pip, but it seems to not be working, peharps a mssing dependency or something. here's the stacktrace
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x7f87a3c381e0>
Traceback (most recent call last):
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/utils/autoreload.py", line 226, in wrapper
    fn(*args, **kwargs)
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 113, in inner_run
    autoreload.raise_last_exception()
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/utils/autoreload.py", line 249, in raise_last_exception
    six.reraise(*_exception)
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/utils/autoreload.py", line 226, in wrapper
    fn(*args, **kwargs)
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/django/apps/config.py", line 90, in create
    module = import_module(entry)
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 985, in _gcd_import
  File "<frozen importlib._bootstrap>", line 968, in _find_and_load
  File "<frozen importlib._bootstrap>", line 957, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/root/PycharmProjects/badgeaas/venv/lib/python3.5/site-packages/admino/__init__.py", line 1, in <module>
    from sites import site
ImportError: No module named 'sites'

@erdem Is there something im missing?

having the same issue!! @othreecodes did something work out for you ??


Selecting particular signature for overloaded functions

OS and language server: MacOS / ccls
How you installed LSP (Package Control or from git?): git

Requesting a feature? I am not sure
Thanks for all of your work on LSP for Sublime Text!
I am trying to use LSP+ccls on MacOS for C++ development.
For overloaded functions, the initial list of functions includes the function names but without signatures. For overloaded functions, there is a further list that appears for me once the function name includes parentheses. This secondary pop-up allows me to browse the available signatures, e.g. "1 of 6 overloads (use the "up" "down" keys to navigate): ...". However, once I get to this menu, I seem to be unable to select a particular signature, just to see what signatures are available.
This feels like a very dumb question... but: I was wondering if there was a way to see the function signature in the initial completion list (e.g., before selecting a particular one), and/or to change the selection once the arguments are initially filled in.

I was wondering if there was a way to see the function signature in the initial completion list (e.g., before selecting a particular one)

The decision to show function parameters is up to the language server. We just display the server's completion items.

change the selection once the arguments are initially filled in.

Not sure what you mean. Is ccls pre-filling all arguments of a functions with a snippet? I haven't used that language server in a while.

However, once I get to this menu, I seem to be unable to select a particular signature, just to see what signatures are available.

The language server protocol mostly mirrors how Visual Studio works. In Visual Studio you also cannot "select" a particular overload, you have to type out each argument separately.
Since you're on a Mac you've probably used Xcode. From what I can remember Xcode works a bit differently in that it shows all overloads in the initial completion widget and then once you "select" a completion it pre-fills the function with a snippet. There is no "signature help" widget in Xcode if I recall correctly.

I was wondering if there was a way to see the function signature in the initial completion list (e.g., before selecting a particular one)

The decision to show function parameters is up to the language server. We just display the server's completion items.

With ccls I see two different completion lists. The first list, which shows up while still typing, lists all of the overloaded function names that match the pattern, but it does not provide any additional information about the actual signatures for the overloaded functions. It seems like once ccls figures out which of the overloaded functions has been selected, LSP will pre-fill the argument list for that particular overloaded function. However, it is sort of luck which function you get, and so if you guess "wrong", you have to go back and delete the pre-filled argument list.
Once the function has been identified, it becomes possible to scroll up/down to see the various function signatures that are available, but at this point the pre-filled argument list from the initial selection seems to be "baked in".

change the selection once the arguments are initially filled in.

Not sure what you mean. Is ccls pre-filling all arguments of a functions with a snippet? I haven't used that language server in a while.

I am not sure that I am doing a good job of describing the behavior, but yes, it will pre-fill based on the signature of the selected overloaded function. However there does not seem to be information available until after the selection is made/the snippet is pre-filled, to find out what the actual signature is.

However, once I get to this menu, I seem to be unable to select a particular signature, just to see what signatures are available.

The language server protocol mostly mirrors how Visual Studio works. In Visual Studio you also cannot "select" a particular overload, you have to type out each argument separately.
Since you're on a Mac you've probably used Xcode. From what I can remember Xcode works a bit differently in that it shows all overloads in the initial completion widget and then once you "select" a completion it pre-fills the function with a snippet. There is no "signature help" widget in Xcode if I recall correctly.

I have been using EasyClangComplete, which presents a list of available overloaded functions with their arguments and then pre-fills snippets corresponding to the particular signature that is suggested.
I wonder if clangd would behave differently. I am using ccls because Homebrew's clangd was having trouble finding standard system headers, and because I was getting weird permission denied errors when trying to use clangd in the location where brew installs it. I may give clangd another try --- I think I know how to get it to work by adding some extra preprocessor directives in the compile_commands.json file.

With clangd, you may use -completion-style=detailed. It should display all overloads in ST's completion widget. The default is -completion-style=bundled.

That sounds like what I am looking for. I will give clangd another try later this week, and see if I can find comparable flags for ccls.

By the way, about the missing headers on macOS: I think I ran into this too before, and documented how to set up LLVM. Running this in the terminal should fix the missing header problems:
# This used to be sufficient
xcode-select --install
# But since 10.14 this is necessary too
open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

With clangd, you may use -completion-style=detailed. It should display all overloads in ST's completion widget. The default is -completion-style=bundled.

At least with clangd 8.0.1, I do not see any difference in the completions for the two different completion styles. The completion widget shows the return type of the function, but not the arguments. FWIW, my project settings look like:
	"settings":
	{
		// ...
		"LSP":
		{
			"clangd":
			{
				"command":
				[
					"/usr/local/bin/clangd",
					"-completion-style=detailed"
				],
				"enabled": true,
			}
		},
		// ...
	}

The settings look good, especially if you can see the extra argument being logged to Sublime's console when it starts clangd.
Interesting thought, letting SignatureHelp update an inserted snippet if the user selects an overload. I wonder if VS Code handles this?
Anyways, please close the issue if it is solved with configuring the language server.

Unfortunately, I don't see any difference in LSP when running with/without clangd -completion-style=detailed --- adding the extra configuration flag does not appear to solve this issue.
It sounds like LSP will use whatever is being provided by clangd/ccls --- is there some way for me to see what is being provided, to see if these flags have any effect on the language server itself?

Yes, there is a log_payloads setting under Preferences: LSP Settings which should write the response to completion requests to the Sublime console (see https://lsp.readthedocs.io/en/latest/troubleshooting/)
There is a chance that the Sublime Text completion UI sees two overloads with the same label and just combines them anyways.

log_payloads indicates that clangd is reporting the full signature for each of the overloaded methods during completion.

Right, so you want to also set
	"prefer_label_over_filter_text": true,

together with -completion-style=detailed in your client config. This will show all overloads in ST's completion widget together with the signatures. And if you select one a snippet is inserted in the buffer that you can tab through.
Personally I prefer using -completion-style=bundled because that flows better with the signature help widget.

with -completion-style=detailed

with -completion-style=bundled

Any follow-up issues or documentation here? Otherwise, let's close :)


[MAINTENANCE] Remove AIRGAP mode from code

Few months ago from now on community meeting it was decided to stop support for AirGap mode. At that time - contributors and maintainers didn't have time to remove all the mentions of this mode of installation and operation from the code.
Currently, existing code around AirGap confuses new members of the community (#2099) and heavily obstructs some other parts of the code. (#2036)
I think that all code and documentation mentioning AirGap mode should be removed, if some of functionality would still be needed - it should be moved under other modules.

#1160


[BUG] Indestructible trees and rocks

Description
Fallen trees, stumps, rocks, shrubs, saplings are indestructible.
Damage registers as I can see the damage done numbers, but the listed terrain is not destroyed.
To Reproduce
Steps to reproduce the behavior:

Install mod
Run game
Attack one of the listed items with an appropriate weapon.

Expected behavior
The listed terrain eventually is destroyed and resources gatherable.

Last update broke my detection of destroyable objects.
I'll update to last version soon !

I'm still getting used to how GitHub works, but as I understand it this bugfix is now complete, correct? Would you be able to upload the patch on Nexus? If not, is there a way for me to download it through GitHub so I can start playing with this mod enabled again?

Hey, I fixed this bug but there are still other problems with the last update so I did not upload it to Nexus yet. You can still download the last version from Github with this dix but you won't be able to rename pins.
I'll try to fix other problems asap

Great, thanks!

I had same issue. glad to see it will get fixed

Does the hotfix work? Can this be merged?


use "calculate_variable" as the only interface

add 1 line in calculate_variable to check if variable is a median variable, then dispatch to calculate_median_variable

renamed to calculate and implemented


Add Github Action for testing on Windows

See https://github.com/Senzing/template-go/issues/90

Pull requests:

https://github.com/Senzing/go-databasing/pull/53


Reading Chromatogram Data

Hello! I'm currently working on extracting chromatogram data from mzDB files using Python and have been looking through your repository for help. Could you suggest a method for accessing the full chromatogram data? We are trying to replicate the results that this paper acheived, for whole regions or large regions.
https://www.sciencedirect.com/science/article/pii/S1535947620332394?via%3Dihub#f4
Thank you very much for your time and help!

Unfortunately this Rust implementation is not feature-complete.
Some commentary first.
The reference implementation is still the mzdb-access Java one.
I started this implementation to make it compliant with other MS data io Rust libraries.
See the mzio repo of the Rusteomics organization.
We are a bit struggling to maintain those libraries, because of the lack of time and contributors, but there might be some mew Rusteomics developments performed next year.
Now, regarding your issue I can work a PR to add the XIC missing feature. Exposing to Python is then very easy thanks to Pyo3.


Code blocks broken

Take any of the examples on codesandbox and add a codeblock like
```js
'use anything';
```

The page would error.
I think now that there's a dedicated Hike component to use in markdown, the user should be allowed to opt in to using codehike to parse regular code blocks.

You are adding the codeblock inside or outside a Hike?
On Tue, Jan 30, 2024, 10:44 AM Bond @.***> wrote:

Take any of the examples on codesandbox
—
Reply to this email directly, view it on GitHub
https://github.com/code-hike/v1/issues/1, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOSWR4YUDX2O4NTO4WRAU3YRC6I7AVCNFSM6AAAAABCQ5JJYSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEYDOMZYGIYTKMI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Outside the Hike component

Yes, currently it assumes there's a Code component defined in ./mdx-components.tsx.
The idea is to make this configurable in the mdx configuration:
const chConfig = {
  parseAllCodeblocksOrSomething: true | false,
}

const withMDX = nextMDX({
  extension: /\.mdx?$/,
  options: {
    remarkPlugins: [[remarkCodeHike, chConfig]],
    recmaPlugins: [recmaCodeHike, chConfig]],
    jsx: true,
  },
})

not sure of the name though, any suggestions?

There's already a skipLanguages option, this could be skipCodeBlocks perhaps. Or, parseOnlyHikeComponents or similar.

codeblocks are opt-in now https://v1.codehike.org/docs/concepts/code


Instance: Fix VM vsock nesting issue

Fixes #11185
The output of the vsock's Addr().String() was incompatible with fmt.Sscanf(addr, "host(%d):%d" pattern when nesting (as the "host" part became "vm").
Rather than account for the various output patterns I just extracted the underlying context ID and port directly using type smuggling.

@monstermunchkin any ideas how this regression got introduce, apparently it was with LXD 5.8, but we were using that same Scanf approach before that.

@monstermunchkin any ideas how this regression got introduce, apparently it was with LXD 5.8, but we were using that same Scanf approach before that.

No idea. As you say, the code itself didn't change.

@tomponline this fixes #11187 btw.

@monstermunchkin any ideas how this regression got introduce, apparently it was with LXD 5.8, but we were using that same Scanf approach before that.

No idea. As you say, the code itself didn't change.

Nor apparently did the output of vsock's Addr.String() function.

@tomponline this fixes #11187 btw.

Thanks, its GH's autocomplete messing with me again.

@tomponline we updated the mdlayher/vsock package in go.mod on 17th Nov, which changed the version from 1.1.1 to 1.2.0. That must be it.

LXD 5.7 works

LXD 5.7 works

That's odd. Could you try 5.7 with an updated go.mod, just to see whether it's caused by any dependency?


Update insights frontend components dependency.

Bug fixes

UI blows up after creating an order

there was an issue with notification component in insights fronted component
updated to new version with fix

Codecov Report

Merging #195 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #195   +/-   ##
=======================================
  Coverage   81.38%   81.38%
=======================================
  Files          88       88
  Lines         849      849
  Branches       77       77
=======================================
  Hits          691      691
  Misses        144      144
  Partials       14       14

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 73f15ce...31606f5. Read the comment docs.


Dependency and file structure

Description
Update project dependencies to better suit the requirements that have been laid out. Resolve Issue #7.
Type of change

[x] New feature (non-breaking change which adds functionality)
[x] This change requires a documentation update

Change Status

[x] Complete, tested, ready to review and merge
[ ] Complete, but not tested (may need new tests)
[ ] Incomplete/work-in-progress, PR is for discussion/feedback

How Has This Been Tested?

[x] Local Dev Build (npm run start)
[x] Netlify automatic release testing

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] My code has been reviewed by at least one peer
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[x] There are no merge conflicts

@arturolei Yeah, that's the idea. I didn't want to full on lock us into a specific file structure until we are all on the same page regarding the potential for full atomic design so I held off on anything more concrete that a component folder which could then be divided up as we see fit. We could have a bunch of sibling directories to hold various pieces of varied complexity (e.g. Atoms, Molecules, & Organisms) then rename views to Templates and have the router effectively handle the final creation of Pages. However, while super nice for organization, the complexity that this system brings may not be welcome so it's up to a vote.


How to achieve fork function?

hi ,i want achieve git fork function base on java language,just like "git remote add upstream" to my remote repository,do you have any example? I did not find it

Note: Please use sites such as http://stackoverflow.com for general questions about JGit usage, not issues in this project. Issues should be used for problems with snippets and suggestions of missing snippets. Snippets from good answers on stackoverflow can then be included here, naturally.


How to resolve the undefined problem?

When i use HelloWorld.vue in my own Program, i get the error in the browser like this: '$ is undefined', 'lucksheet is undefined'.
How to resolve the problem?

Any solution? I meet same error.


[Feature Request] Replace 'JwtClaim' with 'PopClaim' on AquireTokenOptions

Is your feature request related to a problem? Please describe.
Property name JwtClaim is specific to Json Web Tokens (JWT).
Describe the solution you'd like
The intent of the claim is providing proof-of-possession. By renaming the property to PopClaim, it can also be applied to other token types.

In 3.0.1 release


'Expression' expected.

From @marcing on February 28, 2018 11:54
public function test() {     end: }
Label at the end of function generates an error "'Expression' expected".
It is a valid php syntax.
Cheers
Copied from original issue: felixfbecker/php-language-server#613

Possible duplicate of https://github.com/Microsoft/tolerant-php-parser/issues/154


Legends and Labels Support

Need a good and elegant way to handle legends and labels on charts.

In my opinion the reason why it would be useful to have as part of the library is that otherwise a lot of people will end up adding it themselves rewriting the same code independently. Also if you do a good job and you want colours to match what's in the plotting, tooltips to show longer names, options to show the labels in different places, etc. it starts being a non negligible effort.

I agree with @tarelli :+1:

@tarelli - Sounds like you have a good idea of how it might work. Want to code it up and send a PR?

I need this today... Any update whether this is implemented or not??


Prueba pull request

Prueba de pull request con implementaciones ficticias.

In the future, please perform tests like this against your own personal repositories (instead of spamming others).

I'm terribly sorry about it. I'm taking a course about Git and Github and the teacher told me to use your repository for a small test of fork and pull request. He told me to close the pull request fast to avoid problems, but I'm afraid it wasn't enough. I will tell the teacher not to use this repository anymore. I'm sorry.

That explains the number of times we've had this same type of spammy content. :sob:
I would suggest your teacher should create their own repository and have students send their PRs there instead (so that your teacher is the only person receiving the PR spam).

I wrote to the teacher again suggesting him to create his own repository. It's not a difficult task, I don't know why he didn't do that instead of spamming other people's repositories. I hope he will do it before another student takes the course. I apologize again for all the trouble.

Thank you so much for trying to help us communicate with your teacher -- it's really appreciated!

You're welcome, but It was the least I could do.

Hi! Since I already finished the course, and I guess that I won't need the account anymore (I doubt I'll use this tools in the future), I'll delete it soon. I hope the teacher changes the course and nobody uses your repository for tests anymore.


Deprecate redux

Man this was a big one hahahahaha I hope it works

Hey thanks, myself. You've really done an awesome job on this one. But since it's over 7k lines of code changed, I'm just going to trust you that it works. This is the kind of dedication we really love to see here and Coinbot Industries. Keep up the good work!


Update docs for combobox and add debounce example

https://github.com/melt-ui/melt-ui/assets/13283469/d0aa962c-8d47-45fe-a3aa-40879741b070

Doesn't the combobox export a debounce property already? :)


Changes for 2020

For the 2020 season, here are some possible changes to Kotlin For FRC:

Removing conversion from Main.java to Main.kt, in a new season, all teams should be creating brand new projects so this unfortunate conversion can be removed.
On any new release it would be desirable to have the changelog shown so that teams can get updates on any new features. There should also be a way to toggle this off.
Change the repository structure to ensure that the master branch is always clean.
Move to the YEAR.MAJOR.MINOR versioning system.

As of today, the changelog is automatically shown after an update. The toggle is not yet implemented but won't be too hard to get done.

The toggle for the auto-show changelog is complete.

Here is the webpage that details the new branching strategy.
https://nvie.com/posts/a-successful-git-branching-model/


Add original checker to required checker

When wrapping a checker, add an original checker property because it helps with debugging.

Fixed via 9cf8a395512aab490218475b9882448c8f6a9193


Fix some clang-tidy complaints.

This were findings with a newer clang-tidy version we run in the CI.


Filter to only allowed characters

What changed? Why?
Search input magics away disallowed characters now.
Notes to reviewers
How has it been tested?

🟡 Heimdall Review Status

Requirement
Status
More Info

Reviews

🟡
0/1

Denominator calculation

Show calculation

1 if user is bot
0

1 if user is external
0

From .codeflow.yml

1

Additional review requirements

Show calculation

Max
0

0

From CODEOWNERS

0

Global minimum
0

Max

1

1

1 if commit is unverified
0

Sum
1

Review Error for JFrankfurt @ 2024-08-01 15:55:19 UTC
User failed mfa authentication, see go/mfa-help


Classification-based analysis of plate position effect

Quick notes, will update in a day or two
I tried a bunch of things but only two were notable

classification of eGFP (so-called poscon) vs. ORF negcons is very high – this could be useful in debugging
well position prediction is very high, now need to figure out if we can narrow it down to features

Ongoing work in #20

Note: from @alxndrkalinin 's work, we would expect the features responsible for high well position prediction to be nucleus, related to one particular channel. Perhaps he can link that issue here, it's due to either staining or imaging artifact with a strong left to right bias.

I think Anne refers to analysis in this comment: https://github.com/broadinstitute/position-effect-correction/issues/15#issuecomment-1588479454
I haven't made visualizations per channel, but it should be pretty straightforward using this notebook.

Thanks Alex - yes, and I should clarify: what is compelling about the comment he points to is that AFTER cell count adjustment we see nuclei and cell features as still having a left to right gradient, whereas cytoplasm is fairly even left to right (in part 2. Per-position mean feature visualizations). That reminded us that CDoT had reported a problem with one of the stains in the nucleus having a left to right pattern.


Windows8.1 can't apply handler either.

Dear @Dijji
I am a new user of file meta and I think it's so great .It helps me a lot to manage and query my files by customizing my own handler with searching in Windows search at system layer(like Mac OS) .
And I notice that Win10 have defaulted some format handler which can't be changed by file meta as u say in wiki. So I downgrade my system from Win10 to Win8.1 and the problem still.The block formats listed in wiki still can't be manage.Are there any methods to change the default property handler or solve the problem in another way?
Finally appreciate ur great job and hope the problem will be solved soon.

Thank you, it's great that you find File Meta useful.
As far as I know, there is no way around Microsoft's block on replacing certain handlers. However they are doing it, it is buried beyond the reach of ordinary mortals and it is not documented.
I'm fascinated that the issue goes all the way back to Windows 8! Also, to finally have a meaning for "*". I will see if I get the same, and if so, consider allowing it to be set in the Association Manager, though why you would go to the trouble of adding a property to a file extension when it either could not be seen or could not be edited is very unclear.
Dijji

thanks for your responds.
The problem occured when I was going to add Tags for video format(.TS .avi and .mp4) and these formats were listed in https://github.com/Dijji/FileMeta/wiki/Affected-extensions.
.ts and .avi can't be edited in properties or previewed, when I custumized them in customs profile.
But .mp4 works by troggle "" of System.keywords in preview panel(that's how I find the "" means).
lxy0797


dynamic fixed point

test out the gains of this modified data type

The goal is to check the pros and cons of a custom fixed-point representation that has dynamic precision. Can you point to me the example-prjs I should try first?
Thank you

Hi @GiuseppeDiGuglielmo
The 1 layer FC model is here:
https://github.com/hls-fpga-machine-learning/hls4ml/blob/master/keras-to-hls/example-keras-model-files/KERAS_1layer.json
https://github.com/hls-fpga-machine-learning/hls4ml/blob/master/keras-to-hls/example-keras-model-files/KERAS_1layer_weights.h5

@GiuseppeDiGuglielmo Can you please send me your example? I am very interested in your method.
I would like to sort it out before the Xilinx meeting. Thanks a lot!

@zhenbinwu
I will integrate the dfixed type in the hls4ml flow on the layer @nhanvtran  suggested.
My plan is - at the beginning - to use the dfixed with hardcoded NBIT and IBIT. The QoR will be the same as the ap_fixed. I will then figure out if and how to add an additional port to the KERAS_1layer to control the precision of the dfixed at runtime. The integration in my project was easy, but it may take me some time for integrating it in hls4ml. I count to do it at the beginning of the next week (before Xilinx meeting for sure).
This is the code, please feel free to experiment and let us know.
Put this in your headers:
// types.h
#ifndef INC_TYPES_H
#define INC_TYPES_H

#if defined(FIX16)
#include "ap_int.h"
#define NBIT 16
#define IBIT 8
typedef ap_fixed<NBIT, IBIT> FPDATA;
const char PRECISION_STR[] = "Fixed Point 16";

#elif defined(DFIX16)
#include "ap_int.h"
#define NBIT 16
#define IBIT 8
typedef ap_int<NBIT> FPDATA;
const char PRECISION_STR[] = "Dynamic Fixed Point 16";

FPDATA dfix_add(FPDATA a, FPDATA b);
FPDATA dfix_mul(FPDATA a, FPDATA b); // Hardcoded precision. It is the same as ap_fixed.
FPDATA dfix_mul(FPDATA a, FPDATA b, unsigned char ibit); // Dynamic precision.
#endif
#endif  // INC_TYPES_H

The implementation of the add and mul. This is sufficient for a CNN.
// types.c
#include "data_types.h"

#if defined(DFIX16)

FPDATA dfix_add(FPDATA a, FPDATA b) {
#pragma HLS INLINE
    return a + b;
}

// Hardcoded precision. Same QoR of Xilinx ap_fixed<NBIT, IBIT>
FPDATA dfix_mul(FPDATA a, FPDATA b) {
#pragma HLS INLINE
    ap_int<NBIT*2> extended_a = a;
    ap_int<NBIT*2> extended_b = b;
    ap_int<NBIT*2> extended_result = extended_a * extended_b;
    return (extended_result >> (NBIT-IBIT));
}

// Dynamic precision. We can control it from the top module interface or in some inner logic of the module using it.
FPDATA dfix_mul(FPDATA a, FPDATA b, unsigned char ibit) {
#pragma HLS INLINE
    ap_int<NBIT*2> extended_a = a;
    ap_int<NBIT*2> extended_b = b;
    ap_int<NBIT*2> extended_result = extended_a * extended_b;
    return (extended_result >> (NBIT-ibit));
}
#endif

Finally, these are some conversion functions that I use in the testbench (not in the synthesizable code). For test and validation, I usually convert

from float to ap_fixed, and then from ap_fixed to the dfixed (see below)
from dfixed to ap_fixed (see below), and then from ap_fixed to float

#if defined(DFIX16)
void to_dfixed(FPDATA &to, ap_fixed<NBIT, IBIT> from) {
  to = from.range(NBIT-1, 0);
}

void from_dfixed(ap_fixed<NBIT, IBIT> to, FPDATA from) {
  to.range(NBIT-1, 0) = from;
}
#endif

P.S. You may have to adapt a little the code above for your experiments. In particular, this solution does not allow you to control the quantization and overflow mode as ap_fixed does. This is definitely something to discuss about with Xilinx. We can also come up with some custom solutions if you really need a certain type of overflow or quantization.

Thanks for the example! May I ask what does range() do? I can't find its description online for ap_fixed.
I tried to test your code in https://github.com/zhenbinwu/HLSArena/tree/master/DynamicFixed. But I am having problem to create reasonable result.
Dynamic Fixed Point 16: a 1 , b 2
add 3 mul 0 newmul -7.625
Fixed Point 16: a 1 , b
add 3 mul 0 newmul -71.625

where mul is directly from the function and newmul is after conversion.
What am I missing? Thank you very much!

@zhenbinwu,
The functions from_dfixed and to_dfixed convert a fixed-point representation into a variable of ap_int data type and vice versa. I need this conversion to integer because the dfix_mul and dfix_add functions operate on ap_int rather than ap_fixed.
Let's say we have:
ap_fixed<8,4> var_fxd = 1.5;

the bit representation of the value stored in var_fxd is 00011000, where the first four bits (0001) encodes the integer part 1 and the remaining four bits (1000) encode the fractional part (0.5). I am using the range function to return the raw bit representations 00011000 and then assign it to an unsigned for example:
ap_int<8> var_int = var_fxd.range(7,0);

Please, note that ap_int provides a function to_int() but it does only return the integer part of the fixed-point variable.
Let me know if it does make sense.
I will have a look at your code and post an answer here ASAP.

@zhenbinwu ,
I edited a little your code. I have a working example of the "dynamic fixed point" (C-simulation, synthesis, and C-RTL-cosimulation). Can I push it or the HLSArena repository?
Sorry for the delay,
Giuseppe

@GiuseppeDiGuglielmo Thanks for the explanation! The HLSArena is a temp repo for some tests. I added you as collaborator so you can push.

I pushed the code, these are some results for the toy example:
Arch        TargetClk   EstimatedClk    Latency DSPs    FFs     LUTs
FIX32       2.00        1.595           5       4       269     94
DFIX32      2.00        1.595           5       4       253     78
DFIX32-VP   2.00        1.595           6       4       336     503

FIX32 uses ap_fixed<32,16>;
DFIX32 uses the proposed integer representation, but the precision is hard-coded;
DFIX32-VP uses the proposed integer representatio, but with a variable precision ibit=16.

@nhanvtran, @zhenbinwu
A short update regarding the example project in
https://github.com/hls-fpga-machine-learning/hls4ml/tree/master/keras-to-hls
I generated the project with:
python keras-to-hls.py -c keras-config.yml

and momentarily and  for testing purpose I manually edited all of the library dependencies such that the tesbench and my-hls-test/firmware/myproject.cpp could use transparently the custom fixed point.
    layer1_t layer1_out[OUT_HEIGHT_1*OUT_WIDTH_1*N_FILT_1];
    #pragma HLS ARRAY_PARTITION variable=layer1_out complete dim=0
    layer1_t conv2d_layer1_out[OUT_HEIGHT_1][OUT_WIDTH_1][N_FILT_1];
    #pragma HLS ARRAY_PARTITION variable=conv2d_layer1_out complete dim=0
    nnet::conv_2d<input_t, layer1_t, config1>(data, conv2d_layer1_out, w1, b1);
    layer1_t logits1[OUT_HEIGHT_1*OUT_WIDTH_1*N_FILT_1];
    #pragma HLS ARRAY_PARTITION variable=logits1 complete dim=0
    nnet::flatten<layer1_t, OUT_HEIGHT_1, OUT_WIDTH_1, N_FILT_1>(conv2d_layer1_out, logits1);
    nnet::relu<layer1_t, layer1_t, relu_config1>(logits1, layer1_out);

    result_t logits2[N_OUTPUTS];
    #pragma HLS ARRAY_PARTITION variable=logits2 complete dim=0
    nnet::compute_layer<layer1_t, result_t, config2>(layer1_out, logits2, w2, b2);
    nnet::softmax<result_t, result_t, softmax_config2>(logits2, res);

The function conv2d, relu, and compute_layer were easy to support. I have to work a little more on the softmax. I may not have time before this Friday, Sept 7. I hope the example in the HLSArena is sufficient for our discussion.
I will keep you posted.

I found some time and ported the following library functions to the user-defined fixed-point:

nnet::conv_2d
nnet::relu
nnet::compute_layer
nnet::softmax

The experiment configuration file is:
KerasJson: example-keras-model-files/KERAS_conv2d_model.json
KerasH5:   example-keras-model-files/KERAS_conv2d_model_weights.h5
OutputDir: my-hls-test
ProjectName: myproject
XilinxPart: xc7vx690tffg1927-2
ClockPeriod: 5

IOType: io_parallel # options: io_serial/io_parallel
ReuseFactor: 1
DefaultPrecision: ap_fixed<18,8>

I used Vivado 2017.2.
ap_fixed and the custom fixed (with hardcoded precision) provide the same QoR and are functionally equivalent:
Arch        TargetClk   EstimatedClk   Latency DSPs       FFs        LUTs
FIX18_8     5.00        4.33           16      1803       144747     75190
DFIX18_8    5.00        4.33           16      1803       144747     75190

@zhenbinwu,

If we are happy with this, I can refine library code and implement the support into the code generator. At that point we may consider to evaluate the variable precision overhead that we observed in the HLSArena code for DFIX32-FP. Let me know your opinion.
I think we can address the resources blow up for ap_fixed saturation mode by implementing own saturation mode on top of the functions in the HLSArena. What do you think?

@GiuseppeDiGuglielmo  I see. Yeah, I think dynamic fixed point is definitely interesting for hls4ml. With careful design, we can split some operation as ap_int type operation, which can be carried out in FF/LUT. This might help with the implementation design for managing resource.
We have been talking about different ap_fixed type for different layer. Maybe it can be achieved easier with the dynamic-fixed point. Anyway, I think it is an interesting application.

Hi @GiuseppeDiGuglielmo, I implemented the overflow for the from_dfixed() from your dynamic fixed point example. You can find the code here. It should work, but does cause extra resource and latency in my test. Just to let you know to avoid duplicate effort. I will show more on next meeting.

@zhenbinwu, thank you for sharing! I will give it a run before the next meeting, but it looks like you already squeeze it. In case, I will let you know.

I'm a bit worried about timing when using fixed-point. I know that we've used the VHDL fixed-point library and some others, but wouldn't hls4ml need to take in to account the dynamic variation in cycles if using a dynamic fixed point?

@jeinstei sorry for the late reply,
I am not sure what you mean with a dynamic variation in cycles. Given a fixed word size, I dynamic precision should not introduce a variable latency. Can you please make me an example?
We proposed the dynamic fixed point for the situation of a single accelerator (e.g. conv2d) that serves the multiple layers of a DNN (e.g. CNN). The goal is to reduce as much as we can the word size and having a variable precision (integer vs. fractional part).
// Dynamic precision. We can control it from the top module interface or in some inner logic of the module using it.
FPDATA dfix_mul(FPDATA a, FPDATA b, unsigned char ibit) { // ibit is a free variable of the system
#pragma HLS INLINE
    ap_int<NBIT*2> extended_a = a;
    ap_int<NBIT*2> extended_b = b;
    ap_int<NBIT*2> extended_result = extended_a * extended_b;
    return (extended_result >> (NBIT-ibit));
}

You can imagine the life cycle of the system* that use this dynamic precision: the software configures and runs the first layer with a certain precision, then it configures and runs the second layer etc.

The system is heterogeneous because of the combination of FPGA and SW.

In HLS4ML, the entire model (logic and weights) fits the FPGA: "HLS4ML unrolls the model in space" (while the approach above is an "unrolling in time"). Again, HLS4ML is (as far as I have seen) purely in FPGA. Theoretically, each layer may have its own hardcoded precision (ap_fixed) without necessary using the "dynamic fixed point".


add explanation of restic automation

every time i look at restic, i block on this and figured it may be
useful for others
this will at least partly address #533.

Codecov Report

Merging #1245 into master will decrease coverage by 5.56%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1245      +/-   ##
==========================================
- Coverage   52.45%   46.89%   -5.57%
==========================================
  Files         131      131
  Lines       12373    12373
==========================================
- Hits         6490     5802     -688
- Misses       5124     5869     +745
+ Partials      759      702      -57

Impacted Files
Coverage Δ

internal/backend/b2/b2.go
0% <0%> (-79.32%)
:arrow_down:

internal/backend/swift/swift.go
0% <0%> (-73.9%)
:arrow_down:

internal/backend/azure/azure.go
0% <0%> (-61.93%)
:arrow_down:

internal/backend/gs/gs.go
0% <0%> (-59.85%)
:arrow_down:

internal/backend/swift/config.go
34.37% <0%> (-56.25%)
:arrow_down:

internal/archiver/archiver.go
65.36% <0%> (+0.16%)
:arrow_up:

internal/backend/test/tests.go
57.6% <0%> (+0.64%)
:arrow_up:

internal/backend/s3/s3.go
61.03% <0%> (+1.37%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c784a15...7a0b442. Read the comment docs.


Issue Regarding to run project on esp32.

Hello,
Can you please help me to run this OCPP project.
I am new to this one. So, I don't have that much knowledge about OCPP. If it is possible to send step by step instructions, then it will help me.

Hello,
while the project runs fine on ARM32 platforms, it is untested on microcontrollers.
I recommend having a look at the tinygo docs (here's an article specifically about ESP32) to check if/how to run this on an MCU.

Is there any source where we can contact fastly. so I can explain my exact
problem to you.
On Thu, 20 Apr, 2023, 9:43 PM lorenzodonini, @.***>
wrote:

Hello,
while the project runs fine on ARM32 platforms, it is untested on
microcontrollers.
I recommend having a look at the tinygo docs (here's an article
https://medium.com/vacatronics/lets-go-embedded-with-esp32-cb6bb3043bd0
specifically about ESP32) to check if/how to run this on an MCU.
—
Reply to this email directly, view it on GitHub
https://github.com/lorenzodonini/ocpp-go/issues/187#issuecomment-1516603098,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AV6EFOC7HDN2AIJ75WLVKADXCFODBANCNFSM6AAAAAAXCPPR5U
.
You are receiving this because you authored the thread.Message ID:
@.***>


[Gaprindashvili] Add Gaprindashvili translations

Need to figure out why avaliable_languages.json is losing language names

Either the catalog in question doesn't have locale_name translated or there's a problem
in the available-languages task.

Looks like the catalogs don't have locale_name, maybe we do this pr without the update to avaliable_languages.json?

Looks like the catalogs don't have locale_name, maybe we do this pr without the update to avaliable_languages.json?

Right, we cannot have available-languages.json without those pretty locale names.