Generic LaTeX feedback

Hi Kevin, just scrolled through very quickly and found the following $\LaTeX$ quirks.
I'd recommend \usepackage{physics} which gives you many useful formatting tools.

Norms can be achieved with \norm{} instead of \Vert (if using the physics package). Using || is incorrect.
The conditional bar is achieved with \mid and not |.
The KL double bar is obtained with \mathrel{\Vert}.
Derivatives, like df/dx, can be obtained with \dv{f}{x} (see physics package).
The differential dx in an integral is obtained with \dd{x} (see physics package).
\Cref should be used only at the beginning of a sentence. Use \cref within a sentence.

I'll add more when I get to flip through it a bit more.

Is this good feedback? Do you want more of this type?

Hi Alfredo. TBH it's not very actionable - it would take me too ling to make all these changes.
But If you find errors on specific pages, please do share. :)


Protect against NPEs in notifications list

Unfortunately there appears to be potential race with notifications being set before the associated issue has been committed.
This PR adds protection in to the notifications list to log any failures and remove these notifications from the display - with an untranslated flash error.
References #10815 - and prevents the panic but does not completely fix this.
Signed-off-by: Andrew Thornton<EMAIL_ADDRESS>

make lg-tm work


Remove blank line on the top of the page

Should we revert the earlier merge or merge this one ?

Thanks. I think all good now.


Trimester 2 Final Exam

PBL VS FRQ
General Information

The topics are organized in a way that would make the code be most presentable and make the greatest sense
The order is not in the same order as presented in the FRQ but is numbered to allow for easy comparison
The FRQ solutions and reflections are at the bottom of the issue

Question 4: Methods and Control Structures
In the fourth FRQ there was a focus on the use of interfaces and the creation of interfaces that serve as a basic template for all future similar occurrences of a class. We have a similar implementation within our own project representing a similar overall concept that we used an interface to define the basic structure of any crop and the methods that they would typically have, along with soil additives and the methods that they would typically have. In the FRQ, we made an interface for NumberGroup that serves as a template for all future NumberGroup classes such as Range where the shared method contains is overridden. In our code, we can see a similar pattern where SoilAdditive methods are overridden in Glyphosate and Crop methods in Corn in order to accomplish their unique attributes and methods while retaining the overall structure as provided by the interface.
public interface Crops {
    int econValue(int area);
    int growthStage();
    int calculateGrowingSeason();
    int profitValue(int area);
    int waterUsage(int area);
}

public interface SoilAdditives {
    int cost(int area);
    int ecoEffect(int area);
    int growthEffect(int area);
    int profitEffect(int area);
}

Question 2: Classes
In the second FRQ, there was a focus on the definition of classes and the constructor needed to initiate the class, like in HiddenWord class where we defined a constructor to take in the needed String and set it to the instance variable. We have a similar implementation within the Glyphosate class that we used a constructor to initiate the class and set the instance variable of area to perform the calculations to scale cost with the amount used. We can also see this in a general trend throughout other aspects of our project where we define the Inventory class, Corn class, Person class, etc.
// Uses the interface SoilAdditives element to implement the Glyphosate class like question 4
// The definition of the Glyphosate class similar to the concepts of question 2
public class Glyphosate implements SoilAdditives {

    int cost = (int) (Math.random() * 13) + 1; // Glphosate cost's between 1 and 13 dollars per
                                               // acre so we pick a random number in
                                               // this range to get the cost
    int ecoEffect = -10;
    int growthEffect = 10;
    int profitEffect = 2;
    int area;

    public Glyphosate(area) {
        this.area = area;
    }

    @Override
    public int cost(int area) {
        int quantity = area;
        int unitCost = this.cost;
        int totalCost = unitCost * quantity;
        return totalCost;
    }

    @Override
    public int ecoEffect(int area) {
        int quantity = area;
        int unitEcoEffect = this.ecoEffect;
        int totalEcoEffect = unitEcoEffect * quantity;
        return totalEcoEffect;
    }

    @Override
    public int growthEffect(int area) {
        int quantity = area;
        int unitGrowthEffect = this.growthEffect;
        int totalGrowthEffect = unitGrowthEffect * quantity;
        return totalGrowthEffect;
    }

    @Override
    public int profitEffect(int area) {
        int quantity = area;
        int unitProfitEffect = this.profitEffect;
        int totalProfitEffect = unitProfitEffect * quantity;
        return totalProfitEffect;
    }
}

Question 1: 2D Arrays
In question 1 there was a focus on array and 2d array operations where we can use loops to be able to access the various elements from within the array. This was a very common theme regardless of whether the array was 2d or 1d as seen through parts A-C where we used them to perform calculations, like summing up the values in part A or creating the array of row sums in part B, and the nested loops in part C. We have an implementation that also uses this mechanic of dealing with 2d arrays where we create a 2d array of the size growing season with the growing season amount of elements, where we then use nested for loops populate these arrays with values between the growth stage * water usage /2 and growth stage * water usage inclusive, then randomly select an index to create a price, and the shared ability of indexing and accessing the various elements within the 2d array a pattern also seen within the FRQ. This allows us to have an engaging and semi-random experience for the user profits to be able to simulate some of the variability in crop prices which can depend on many factors like moisture content and the amount of crops produced in this game setting.
import java.lang.Math;
import java.util.Random;

// Uses the interface element Crops to implement the Corn class like question 4
public class Corn implements Crops {

    int initialWeek;
    int week;
    int developmentStage;

    public Corn(int initialWeek, int week) {
        this.initialWeek = initialWeek;
        this.week = week;
    }

    @Override
    public int econValue(int area) {

        // creates a matrix of the size of the current week of the growing season then
        // populate it with random values between the growth state * waterusage/2 to
        // growth state * water usage and then pick a random value from the matrix.
        // Use of 2d arrays like question 1
        int growthStage = growthStage();
        int waterUsage = waterUsage(area);
        int growingSeason = calculateGrowingSeason();

        int[][] econValueMatrix = new int[growingSeason][growingSeason];

        Random random = new Random();

        for (int i = 0; i < growingSeason; i++) {
            for (int j = 0; j < growingSeason; j++) {
                int minValue = growthStage * waterUsage / 2;
                int maxValue = growthStage * waterUsage;
                econValueMatrix[i][j] = minValue + random.nextInt(maxValue - minValue + 1);
            }
        }

        int randomValue = econValueMatrix[random.nextInt(growingSeason)][random.nextInt(growingSeason)];
        return randomValue;
    }

    public int calculateGrowingSeason() {
        // Calculate growing season based on current week
        int currentWeek = getCurrentWeek(); // Replace with actual implementation
        int midPoint = 26; // Midpoint of the growing season
        int earlySeasonDebuff = 2; // Debuff for being too early in the season
        int lateSeasonDebuff = 1; // Debuff for being too late in the season

        if (currentWeek < midPoint) {
            return earlySeasonDebuff * (midPoint - currentWeek);
        } else {
            return lateSeasonDebuff * (currentWeek - midPoint);
        }
    }

    private int getCurrentWeek() {
        // Replace with actual implementation to get the current week
        return this.week;
    }

    @Override
    public int growthStage() {
        // get week via request from FE
        double randNum = 2 + (Math.random() * 4); // generates a random number between 2 and 5
        int weekGrowthFactor = (int) (randNum);
        int developmentStage = this.developmentStage;
        int initialWeek = this.initialWeek; // week when corn was placed
        int week = this.week;
        if (week - initialWeek == weekGrowthFactor) {
            developmentStage = developmentStage + 1;
        }
        return developmentStage;
    }

    @Override
    public int profitValue(int area) {
        int waterNeeded = waterUsage(area);
        int waterCost = 10; // 10 dollars allows for 1000 gallons of water
        int totalWaterCost = waterCost * waterNeeded;
        int baseCropProfit = 6134;
        int totalProfit = baseCropProfit - totalWaterCost;
        return totalProfit * area; // 134 dollars profit
    }

    @Override
    public int waterUsage(int area) {
        // 1 unit = 1000 gallons of water
        int waterNeeded = 600;
        return waterNeeded * area;
    }
}

Question 3: Array List
In question 3 there was a focus on arrayList through the SparseArray class which has an arrayList labeled entries which was full of SparseArrayEntry objects where we wanted to be able to operate on this arrayList to find the populated indices of the SparseArray represented and to be able to remove all elements of a given column and adjust the array size and indices of the remaining elements. We have similar properties in our Inventory class where the class contains an ArrayList of either SoilAdditive or Crop objects representing the user's use and quantity of these objects, the use of the ArrayList allows us to similarly adjust the contents of the user's inventory like we did in the problem like adding and removing the elements which would resize their inventory, and not being of a fixed size. From this, we can then also perform operations based on the contents of their inventory and get readouts for their overall statistics from water usage, to total farm growth rate, etc similar to the problem and the ability for us to create the getValueAt() method from the data stored within the array list. Which we then achieve by creating an Inventory object for each user and then editing the Crops and soilAdditives dynamically as they play the game.
import java.util.ArrayList;
import java.util.List;

public class Inventory {
    private List<Crops> cropsList;
    private List<SoilAdditives> soilAdditivesList;
    private int farmArea;
    // Similar to FRQ 3 we have an array list that a player holds in their inventory
    // this also allows us for us to add and change elements as they
    // go through out the game ensuring more flexibility, we are also looping
    // through the various elements and being able to get a more
    // in-depth amount of player stats

    public Inventory(int farmArea) {
        cropsList = new ArrayList<>();
        soilAdditivesList = new ArrayList<>();
        this.farmArea = farmArea;
    }

    public void addCrop(Crops crop) {
        cropsList.add(crop);
    }

    public void removeCrop(Crops crop) {
        cropsList.remove(crop);
    }

    public void addSoilAdditive(SoilAdditives soilAdditive) {
        soilAdditivesList.add(soilAdditive);
    }

    public void removeSoilAdditive(SoilAdditives soilAdditive) {
        soilAdditivesList.remove(soilAdditive);
    }

    public int getEconValue() {
        int totalEconValue = 0;
        for (Crops crop : cropsList) {
            totalEconValue += crop.econValue(farmArea);
        }
        return totalEconValue;
    }

    public int getGrowthStage() {
        int totalGrowthStage = 0;
        for (Crops crop : cropsList) {
            totalGrowthStage += crop.growthStage();
        }
        return totalGrowthStage;
    }

    public int getProfitValue() {
        int totalProfitValue = 0;
        for (Crops crop : cropsList) {
            totalProfitValue += crop.profitValue(farmArea);
        }
        return totalProfitValue;
    }

    public int getWaterUsage() {
        int totalWaterUsage = 0;
        for (Crops crop : cropsList) {
            totalWaterUsage += crop.waterUsage(farmArea);
        }
        return totalWaterUsage;
    }

    public int getCost() {
        int totalCost = 0;
        for (SoilAdditives soilAdditive : soilAdditivesList) {
            totalCost += soilAdditive.cost(farmArea);
        }
        return totalCost;
    }

    public int getEcoEffect() {
        int totalEcoEffect = 0;
        for (SoilAdditives soilAdditive : soilAdditivesList) {
            totalEcoEffect += soilAdditive.ecoEffect(farmArea);
        }
        return totalEcoEffect;
    }

    public int getGrowthEffect() {
        int totalGrowthEffect = 0;
        for (SoilAdditives soilAdditive : soilAdditivesList) {
            totalGrowthEffect += soilAdditive.growthEffect(farmArea);
        }
        return totalGrowthEffect;
    }

    public int getProfitEffect() {
        int totalProfitEffect = 0;
        for (SoilAdditives soilAdditive : soilAdditivesList) {
            totalProfitEffect += soilAdditive.profitEffect(farmArea);
        }
        return totalProfitEffect;
    }

    public int userProfit() {
        return getProfitValue() * getProfitEffect() - getCost();
    }

    public int userEco() {
        return getEconValue() * getEcoEffect();
    }

    public int userGrowth() {
        return getGrowthStage() * getGrowthEffect();
    }

    public int userWater() {
        return getWaterUsage();
    }

}

General Reflections

Overall I was surprised as by how linked the PBL and traditional formats are especially in the control structures questions as this is an overall extension of many similar elements that we have seen from Collections being a common interface that we have implemented in the past. This allowed us to be very familiar with the concept in a less traditional setting.
Another additional benefit of the PBL approach was in Question 2 as we already have a very firm understanding through declaring countless classes and constructors over the numerous different elements we have created and implemented from the mini project to the trimester project and it was a very familiar concept.
2D arrays in general are also a very familiar topic from overall just needing a basic understanding of loops and indexing elements and was in general very easy and a skill very easily developed through PBL.
However, the flaws of PBL really came through in question 3 and dealing with the entries ArrayList and the removeCol method of the SpareArray where I really did struggle through not being familiar with why the ArrayList was behaving the way it was from the .remove() method not working in any kind of loop and a special .removeIf() method needing to used. If we had a more formal explanation of how ArrayLists worked and their nuances maybe this problem would have been less challenging and also is a personal point for me to develop off of.
Another personal topic that I would like to work on is the ability to progress through these topics on paper in a shorter time frame to be able to work on these problems as if I were truly in the AP exam and thus would most probably be best prepared for the exam.
Overall it seems that PBL excels at allowing us to be able to apply or implement many concepts but at times it feels as if the understanding under the hood is lacking at times to how things function and this is where I think some more instruction would be helpful. This would allow us to have a more cohesive understanding of why things don't work why they don't work instead of just trying to solve them.

Solutions to FRQ's + FRQ Reflections

The reflections for each FRQ and its parts are in their respective blogs

Question

FRQ 1: 2D Arrays

FRQ 2: Classes

FRQ 3: Array Lists

FRQ 4: Methods And Control Structures

I like that you organized things in order, Q4 followed by Q2.  This show great understanding and direct match of concepts in your work.
Looks like a perfect usage of interface.
I liked that you used @Overrides to show with perfect clarity that they are defining something either abstract or overriden.
On Q1 I had a little more disappointment in code viewing I would have added JavaScript that is rendering growing season to show the growing season.
Q2 shows great definitions of List with new ArrayList implementations in constructor.

I hear you on instruction parts, but there are many unintended consequences of that path.
For instance, "point focus", "copying" as work is same, "answer focus" vs "learning focus".   In truth, knowing that delete in middle of an ArrayList impact iteration is really understood when you apply it or get a tricky FRQ.   No one learns this from lecture.    I believe even in testing there is a method key for ArrayList, but we need to look it up.
We will have some paper FRQ days over the next 8 weeks.  But, I won't be grading them.

FRQ Grading
FRQ 1

Very Similar to my solution to this problem
A) Great understanding of for loops as an iterative structure to sum up the elements of an array.
B) Again this problem was very similar to part A, but with the added complexity of an additional dimension. But you did everything correctly, demonstrating a solid understanding of how to traverse the dimensions of 2D Arrays.
C) Your code was great, building off previous methods you had coded, and comparing the different sums of each of the rows of the 2D Array. I like that you made a boolean variable to hold whether a 2D Array was diverse or not.
Grade: 1.0/0.9

FRQ 2

I liked how you modified the passed hiddenWord in the constructor to ensure that all spaces were done away with and all letters were capitalized
The class building is great, you have encapsulated the HiddenWord attribute, and have added a constructor and a getter.
Your approach is very interesting with the character array and pre filling it out with the *s, and then changing the char array as you build the hint.
Grade: 1.0/0.9

FRQ 3

Really interesting approach to this problem.
Building your SparseArray insde the class itself specifically was pretty interesting
You demonstrated great knowledge of working with arrays, and you also were great with class structures (I felt that this problem required a lot of class design).
Grade 1.0/0.9

FRQ 4

A) Pretty simple here. Great interface.
B) Your implementation of the Range is great. And the contains method is correctly overriden and correctly coded. Great work.
C) Great work with MultipleGroups. It really shows the power of an interface, its so versatile, and you are not locked in to one type of data object.
Grade 1.0/0.9

Question 4: Methods and Control Structures

Comment: Your comparison of the FRQ's focus on interfaces with your project's implementation is well-articulated. The use of interfaces to define common functionalities in different classes, such as Crops and SoilAdditives, is a smart design choice. It shows a clear understanding of interface usage in object-oriented programming.
Consideration: While your approach is robust, consider exploring more about how interfaces can enhance polymorphism in your project. This might give you a deeper understanding of designing flexible and scalable systems.

Question 2

Comment: The explanation of class definitions and constructors is comprehensive. The Glyphosate class demonstrates a clear understanding of how to instantiate objects and set their properties. It’s good to see how concepts from the FRQ have been practically applied in your project.
Consideration: Think about discussing the principles of encapsulation and how they could be applied to your class design. For instance, consider whether any class variables should be private and accessed via getter and setter methods.

Question 1

Comment: Your use of 2D arrays in the Corn class is a great example of handling multi-dimensional data. The implementation of random pricing within a matrix structure is innovative and adds complexity to your project.
Consideration: It might be beneficial to explore error handling for array operations. Consider what happens if the array indices go out of bounds, or if the array is not initialized correctly.

Question 3

Comment: The use of ArrayList in your Inventory class is well executed. It shows a good understanding of dynamic data structures and how they can be manipulated for various operations like adding and removing items.
Consideration: Reflect on the efficiency of your ArrayList operations. Consider the time complexity of various operations and whether there are any performance implications, especially as the size of your inventory grows.

FRQ Grading
FRQ 1

Very Similar to my solution to this problem
A) Great understanding of for loops as an iterative structure to sum up the elements of an array.
B) Again this problem was very similar to part A, but with the added complexity of an additional dimension. But you did everything correctly, demonstrating a solid understanding of how to traverse the dimensions of 2D Arrays.
C) Your code was great, building off previous methods you had coded, and comparing the different sums of each of the rows of the 2D Array. I like that you made a boolean variable to hold whether a 2D Array was diverse or not.
Grade: 1.0/0.9

FRQ 2

I liked how you modified the passed hiddenWord in the constructor to ensure that all spaces were done away with and all letters were capitalized
The class building is great, you have encapsulated the HiddenWord attribute, and have added a constructor and a getter.
Your approach is very interesting with the character array and pre filling it out with the *s, and then changing the char array as you build the hint.
Grade: 1.0/0.9

FRQ 3

Really interesting approach to this problem.
Building your SparseArray insde the class itself specifically was pretty interesting
You demonstrated great knowledge of working with arrays, and you also were great with class structures (I felt that this problem required a lot of class design).
Grade 1.0/0.9

FRQ 4

A) Pretty simple here. Great interface.
B) Your implementation of the Range is great. And the contains method is correctly overriden and correctly coded. Great work.
C) Great work with MultipleGroups. It really shows the power of an interface, its so versatile, and you are not locked in to one type of data object.
Grade 1.0/0.9

Very thoughtful assessment.  I see you pulling yourself into review, feels like you are investing in this review.

Question 4: Methods and Control Structures

Comment: Your comparison of the FRQ's focus on interfaces with your project's implementation is well-articulated. The use of interfaces to define common functionalities in different classes, such as Crops and SoilAdditives, is a smart design choice. It shows a clear understanding of interface usage in object-oriented programming.
Consideration: While your approach is robust, consider exploring more about how interfaces can enhance polymorphism in your project. This might give you a deeper understanding of designing flexible and scalable systems.
Grade 1.0/0.9

Question 2

Comment: The explanation of class definitions and constructors is comprehensive. The Glyphosate class demonstrates a clear understanding of how to instantiate objects and set their properties. It’s good to see how concepts from the FRQ have been practically applied in your project.
Consideration: Think about discussing the principles of encapsulation and how they could be applied to your class design. For instance, consider whether any class variables should be private and accessed via getter and setter methods.
Grade 1.0/0.9

Question 1

Comment: Your use of 2D arrays in the Corn class is a great example of handling multi-dimensional data. The implementation of random pricing within a matrix structure is innovative and adds complexity to your project.
Consideration: It might be beneficial to explore error handling for array operations. Consider what happens if the array indices go out of bounds, or if the array is not initialized correctly.
Grade 1.0/0.9

Question 3

Comment: The use of ArrayList in your Inventory class is well executed. It shows a good understanding of dynamic data structures and how they can be manipulated for various operations like adding and removing items.
Consideration: Reflect on the efficiency of your ArrayList operations. Consider the time complexity of various operations and whether there are any performance implications, especially as the size of your inventory grows.
Grade 1.0/0.9

This is very powerful and classic glows and grows writeup.   This gives me ultimate confidence grader is do depth analysis and really interacting with person they are grading.


Clarify UniFi OS config instructions and add manual update commands

This is based on @philsward helpful feedback in #42.

@willswire would you be able to merge? There seems to be more popularity with this project due to, I believe, the end of Google Domains

LGTM!


Leaflet.Hash disables mouse zooming

As long as Leaflet.Hash is loaded, mouse scrolling will yield:
mapzen.com/tangram/tangram.min.js:10 Uncaught TypeError: e._move is not a function

How to proceed from here?

Does Tangram need an update to be compatible with https://github.com/mlevans/leaflet-hash
or do we need to send a PR to https://github.com/mlevans/leaflet-hash to make it work properly again?

It seems someone is stealing events here.

Thanks for the issue – I'm seeing this error whether leaflet.hash is loaded or not. Tangram was just upgraded to use the new Leaflet v 1.0 beta, and we've been making modifications to the library to accommodate. This will be addressed in the next Tangram release, so I'll move this issue there and rename it.

This issue was moved to tangrams/tangram#199


gpg: add the 2024 pub key

...and remove the old apt-get which is no longer required
Ref https://github.com/scylladb/scylla-pkg/issues/3738

@benipeled we need this also in SCT, right ?

@benipeled we need this also in SCT, right ?

Correct, SCT is on the list to be updated among other places

Verified by https://jenkins.scylladb.com/job/scylla-master/job/releng-testing/job/next-machine-image/265/
Include artifacts-{aws,gce,azure}-tests


[consensus][observability] Capture time as a gauge

Continually capture the time of consensus process via the event loop to ensure that clock skew between validators is reasonable and to find any unusual (possibly byzantine) clock behavior.

/canary

/land


created new files and updated index.html

Created new files and updated index.html

ok

good to go

ok

reviewed ok

ok


How to debug or  How to rewrite the code to start through the main function such as Informer or Autoformer

I want to debug the code and use my own sql data,I change the "Fire(ForecastExperiment)" into "    forcastexp = ForecastExperiment(config_path='storage/experiments/ECL/96M/repeat=0/config.gin')
forcastexp.instance(model_type='deeptime')",but it not work.I would be very grateful if you can  rewrite the code to start through the main function such as Informer or Autoformer.

Hey, thanks for the interest in our work, unfortunately I do not have the bandwidth to do so. You could try copy and pasting the model code into the Autoformer codebase, it should take a similar input format. Any reason why you don't want to just use the code as is?

Maybe the pull request #15 can help to load the trained model. By default, if it was trained, it will load the trained model and run the validate function.


[SPARK-43433][PS] Match GroupBy.nth behavior to the latest Pandas

What changes were proposed in this pull request?
This PR proposes to match GroupBy.nth behavior to the latest Pandas.
Why are the changes needed?
To match the behavior of Pandas 2.0.0 and above.
Does this PR introduce any user-facing change?
Test DataFrame
>>> psdf = ps.DataFrame(
...     {
...         "A": [1, 2, 1, 2],
...         "B": [3.1, 4.1, 4.1, 3.1],
...         "C": ["a", "b", "b", "a"],
...         "D": [True, False, False, True],
...     }
... )
>>> psdf
   A    B  C      D
0  1  3.1  a   True
1  2  4.1  b  False
2  1  4.1  b  False
3  2  3.1  a   True

Before fixing
>>> psdf.groupby("A").nth(-1)
     B  C      D
A
1  4.1  b  False
2  3.1  a   True
>>> psdf.groupby("A")[["C"]].nth(-1)
   C
A
1  b
2  a
>>> psdf.groupby("A")["B"].nth(-1)
A
1    4.1
2    3.1
Name: B, dtype: float64

After fixing
>>> psdf.groupby("A").nth(-1)
   A    B  C      D
2  1  4.1  b  False
3  2  3.1  a   True
>>> psdf.groupby("A")[["C"]].nth(-1)
   C
2  b
3  a
>>> psdf.groupby("A")["B"].nth(-1)
2    4.1
3    3.1
Name: B, dtype: float64

How was this patch tested?
Enabling the existing tests & updating the doctests.
Was this patch authored or co-authored using generative AI tooling?
No.

I don't see any failure on my local testing, but it complains on GitHub CI. Let me take a deeper look why this happens.

Merged to master.


Fabric 1.18 RC3 World Corruption

Chunks go missing where any element of the mod resided.
I know this was an RC, but thought worth mentioning in case it was missed and went live corrupting worlds.
(I have many backups 😄 )
This was tested on
MC 1.18
Fabric Loader 0.12.8
Fabric API 0.44.0
I went though my list of mods testing the cause of this, as soon as TSS was added this occurred, so I am in no doubt this mod causes it.

thanks for resolving :)


[SPARK-43204][SQL] Align MERGE assignments with table attributes

What changes were proposed in this pull request?

This PR extends ResolveRowLevelCommandAssignments to also cover MERGE assignments.
Why are the changes needed?

Similar to SPARK-42151, these changes are needed so that we can rewrite MERGE statements into executable plans for tables that support row-level operations. In particular, our row-level mutation framework assumes Spark is responsible for building an updated version of each affected row and that row is passed back to the data source.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

This PR comes with tests.

@cloud-fan @sunchao @viirya @huaxingao @dongjoon-hyun @gengliangwang, this is a follow-up to PR #40308.

thanks, merging to master!

Thanks for reviewing, @cloud-fan!


Dataset.from_generator consistently freezes at ~1000 rows

Describe the bug
Whenever I try to create a dataset which contains images using Dataset.from_generator, it freezes around 996 rows. I suppose it has something to do with memory consumption, but there's more memory available. I
Somehow it worked a few times but mostly this makes the datasets library much more cumbersome to work with because generators are the easiest way to turn an existing dataset into a Hugging Face dataset.
I've let it run in the frozen state for way longer than it can possibly take to load the actual dataset.
Let me know if you have ideas how to resolve it!
Steps to reproduce the bug
from datasets import Dataset
import numpy as np

def gen():
    for row in range(10000):
        yield {"i": np.random.rand(512, 512, 3)}

Dataset.from_generator(gen)
# -> 90% of the time gets stuck around 1000 rows

Expected behavior
Should continue and go through all the examples yielded by the generator, or at least throw an error or somehow communicate what's going on.
Environment info

datasets version: 2.8.0
Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.29
Python version: 3.8.10
PyArrow version: 12.0.1
Pandas version: 1.5.1

By default, we write data to disk (so it can be memory-mapped) every 1000 rows/samples. You can control this with the writer_batch_size parameter. Also, when working with fixed-size arrays, the ArrayXD feature types yield better performance (e.g., in your case, features=datasets.Features({"i": datasets.Array3D(shape=(512,512,3), dtype="float32")}) should be faster).
Our support for multi-dim arrays could be better, and we plan to improve it as part of https://github.com/huggingface/datasets/issues/5272.

By default, we write data to disk (so it can be memory-mapped) every 1000 rows/samples. You can control this with the writer_batch_size parameter. Also, when working with fixed-size arrays, the ArrayXD feature types yield better performance (e.g., in your case, features=datasets.Features({"i": datasets.Array3D(shape=(512,512,3), dtype="float32")}) should be faster).
Our support for multi-dim arrays could be better, and we plan to improve it as part of #5272.

Thanks for the explanation! The Image array was just for demonstration, I use PIL Images in practice. Does that make a difference? What's the best approach for a dataset with PIL Images as rows?

It's best to use the datasets.Image() feature type for PIL images :)


Add recommendations around VMs

For some users with more powerful devices it makes sense to run most of their day-to-day work in a VM. Some of the guide would apply to the secure host OS but some other parts (like homebrew and Browser) would be better for a VM. If you add a section on VMs, also maybe add how to configure e.g. VMWare to have proper guest isolation and how to use the restore to get back to a 'known' good state.

I got it started, but there is a lot more to say on the subject, so don't be shy with pull requests.


add support for siteorigin generated CSS files

siteorigin generates CSS files for certain pages/layouts and tries to write them to disk. could we have wp-stateless capture these files and uplodad them to GCS?

@ewsopp I think this is not finished. Feature requires manual syncing in order CSS files to appear in GCS.

@antonkorotkov Working on it.

[ ] Automated sync is working.
[ ] But removing them is not working yet.


Rename owning team to Vega

Since that's already in use elsewhere

Codecov Report

Merging #11 (017df1f) into main (b2847d3) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #11   +/-   ##
=======================================
  Coverage   69.62%   69.62%
=======================================
  Files           2        2
  Lines          79       79
=======================================
  Hits           55       55
  Misses         23       23
  Partials        1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b2847d3...017df1f. Read the comment docs.


Exploratory Sprint: Create installer for desktop/local environments

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/1107
Originally created by @dmor on 2019-10-09 13:10:37

For users who want to install Meltano locally, we could package up a set of scripts to manage the installation process so they don't have to go through the command line setup process.

[ ]  Write Opportunity Assessment
[ ]  Define which environments to cover
[ ]  Feasibility conversation for overall effort
[ ]  Scope MVC requirements
[ ]  Feasibility conversation for MVC
[ ]  Go/no-go decision and scheduling

View 2 previous comments from the original issue on GitLab


Update www.servernet.in SSL certificate

@Servernet
Now, I Updated the https certificate.. verify it and close this issue.


exportDataAsCsv

Hi,
I 'm using angular 1.4.9 and If I use this function without params
$scope.gridOptions.api.exportDataAsCsv() I got this kind of error

TypeError: Cannot read property 'processHeaderCallback' of undefined

but I If pass empty object {} $scope.gridOptions.api.exportDataAsCsv({}) everythink works out
Thanks
Best Regards
Michal

already fixed in latest, if you want to try the v5 alpha, it is fixed there.


Various RDS, RDS/Cloudformation, RDS/KMS fixes and additions

We need to mock out deploying RDS instances with full disk encryption
and detailed tagging. We also need to be able do deploy these instances
with Cloudformation, and then access them with both boto and boto3.
The following changes were made to support this use case:

Join RDS and RDS2 backends - this makes RDS resources created via
either of the two boto RDS APIs visible to both, more closely
mirroring how AWS works
Fix RDS responses that were returning JSON but should be returning XML
Add mocking of RDS Cloudformation calls
Add mocking of RDS full disk encryption with KMS
Add mocking of RDS DBParameterGroups
Fix mocking of RDS DBSecurityGroupIngress rules
Make mocking of RDS OptionGroupOptions more accurate
Fix mocking of RDS cross-region DB replication
Add tag support to RDS:

DBs
DBSubnetGroups
DBSecurityGroups

@ritsok and @2rs2ts helped with this work!

Coverage decreased (-0.5%) to 93.467% when pulling e2169822b4fb05f961f24e3299431dc4388fcede on mnussbaum:rds_additions_and_fixes into a20906ff155494c4994db9a3faeacb141079363c on spulec:master.

Coverage decreased (-0.6%) to 93.368% when pulling 247698f3288a6e81f70ee8cdeedfcf91f3f854f5 on mnussbaum:rds_additions_and_fixes into b5ff3345bee181f136aaf235428078e2aaa910a1 on spulec:master.

Coverage decreased (-0.6%) to 93.368% when pulling 7b5fbe2c6d69640cdef714372cdbb9e41c780376 on mnussbaum:rds_additions_and_fixes into b5ff3345bee181f136aaf235428078e2aaa910a1 on spulec:master.

I've amended the commit to fix the test failures that were occurring under python 2.6

Awesome, thank you for all of the work on this!

Thanks for merging!


sanitize claim values before passing

rather than just sanitizing for injected headers oidc_util_set_app_info should also sanitize against other malicious HTML/scripting input here:
https://github.com/pingidentity/mod_auth_openidc/blob/master/src/util.c#L1213
to prevent malicious OPs from injecting clams like "name" : "<script alert(\"\");>" etc. which will we displayed by the actual app

thinking a bit more about it and looking at possible implementations I believe it is better to leave it up to the application that receives the claims after all; it would be considered to be part of good application security practice anyway


Use query parameters in GET requests for filtering

currently this is done with json/body params. It is working, because guzzle magically turns body params into query params when doing GET requests, but imho this is not clean and we should not rely on this behaviour.

I have some Integration-Tests that are failing with the changes. It is probably just a thing with my data-constelation but i dont have time right now to check.

I hope i can review this PR in depth next week.

PersonUpdateRequestTest:
Response for this Integration Test:
{
  "message": "Forbidden to edit the field statusId",
  "translatedMessage": "Keine Berechtigung um das Feld statusId zu sehen oder zu editieren",
  "messageKey": "error.forbidden.edit.field",
  "args": {
    "field": "statusId"
  },
  "errors": []
}

[ ] Prevent "statusId" from being sent to updateRequest
[ ] Write the message and translatedMessage to the CTPermissionException for better readability


Snippets are not always on top of the context menu

VSCode Version: 1.2.1
OS Version: Windows 8.1

Steps to Reproduce:

open JS file
type "for" to trigger the for loop snippet or "if" or just "fun" for function

The snippet is not on top and when the "tab" button is pressed, nothing happens.

@aeschli What to do here? cc @jrieken

Add a setting - there is people that don't like snippets, there is people that want snippets more than anything.

Also see #2343

Related issue: https://github.com/Microsoft/vscode/issues/6667
We already know there are people that want snippets on top of everything and other people that want them at the bottom. As I suggested in the related issue, I'm pretty sure the snippets lovers would we just fine if the snippets are shown on top only if the cursor is on a empty line and outside function parenthesis and object literal brackets.

duplicate of #9286


Is there a way to skip the "Select your institution" page?

Is there a way to skip the "Select institution" page? I'm asking because I already know the institution that my client is using and don't want to ask it again.

I tried to pass the "routing_number" in the link token creation, but no luck.

Sorry for the delay in responding! For US/Canada, there is not. Using the Routing Number field like you're doing can help pre-select the institution, and you can also consider using the Returning User Experience (https://plaid.com/docs/link/returning-user/) as another way to pre-populate the institution, but it is not possible to skip the pane altogether

Just a gentle +1 that this is a real issue for many Plaid customers where we already know what institution the user is connecting to, and making them pick it twice sucks. This used to be a feature, and it was removed for some unknown reason.
It is good to hear the routing number can be used to help, but if I already have the ins_ id, there is no reason we can't at least show only results for that ins on the institution select screen (or better yet, skip it entirely).

Moreover the routing number feature is very unreliable and often fails to highlight the bank, even if using Plaid's own supplied routing numbers.

Commenting because this shouldn't be closed and should be supported.  Please make this possible.

Same use case here, we already know the institution_id before opening link (we created our own screen to pick the institution with some extra options). Please let the institution screen to be skipped (or at least have it pre-selected) if an institution id is provided when creating the link token.

@phoenixy1 any plans to support this? thank you

@hugo-advizr no, there are no current plans to add this beyond what is present via e.g. the Returning User Experience.

Would be great if you all could get this fixed - or offer some other alternative that accomplishes the same goal.  Your competitors offer this and it works really great for end users.  Thanks!


Create Gherkin tests for Device Swap API

What type of PR is this?

enhancement/feature
tests

What this PR does / why we need it:
Including test proposal for Device Swap 0.1.0
Which issue(s) this PR fixes:

Fixes #20

file name to follow guidance (comment from @rartych )

@bigludo7 comments solved, and naming changed according to @rartych comments


Cannot download version libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb of libssl

Description
This is related to #7105. It looks like a new version (libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb) was released 5/30/2023 and the older version was removed.
This is a quick fix to https://github.com/actions/runner-images/blob/035dc70f77d4353de9fbacd66be4f0797d4d9f62/images/linux/scripts/installers/sqlpackage.sh and I can certainly create the PR if it would help.
Platforms affected

[ ] Azure DevOps
[ ] GitHub Actions - Standard Runners
[ ] GitHub Actions - Larger Runners

Runner images affected

[ ] Ubuntu 20.04
[X] Ubuntu 22.04
[ ] macOS 11
[ ] macOS 12
[ ] macOS 13
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
I'm running my own
Is it regression?
No
Expected behavior
Expect the installation of sqlpackage.
Actual behavior
Fails to download:
azure-arm.build_vhd: Downloading 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' to '/tmp/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb'...
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 19 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 18 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 17 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 16 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 15 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 14 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 13 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 12 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 11 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 10 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 9 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 8 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 7 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 6 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 5 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 4 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 3 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 2 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 1 attempts left
azure-arm.build_vhd: Error — Either HTTP response code for 'http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.18_amd64.deb' is wrong - '404' or exit code is not 0 - '0'. Waiting 30 seconds before the next attempt, 0 attempts left

Repro steps
Run build for Ubuntu 22.04.

Hello @lukerogers . Thanks for your report, we will take a look.

Should be fixed in main, please sync your local copy.


WIP: Fix scrapers

Fixes the following scrapers that broke:

[x] TXVaccineCountyRace

Couple problems. state_fips was returning a string. County Name is now just County. And there are some rows with "Other" as the county name

[x] TXVaccineCountyAge

Same as TXVaccineCountyRace

[x] NCCountyVaccine

This scraper doesn't exist. It's under NCVaccine. Removed the DAG

[x] NCVaccine

Only problem was trailing whitespace at the end of the county names. Trimmed off whitespace and works

[ ] PennsylvaniaVaccineDemographics

This class isn't exported by scrapers/official/__init__.py

[ ] MinnesotaCountyVaccines
[x] TexasCountyVaccine

Some rows had non-country location_name's. Fixed by removing rows that were not a valid county

[ ] CaliforniaVaccineCounty
[ ] WYCountyVaccinations [disabled on frontend]
[ ] WYStateVaccinations
[ ] GeorgiaCountyVaccine
[x] VirginiaVaccine (https://trello.com/c/QqObNrn8/1054-vaccine-data-dropped-in-va-because-data-stale)

Remote column names changed. Updated code to use new column names

[x] ArizonaVaccineCounty

County columns had Tribes as one of the counties. Fixed by removing all rows with location_name == 'Tribes

[ ] CovidTrackingProjectDemographics
[x] MissouriVaccineCounty

Column names had changed. Updated column names to match remote data

Looks great - let's ship it!


Autofixture.Xunit issue with .net framework 4.6.2

Hi, I have been using sitecore fakedb for unit tests. I have installed latest release of Sitecore.FakeDb.AutoFixture and Install-Package xunit. when i try to install AutoFixture.Xunit2 I'm getting below issues.
Attempting to gather dependency information for package 'AutoFixture.Xunit<IP_ADDRESS>' with respect to project 'Tests', targeting '.NETFramework,Version=v4.6.2'
Gathering dependency information took 1.45 sec
Attempting to resolve dependencies for package 'AutoFixture.Xunit<IP_ADDRESS>' with DependencyBehavior 'Lowest'
Install-Package : Unable to resolve dependencies.
At line:1 char:1

Install-Package AutoFixture.Xunit2

  + CategoryInfo          : NotSpecified: (:) [Install-Package], Exception
  + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PackageManagement.PowerShellCmdlets.InstallPackageCommand

As far as I can see the Sitecore.FakeDb.AutoFixture is not compatible with AutoFixture 4:
Dependencies:
AutoFixture (>= 3.0.0 && < 4.0.0)
Sitecore.FakeDb (>= 1.7.2)

Therefore, either use AutoFixture 3.x, or ask @sergeyshushlyapin to update the library 😉

Closing this as no actions are required from out side.


Make E2Lib.getOwner CPPI wrapper check entity

E2Lib.getOwner would work "correctly" when CPPI was present, the function was given just a E2 entity instead of an e2context and nothing as target.
Calling E2Lib.getOwner( targetE2ent ) (instead of E2Lib.getOwner( self, entity )) would result in entity and targetE2ent.entity being nil, and due to the line intended to catch the current E2 or the owner, it would correctly return the owner of the targeted E2.
if entity == self.entity or entity == self.player then return self.player end => if nil==nil or ...
Calling E2Lib.getOwner( nonE2ent ) would not work and always return nil. The same would happen if CPPI wasn't present, due to if entity == nil then return end in the original function.
An alternative to "fixing" this "bug" would be to officially support the "simple" E2Lib.getOwner( target ) in all cases (currently only works with CPPI and E2 as target).

imo, adding some checks to make it also support getOwner( any_entity ) would be best, since I have also made this mistake myself several times.


Add support for payload Nans

Add support for payload Nans; both signalling and quiet nans that use the significand field for diagnostic information
examples: NaN12, -NaN13, sNaN19

LGTM also


Fix some 404s

Something went wrong with PR preview build please check

Something went wrong with PR preview build please check

Preview is available here:http://developers-pr439.previews.kube001.services.stellar-ops.com

Preview is available here:http://developers-pr439.previews.kube001.services.stellar-ops.com

Preview is available here:http://developers-pr439.previews.kube001.services.stellar-ops.com

Preview is available here:http://developers-pr439.previews.kube001.services.stellar-ops.com


Add Java compatibility with API of NotBlankString

Close #154.

We will not implement the idea https://github.com/kotools/types/discussions/156#discussioncomment-7347670.


Is it possible to replace by ADMP401 Mic and add a RTC module?

Dear sir, I am an ecologist. I am trying to make a automatic acoustic recorder which I can deployed in the wild. Your project is the best I found. I want to use time as filenames. How can I add a RTC module for it? And how can I replace ADMP441 by ADMP401 Mic? Because ADMP401 with a higher sensitivity. Thank you very much.

Sorry for that, ADMP441 has a higher sensitivity than ADMP401.


Runtime dependency to nonexistent package @azure/functions-core

Hi,
it seems that there is an error in the latest non-beta version (2.5.0), which has a runtime dependency to @azure/functions-core in AzureFunctionsHook. This is breaking our build process, since the package cannot be resolved:
WARNING in ./node_modules/applicationinsights/out/AutoCollection/AzureFunctionsHook.js 51:40-72
Module not found: Error: Can't resolve '@azure/functions-core' in './node_modules/applicationinsights/out/AutoCollection'
 @ ./node_modules/applicationinsights/out/applicationinsights.js 17:27-73
 @ ./src/main.ts 28:33-63

The package doesn't even exist in the public registry in the first place:
$ npm install '@azure/functions-core'
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/@azure%2ffunctions-core - Not found
npm ERR! 404
npm ERR! 404  '@azure/functions-core@*' is not in this registry.

I'm not sure if this is a private package or if it doesn't exist at all, but this change is breaking our webpack build and also our Function App deployment - deploying the (broken) webpack build results in Azure runtime is unavailable and 503 responses.
I think this was introduced in #1070 which was released in 2.4.2, since reverting/pinning our version to 2.4.1 fixed the issue.
If there are any additional details I can provide please let me know.
Version
Node v16 with packages (only relevant ones):
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

@MHarutunian can you share your webpack configuration? this is a package only available in Azure Functions and we added typings file here

Thank you for the quick feedback - I expected that this was some kind of internal package, but searching for it only brought up azure-functions-core-tools. The webpack configuration is pretty basic:
// we use this to resolve path aliases, shouldn't affect the issue at hand I believe
const TsconfigPathsPlugin = require("tsconfig-paths-webpack-plugin");
module.exports = {
  entry: "./src/main.ts",
  target: "node16",
  mode: "production",
  optimization: {
    minimize: false,
    moduleIds: "named",
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        loader: "ts-loader",
      },
    ],
  },
  resolve: {
    extensions: [".ts", ".js"],
    plugins: [
      new TsconfigPathsPlugin({
        extensions: [".ts", ".js"],
      }),
    ],
  },
  output: {
    clean: true,
    library: {
      type: "commonjs-module",
    },
  },
};

What might be interesting is that webpack actually rewrites the line in question to this:
this._functionsCoreModule = __webpack_require__(
  Object(
    (function webpackMissingModule() {
      const e = new Error("Cannot find module '@azure/functions-core'");
      e.code = "MODULE_NOT_FOUND";
      throw e;
    })()
  )
);

Running the Function App locally with func start still works (although the AzureFunctionsHook is probably "disabled" due to the webpack generated error being thrown). Deploying the whole thing to Azure is crashing the entire App though, which I assumed to be because of this require().

@MHarutunian if you are using Azure Functions correlation of telemetry should be handled automatically, custom code were needed before to have it working, more info here, if you have multiple Azure Functions calling each other you should be able to visualize the data correlated correctly when using Application Insights App Map in the portal or similar, I'm not sure about 503 errors, sounds like an error deploying the app to Azure Functions, maybe someone in https://github.com/Azure/azure-functions-nodejs-worker can help with that if you continue to see the error.

I just got back to this now, but it seems it's working as expected. We did have the setup code before, but I removed it with the fix for webpack I mentioned above, and the correlation is working automatically and visible in the Application Map in Azure. Thanks for the additional hints!
I still think it would be worth mentioning in the README somewhere, because bundling Function App code to reduce cold start times is not uncommon, and it wasn't obvious to me what the @azure/functions-core warnings were about. Not sure how other bundlers handle this, but with webpack the aforementioned auto-correlation feature will not work without additional configuration. This is what we ended up using successfully:
externals: ["@azure/functions-core"],

Im using nextjs 14.4.2, I get the same errors - after updating from nextjs 14.1.2. Adding the suggested externals: ["@azure/functions-core"], doesn't seems to solve the issue.
Any ideas? Thanks in advance <3
 ○ Compiling /instrumentation ...
 ⚠<EMAIL_ADDRESS>Module not found: Can't resolve '@azure/functions-core' in '/[PROJECT_FOLDER]/node_modules/@azure/monitor-opentelemetry/dist'

Import trace for requested module:
<EMAIL_ADDRESS>./node_modules/applicationinsights/out/src/main.js
./node_modules/applicationinsights/out/src/index.js

./node_modules/<EMAIL_ADDRESS>Critical dependency: the request of a dependency is an expression

Import trace for requested module:
./node_modules/<EMAIL_ADDRESS>./node_modules/<EMAIL_ADDRESS>./node_modules/<EMAIL_ADDRESS>./node_modules/<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>./node_modules/applicationinsights/out/src/main.js
./node_modules/applicationinsights/out/src/index.js

<EMAIL_ADDRESS>Critical dependency: the request of a dependency is an expression

Import trace for requested module:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>./node_modules/diagnostic-channel-publishers/dist/src/azure-coretracing.pub.js
./node_modules/diagnostic-channel-publishers/dist/src/index.js
./node_modules/applicationinsights/out/src/logs/diagnostic-channel/initialization.js
./node_modules/applicationinsights/out/src/logs/autoCollectLogs.js
./node_modules/applicationinsights/out/src/main.js
./node_modules/applicationinsights/out/src/index.js

<EMAIL_ADDRESS>Module not found: Can't resolve '@opentelemetry/exporter-jaeger' in '/[PROJECT_FOLDER]/node_modules/@opentelemetry/sdk-node/build/src'

Import trace for requested module:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>./node_modules/applicationinsights/out/src/main.js
./node_modules/applicationinsights/out/src/index.js


ComplexTextExtractor caching too much

There seems to be a weird issue with caching (I'm guessing) in the ComplexTextExtractor
Take for instance:
ext = ComplexTextExtractor()
stims = [ComplexTextStim(text='hello'), ComplexTextStim(text='different!')]
res = ext.transform(stims)
[r._data for r in res]
Out[13]: [['hello'], ['hello']]

Basically, it outputs the same word, although more than one was passed in

Yeah this looks like an issue with caching not being content-aware. I suspect this is also an issue with ImageStims initialized from data rather than a file. The only reason it's not an issue for normal TextStims is because we insert the data into the name (which is used for caching).
@tyarkoni thoughts on a solution? we could omit caching when filename is undefined?

Probably the right thing to do would be to hash based on contents rather than filename (or to use both). One thing we could do is, in the base Stim.__hash__ method, instead of automatically including self.filename and self.name in the hash, we could instead include a property called _hash_contents, which would default to those two properties, but could be overwritten in subclasses. For ComplexTextStim, we could then use text instead of name. For ImageStim, we could eventually include a hash of the image data (though I'd be fine with just fixing ComplexTextStim in the short term).

Yeah for what it's worth, I don't think it should be expensive to hash even large stimuli. For example, hashing a video stimulus (Merlin clip) using a blocksize of 65536 bytes (and updating the hash  for every block), only takes a few seconds on my computer.


UnicodeEncodeError: 'charmap' codec can't encode character '\ufffd' in position

Hello, Can you help me?
I'm a G Suite administrator in the company and I have this problem
C:\gam-64>gam print users allfields > users.csv
but their result is
Traceback (most recent call last):
File "gam.py", line 14532, in
File "gam.py", line 14026, in ProcessGAMCommand
File "gam.py", line 11366, in doPrintUsers
File "gam.py", line 11099, in writeCSVfile
File "csv.py", line 158, in writerows
File "c:\python37\lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\ufffd' in position 94: character maps to
[9596] Failed to execute script gam
I have gam version 4.88

I'm getting this too, in Powershell but only when trying to assign it to a variable or pipe it to Export-Csv.
$Courses = gam print courses

Have a similar issue as well.
UnicodeEncodeError: 'charmap' codec can't encode character '\u2004' in position 60: character maps to
[16284] Failed to execute script gam
version 4.89

I am getting this error too when I run
c:\GAM>gam print users allfields > output.csv
GAM is able to count all the users I have but errors out at the point of outputting the results to the csv. I have about 3000 users but it outputs only 73 of them into the csv
See error below
Traceback (most recent call last):
File "gam.py", line 14761, in
File "gam.py", line 14253, in ProcessGAMCommand
File "gam.py", line 11589, in doPrintUsers
File "gam.py", line 11322, in writeCSVfile
File "csv.py", line 158, in writerows
File "c:\python37\lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 42: character maps to
[16696] Failed to execute script gam
I am user gam version GAM 4.90 - https://git.io/gam

Set the following environment variable is whatever method is appropitate to your operating system:
GAM_CHARSET=utf-8
<EMAIL_ADDRESS>
On Aug 13, 2019, at 6:28 AM, coolsammy<EMAIL_ADDRESS>wrote:
I am getting this error too when I run
c:\GAM>gam print users allfields > output.csv
GAM is able to count all the users I have but errors out at the point of outputting the results to the csv. I have about 3000 users but it outputs only 73 of them into the csv
See error below
Traceback (most recent call last):
File "gam.py", line 14761, in
File "gam.py", line 14253, in ProcessGAMCommand
File "gam.py", line 11589, in doPrintUsers
File "gam.py", line 11322, in writeCSVfile
File "csv.py", line 158, in writerows
File "c:\python37\lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 42: character maps to
[16696] Failed to execute script gam
I am user gam version GAM 4.90 - https://git.io/gam https://git.io/gam
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL4ZNOUYABIMKAJRUDTQEKZHBA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FUZ3I#issuecomment-520834285, or mute the thread https://github.com/notifications/unsubscribe-auth/ACCTYLZHUVQL4SXVNT5VKITQEKZHBANCNFSM4H56U5GA.

Set the following environment variable is whatever method is appropriate to your operating system: GAM_CHARSET=utf-8 --<EMAIL_ADDRESS>…
On Aug 13, 2019, at 6:28 AM, coolsammy @.***> wrote: I am getting this error too when I run c:\GAM>gam print users allfields > output.csv GAM is able to count all the users I have but errors out at the point of outputting the results to the csv. I have about 3000 users but it outputs only 73 of them into the csv See error below Traceback (most recent call last): File "gam.py", line 14761, in File "gam.py", line 14253, in ProcessGAMCommand File "gam.py", line 11589, in doPrintUsers File "gam.py", line 11322, in writeCSVfile File "csv.py", line 158, in writerows File "c:\python37\lib\encodings\cp1252.py", line 19, in encode UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 42: character maps to [16696] Failed to execute script gam I am user gam version GAM 4.90 - https://git.io/gam https://git.io/gam — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#965?email_source=notifications&email_token=ACCTYL4ZNOUYABIMKAJRUDTQEKZHBA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FUZ3I#issuecomment-520834285>, or mute the thread https://github.com/notifications/unsubscribe-auth/ACCTYLZHUVQL4SXVNT5VKITQEKZHBANCNFSM4H56U5GA.

How do I set the environment variable for Windows?

http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-gui/
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-command-line-and-registry/
In Command prompt do: set
On Tue, Aug 13, 2019 at 7:46 AM coolsammy<EMAIL_ADDRESS>wrote:

Set the following environment variable is whatever method is appropriate
to your operating system: GAM_CHARSET=utf-8 --<EMAIL_ADDRESS>… <#m_-6776759313345246476_>
On Aug 13, 2019, at 6:28 AM, coolsammy @.***> wrote: I am getting this
error too when I run c:\GAM>gam print users allfields > output.csv GAM is
able to count all the users I have but errors out at the point of
outputting the results to the csv. I have about 3000 users but it outputs
only 73 of them into the csv See error below Traceback (most recent call
last): File "gam.py", line 14761, in File "gam.py", line 14253, in
ProcessGAMCommand File "gam.py", line 11589, in doPrintUsers File "gam.py",
line 11322, in writeCSVfile File "csv.py", line 158, in writerows File
"c:\python37\lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in
position 42: character maps to [16696] Failed to execute script gam I am
user gam version GAM 4.90 - https://git.io/gam https://git.io/gam — You
are receiving this because you are subscribed to this thread. Reply to this
email directly, view it on GitHub <#965
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL4ZNOUYABIMKAJRUDTQEKZHBA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FUZ3I#issuecomment-520834285>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYLZHUVQL4SXVNT5VKITQEKZHBANCNFSM4H56U5GA
.
How do I set the environment variable for Windows?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL3PAVOVBZX44LQY7GLQELCNLA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4F4UBA#issuecomment-520866308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYL6E5AYJYZMWUTZVI63QELCNLANCNFSM4H56U5GA
.

--
Ross Scroggs
<EMAIL_ADDRESS>

Hit send by mistake:
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-gui/
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-command-line-and-registry/
In command prompt do: set GAM_CHARSET=utf-8
Test. If that fixes the problem then set in Control panel
Ross
On Tue, Aug 13, 2019 at 7:46 AM coolsammy<EMAIL_ADDRESS>wrote:

Set the following environment variable is whatever method is appropriate
to your operating system: GAM_CHARSET=utf-8 --<EMAIL_ADDRESS>… <#m_-6776759313345246476_>
On Aug 13, 2019, at 6:28 AM, coolsammy @.***> wrote: I am getting this
error too when I run c:\GAM>gam print users allfields > output.csv GAM is
able to count all the users I have but errors out at the point of
outputting the results to the csv. I have about 3000 users but it outputs
only 73 of them into the csv See error below Traceback (most recent call
last): File "gam.py", line 14761, in File "gam.py", line 14253, in
ProcessGAMCommand File "gam.py", line 11589, in doPrintUsers File "gam.py",
line 11322, in writeCSVfile File "csv.py", line 158, in writerows File
"c:\python37\lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in
position 42: character maps to [16696] Failed to execute script gam I am
user gam version GAM 4.90 - https://git.io/gam https://git.io/gam — You
are receiving this because you are subscribed to this thread. Reply to this
email directly, view it on GitHub <#965
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL4ZNOUYABIMKAJRUDTQEKZHBA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FUZ3I#issuecomment-520834285>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYLZHUVQL4SXVNT5VKITQEKZHBANCNFSM4H56U5GA
.
How do I set the environment variable for Windows?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL3PAVOVBZX44LQY7GLQELCNLA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4F4UBA#issuecomment-520866308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYL6E5AYJYZMWUTZVI63QELCNLANCNFSM4H56U5GA
.

--
Ross Scroggs
<EMAIL_ADDRESS>

Thanks for the suggestion Ross but it still didn't fix it.
On Aug. 13, 2019 10:53 a.m., Ross Scroggs<EMAIL_ADDRESS>wrote:
Hit send by mistake:
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-gui/
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-command-line-and-registry/
In command prompt do: set GAM_CHARSET=utf-8
Test. If that fixes the problem then set in Control panel
Ross
On Tue, Aug 13, 2019 at 7:46 AM coolsammy<EMAIL_ADDRESS>wrote:

Set the following environment variable is whatever method is appropriate
to your operating system: GAM_CHARSET=utf-8 --<EMAIL_ADDRESS>… <#m_-6776759313345246476_>
On Aug 13, 2019, at 6:28 AM, coolsammy @.***> wrote: I am getting this
error too when I run c:\GAM>gam print users allfields > output.csv GAM is
able to count all the users I have but errors out at the point of
outputting the results to the csv. I have about 3000 users but it outputs
only 73 of them into the csv See error below Traceback (most recent call
last): File "gam.py", line 14761, in File "gam.py", line 14253, in
ProcessGAMCommand File "gam.py", line 11589, in doPrintUsers File "gam.py",
line 11322, in writeCSVfile File "csv.py", line 158, in writerows File
"c:\python37\lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in
position 42: character maps to [16696] Failed to execute script gam I am
user gam version GAM 4.90 - https://git.io/gam https://git.io/gam — You
are receiving this because you are subscribed to this thread. Reply to this
email directly, view it on GitHub <#965
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL4ZNOUYABIMKAJRUDTQEKZHBA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FUZ3I#issuecomment-520834285>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYLZHUVQL4SXVNT5VKITQEKZHBANCNFSM4H56U5GA
.
How do I set the environment variable for Windows?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL3PAVOVBZX44LQY7GLQELCNLA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4F4UBA#issuecomment-520866308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYL6E5AYJYZMWUTZVI63QELCNLANCNFSM4H56U5GA
.

--
Ross Scroggs
<EMAIL_ADDRESS>—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=AGB6ZJBOL5RFK6QU24SMSHDQELDFPA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4F5JZQ#issuecomment-520869094, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGB6ZJA5MT5ERB5TDYA7PTLQELDFPANCNFSM4H56U5GA.

Try Advanced Gam (https://github.com/taers232c/GAMADV-XTD3 https://github.com/taers232c/GAMADV-XTD3)
https://github.com/taers232c/GAMADV-XTD3/wiki/How-to-Upgrade-from-Standard-GAM https://github.com/taers232c/GAMADV-XTD3/wiki/How-to-Upgrade-from-Standard-GAM
Ross
<EMAIL_ADDRESS>
On Aug 13, 2019, at 9:13 AM, coolsammy<EMAIL_ADDRESS>wrote:
Thanks for the suggestion Ross but it still didn't fix it.
On Aug. 13, 2019 10:53 a.m., Ross Scroggs<EMAIL_ADDRESS>wrote:
Hit send by mistake:
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-gui/
http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-command-line-and-registry/
In command prompt do: set GAM_CHARSET=utf-8
Test. If that fixes the problem then set in Control panel
Ross
On Tue, Aug 13, 2019 at 7:46 AM coolsammy<EMAIL_ADDRESS>wrote:

Set the following environment variable is whatever method is appropriate
to your operating system: GAM_CHARSET=utf-8 --<EMAIL_ADDRESS>… <#m_-6776759313345246476_>
On Aug 13, 2019, at 6:28 AM, coolsammy @.***> wrote: I am getting this
error too when I run c:\GAM>gam print users allfields > output.csv GAM is
able to count all the users I have but errors out at the point of
outputting the results to the csv. I have about 3000 users but it outputs
only 73 of them into the csv See error below Traceback (most recent call
last): File "gam.py", line 14761, in File "gam.py", line 14253, in
ProcessGAMCommand File "gam.py", line 11589, in doPrintUsers File "gam.py",
line 11322, in writeCSVfile File "csv.py", line 158, in writerows File
"c:\python37\lib\encodings\cp1252.py", line 19, in encode
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in
position 42: character maps to [16696] Failed to execute script gam I am
user gam version GAM 4.90 - https://git.io/gam https://git.io/gam — You
are receiving this because you are subscribed to this thread. Reply to this
email directly, view it on GitHub <#965
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL4ZNOUYABIMKAJRUDTQEKZHBA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4FUZ3I#issuecomment-520834285>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYLZHUVQL4SXVNT5VKITQEKZHBANCNFSM4H56U5GA
.
How do I set the environment variable for Windows?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL3PAVOVBZX44LQY7GLQELCNLA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4F4UBA#issuecomment-520866308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYL6E5AYJYZMWUTZVI63QELCNLANCNFSM4H56U5GA
.

--
Ross Scroggs
<EMAIL_ADDRESS>—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=AGB6ZJBOL5RFK6QU24SMSHDQELDFPA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4F5JZQ#issuecomment-520869094, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGB6ZJA5MT5ERB5TDYA7PTLQELDFPANCNFSM4H56U5GA.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/jay0lee/GAM/issues/965?email_source=notifications&email_token=ACCTYL7SMP3LUJTBANN5MPLQELMTFA5CNFSM4H56U5GKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4GFOLI#issuecomment-520902445, or mute the thread https://github.com/notifications/unsubscribe-auth/ACCTYL3F3IOOVFGVPEIGZETQELMTFANCNFSM4H56U5GA.

Using Advanced GAM resolves the issues and gives more options to filter results.
Thanks for the incredible work you guys have put into this tool. I really appreciate all the effort.
I will love to help out in anyway that I can, even if it is with adding sample commands in the Wiki.
Cheers.


Css Improve in #1170

Project title
CSS Improve in #1170
Project Description
Stack:
And in order to tick the check box just but x inside them for example - [x] like this. Please delete options that are not relevant.

[x] CSS

@gurjeetsinghvirdee assign this issue na


Frequency response of sinc filters

Hi,
We are very interested in the frequency response changing phenomenon for phoneme classification when a narrow band noise is added to the wave. However, we could not reproduce the curves in the paper.
Here is what we achieved.  Even though there is a clear valley from 2k to 2.5k Hz,  a huge decay around 500 Hz always exists, which is different from the figures in your paper. Besides, this 500 Hz valley exists even when no noise is added to the input. And we could not find the reason behind it.

Do you have any idea what might cause this problem? Or is it possible for you to release the frequency response code?
Many thanks!

Hi,
in your case you are doing speech recognition, right? In the first SincNet
paper (https://arxiv.org/abs/1808.00158) you can find results for speaker
recognition only. In a  follow up study (
https://arxiv.org/pdf/1811.09725.pdf) we also did some speech recognition
experiments.  In particular, in Fig. 5 you find the cumulative frequency
response for a phoneme classification with TIMIT (english). Note that in
the figure there is a hole in the cumulative frequency response due to the
noise added in the bandwidth between  2000 and 2500 Hz. We didn't observe
many peaks for speech recognition like for the speaker recognition. We just
allocate more filters in the lower part of the spectrum. Note also that the
filters learned by SincNet might change on different tasks, languages, etc
and this is in part the strength of this approach.
Best,
Mirco
On Mon, 9 Dec 2019 at 08:31, zehaitu<EMAIL_ADDRESS>wrote:

Hi,
We are very interested in the frequency response changing phenomenon for
phoneme classification when a narrow band noise is added to the wave.
However, we could not reproduce the curves in the paper.
Here is what we achieved. Even though there is a clear valley from 2k to
2.5k Hz, a huge decay around 500 Hz always exists, which is different from
the figures in your paper. Besides, this 500 Hz valley exists even when no
noise is added to the input. And we could not find the reason behind it.
[image: frequency response]
https://user-images.githubusercontent.com/57004778/70439282-915e3980-1a87-11ea-9a3b-32cb916931e3.png
Do you have any idea what might cause this problem? Or is it possible for
you to release the frequency response code?
Many thanks!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/79?email_source=notifications&email_token=AEA2ZVUT6AYC6QOHUQJH2GLQXZCEJA5CNFSM4JYJHCF2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H7COD5A,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEA2ZVUV2RETK5EOQ7QFZPDQXZCEJANCNFSM4JYJHCFQ
.

Hi,
Thank you so much for your reply. Currently, I am actually trying to replicate that curve in https://arxiv.org/pdf/1811.09725.pdf, thus I am using TIMIT and the noise is in the the bandwidth between 2000 and 2500 Hz. Other setups like filter length, number of filters are the same as the open source code for SincNet for phoneme classification with in the pytorch-kaldi project. However, I found this strange valley around 500 Hz.
By allocating more filters in the lower part of the spectrum, do you mean artificially or the filters are learnt to move to lower frequency domain?
Many thanks!

Hi,
it is the second one because normally the cumulative frequency response is
higher in the lower frequency domain.
The valley around 500 Hz is quite weird. We observed this kind of
valleys/weird peaks just at the beginning of training, but then they should
gradually disappear. How long have you trained the filters? What happens if
you start the training from scratch?
Best,
Mirco
On Mon, 9 Dec 2019 at 10:36, zehaitu<EMAIL_ADDRESS>wrote:

Hi,
Thank you so much for your reply. Currently, I am actually trying to
replicate that curve in https://arxiv.org/pdf/1811.09725.pdf, thus I am
using TIMIT and the noise is in the the bandwidth between 2000 and 2500 Hz.
Other setups like filter length, number of filters are the same as the open
source code for SincNet for phoneme classification with in the
pytorch-kaldi project. However, I found this strange valley around 500 Hz.
By allocating more filters in the lower part of the spectrum, do you mean
artificially or the filters are learnt to move to lower frequency domain?
Many thanks!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mravanelli/SincNet/issues/79?email_source=notifications&email_token=AEA2ZVWC54BBKFJPKLWLCM3QXZQWVA5CNFSM4JYJHCF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGJTJQI#issuecomment-563295425,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEA2ZVTLM7AAE7WW4FEK7ODQXZQWVANCNFSM4JYJHCFQ
.

Hi Mirco,
This figure is from the model saved after 20th epoch. I checked the models from 5th, 10th 15th and 20th epoch, and they are pretty much the same. And the network tends to overfit after 20 epochs so I did not checked the following models. I have not tried to start training from scratch so far but I guess I should have a try and see what will happen. Thanks for your advice!

Is it possible to plot cumulative frequency plot from learnt filters of 1st convolutional layer where we have passed Mel spectrogram as input feature to the first layer instead of raw waveform?


[Distribution Build Failure] Alerting build failing due to missing org.opensearch:common-utils:<IP_ADDRESS>-SNAPSHOT

Describe the bug
Alerting failed to build because org.opensearch:common-utils:<IP_ADDRESS>-SNAPSHOT was missing, it looks like common-utils was not built with the snapshot postfix.
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':alerting-core:compileKotlin'.
> Could not resolve all files for configuration ':alerting-core:compileClasspath'
....
Searched in the following locations:
  - file:/Users/petern/.m2/repository/org/opensearch/common-utils/<IP_ADDRESS>-SNAPSHOT/maven-metadata.xml
  - file:/Users/petern/.m2/repository/org/opensearch/common-utils/<IP_ADDRESS>-SNAPSHOT/common-utils-<IP_ADDRESS>-SNAPSHOT.pom
  - https://repo.maven.apache.org/maven2/org/opensearch/common-utils/<IP_ADDRESS>-SNAPSHOT/maven-metadata.xml
  - https://repo.maven.apache.org/maven2/org/opensearch/common-utils/<IP_ADDRESS>-SNAPSHOT/common-utils-<IP_ADDRESS>-SNAPSHOT.pom
  - https://jcenter.bintray.com/org/opensearch/common-utils/<IP_ADDRESS>-SNAPSHOT/maven-metadata.xml
  - https://jcenter.bintray.com/org/opensearch/common-utils/<IP_ADDRESS>-SNAPSHOT/common-utils-<IP_ADDRESS>-SNAPSHOT.pom
Required by:
    project :alerting-core
...
2021-09-08 11:31:16 ERROR    Error building alerting, retry with: ./bundle-workflow/build.sh manifests/opensearch-1.1.0.yml --component alerting --snapshot

ls /Users/petern/.m2/repository/org/opensearch/common-utils/<IP_ADDRESS>
common-utils-<IP_ADDRESS>-javadoc.jar
common-utils-<IP_ADDRESS>-sources.jar
common-utils-<IP_ADDRESS>.jar
common-utils-<IP_ADDRESS>.pom

To Reproduce

./bundle-workflow/build.sh manifests/opensearch-1.1.0.yml --component alerting --snapshot

Expected behavior
Build should succeed

Reproduces locally, inspecting the last passing build to see why this worked

@peternied Did you build common-utils first? Otherwise it will fail with the repro command above, which is expected.

Found it, PRs to support snapshots was lost when branch was moved to from main->1.1.  Creating pull requests...

https://github.com/opensearch-project/common-utils/pull/58
https://github.com/opensearch-project/common-utils/pull/57

I am going to back out the branch update to 1.1 and have the common-utils team switch back when its ready.

Snapshot build is passing, waiting on completed build cycle

This issue is confirmed as no longer blocking the build.


Integrating Capistrano support (along with Vagrant for testing the deployment locally)

Resolves #6

This should have the Vagrant configuration removed, as testing this is cumbersome.

Deprecated by https://github.com/pulibrary/ouranos/pull/18


Trying to get in touch regarding a security issue

Hey there!
I'd like to report a security issue but cannot find contact instructions on your repository.
If not a hassle, might you kindly add a SECURITY.md file with an email, or another contact method? GitHub recommends this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.
Thank you for your consideration, and I look forward to hearing from you!
(cc @huntr-helper)

Hi, josh at junon dot me is just fine, thanks for asking!


$param(s) to allow parametrized schemas

Support schemas with parameters, e.g.
{
  id: 'range',
  $params: {
    min: { type: 'number'},
    max: { type: 'number'}
  },
  type: 'integer',
  minimum: { $param: 'min' },
  maximum: { $param: 'min' }
}

Then another schema can refer to the first schema using $ref and "pass" parameters:
{
  type: 'object',
  properties: {
    foo: { $ref: 'range', $params: { min: 0, max: 9 } },
    bar: { $ref: 'range', $params: { min: 10, max: 99 } },
  }
}

So the second schema can create different constraints by using only one range schema.
The example is trivial and using params in such way is not very useful, but $params become useful with more complex schemas when only a small part of the schema changes based on parameters.
If range is compiled on its own it would return a higher order function that can be passed parameters to return a validation function:
var getRangeValidate = ajv.compile(rangeSchema); // first example
var validate = getRangeValidate(0, 9);
validate(0); // true
validate(9); // true
validate(10); // false
validate(-1); // false

The last example you can already do (almost) with $data (by merging data and parameters in the same object and merging schemas for data and parameters too), but it feels clunky and hiding the distinction between data and parameters.
Referring to schemas with params cannot be done though.

The advantage of that solution over any pre-processing/macro step is that the schema would be compiled only once and creating a validating function with different parameters would not require any code-generation so it will be efficient, unlike any pre-processing/macro step.

I understand it a lot better with the example. I think it would be pretty cool feature, can you think of some really solid, real world examples where this would make a lot of difference?

My use case is to be able to use schemas as rules that can be configured, similar to eslint, but applied to some data rather than code. Rules and their configuration are supplied by different authors/users, so parametrising them allows to have fewer rules.
Example:
{
  type: 'object',
  $params: {
    minLength: {
      type: 'integer',
      minimum: 0
    }
  },
  required: ['description'],
  properties: {
    description: {
      type: 'string',
      minLength: { $param: '/minLength' }
    }
  }
}

The rule above requires that the property 'description' is present in the object and that its length is bigger than the parameter that is in the rule configuration. That allows to separate rule definition from rule usage and rather than having a separate rule for each minimum length to have just one rule.
In general, the use case is to have a single schema instead of multiple similar schemas. In many cases it's possible to achieve the same results by extending/combining schemas but:

extending is more verbose
there is no simple solution for extending recursive schemas

So if you try to define schemas for tree with different restrictions for node, then with parametrised schemas you can have one schema for a tree that accepts parameters that affect node definition. Without parametrised schemas you will need to have a new schema for each tree and it would be not possible to use extension/combining.
Parametrised schemas are as useful as higher-order functions. You can live without them, but using them leads to more re-usable and concise code. So if you consider schema itself to be a function that validates data, then the schema with parameters is a higher-order function that returns the function that validates data.

I see commits in Jun 2017.   Is this feature actually included in ajv (as experimental or an additional capability)?  Or was it just a POC in 2017?  Thanks


WIP: Windows "app" support

As discussed over in the Shoes packaging plan, our current aim is to get a "good enough" sort of distributable packaged up for all 3 OS's. This is the rough beginnings of support for Windows!
The moving pieces here are very similar to Furoshiki::JarApp with regards to building out a tmp dir based on a downloaded template zip file, then plugging the right pieces in. Plenty to be revised, consolidated, etc. here between the two flavors.
Where's My Java?
There's one big difference between this and the Mac template, though, and that's the handling of the JRE.
The Mac template embeds a full JRE (roughly ~65MB) so that the thing can be guaranteed to run standalone without any intervention.
If you drag down the Windows template zip, it looks like this instead, clocking in at less than 2MB:
Archive:  /Users/jason/.furoshiki/cache/windows-app-template-0.0.1.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  04-02-17 21:41   windows-app-template/
      574  04-02-17 21:41   windows-app-template/app.bat
        0  04-02-17 21:17   windows-app-template/helpers/
    69120  04-02-17 21:17   windows-app-template/helpers/bzip2.dll
  1177600  04-02-17 21:17   windows-app-template/helpers/libeay32.dll
  1008128  04-02-17 21:17   windows-app-template/helpers/libiconv2.dll
   103424  04-02-17 21:17   windows-app-template/helpers/libintl3.dll
   232960  04-02-17 21:17   windows-app-template/helpers/libssl32.dll
   164864  04-02-17 21:17   windows-app-template/helpers/unzip.exe
   162816  04-02-17 21:17   windows-app-template/helpers/unzip32.dll
   449024  04-02-17 21:17   windows-app-template/helpers/wget.exe

What's up with that? Am I just assuming that the user will have a JRE available? Not at all!
app.bat looks around to see if it can run java, and if it can't, it downloads a suitable JRE from our Github releases. It tucks this JRE in a well-known spot (~/.shoes/jdk) so subsequent Shoes app runs can find and use it.
This approach allows folks that already have a JVM to just use it, while others will get something workable without having to make every package massive. Thoughts?
Smaller Notes

Startup is currently just a .bat but before 4.0 I want to replace that with a small native Windows binary that'll do a better job than Windows batch coding :shudders:
Template zip is hosted as a Github release on https://github.com/jasonrclark/windows-app-templates. Currently still using the explicit version binding from furoshiki to the template it uses
Simplified things in the template by assuming we'll copy the jar to app.jar. Now we only need to rename the start executable (currently app.bat) but the executable can look to a standard location for the actual jar to run.

I want to replace that with a small native Windows binary that'll do a better job than Windows batch coding :shudders:

Shoes 3.3 has a C native launcher in platforms/msw that  you can modify but it bleeds assumptions up the food chain that Shoes 4 may not like.

Appreciate the pointer @ccoupe. Will definitely look there as a starting place when I get to building out that binary. While it will almost certainly take a different form eventually, you've got a ton of hard-won experience in writing native Windows code there that I'd be foolish to ignore!

Be aware that Windows 10 really, really wants code signed exe's from certificate authorities that  MSFT likes. Big Dollars for one of those, per year, every year. If your user's create an  launcher  exe (as users would do ) then they need a code certificate or you vouch for their code. Rock. Hard Spot.

Some of these might be affordable.  It appears that depending upon who you certify with, someone needs to personally identify to the certificate auth as the "owner" of the certificate, including their national ID and permanent residence address.
Here's what I see as referenced from https://msdn.microsoft.com/en-us/windows/hardware/drivers/dashboard/get-a-code-signing-certificate
Plus google search "Microsoft code signing certificate open source projects"
Certum - Open Source Certificate
https://www.certum.eu/certum/cert,offer_en_open_source_cs.xml
Can't tell from the details if this is a .exe signing certificate.
€86.00 gross
The SSLStore - https://www.thesslstore.com/promoads/cheap-code-signing.aspx
COMODO branded
$85/yr
Symantec (https://products.websecurity.symantec.com/orders/enrollment/winqualOrder.do)
No price given - "CSR generation failed. CSR required to continue"

shrug *

Certum - https://en.sklep.certum.pl/data-safety/code-signing-certificates/standard-code-signing.html
€154.12 Incl. Tax / Yr
Entrust - https://www.entrust.com/code-signing-certificates/
$524.25 / yr
Globalsign - https://www.globalsign.com/en/code-signing-certificate/
$289/yr
Digicert - https://www.digicert.com/order/order-1.php
$224/yr

Honestly, at the < $200 range, if at least one other would go halfsies with me on it I would happily fund the windows signing cert.

If your launcher attempts to install an exe (as Shoes 3.3 does) and call it to to eval a script/shy (as Shoes 3.3 does) then Win 10 thinks it's virus. - Technically, it could be a virus/trojan/bad-actor. For Shoes4, installing java,  if needed, will need a certificate - odds are java is signed. Invoking that java to install JRuby - might work on win10 until the overlords at MSFT catch that pattern in their AV engine and want a cert from jRuby. Invoking the freshly installed Jruby to install Shoe4 and then call that to run an unknown ruby script from jrandom_user?
As I said, there are "bubble up" issues design changes that follow.  Did I  mention the horrible code singing dance which can only be done on Windows? Shoes 3.3 can package a users app (shy)  like HacketyHack did and we punt the code signing problem to the user and installer customization (NSIS or QtIFW).  His app, not ours.

@pjfitzgibbons Thanks for the offer, and if it comes down to that I'd happily fund things as well.
Haven't done any further research, but appreciate all the hard-won wisdom you're providing @ccoupe!
Hoping to get some time tonight to finally revise this and get it to a happy-enough state before EOW. Shoes on!

Moved over the template file repo to shoes org, updated this to match and I think this is ready for go-time. Will merge over the weekend unless someone sees something that needs updating.

Back from RailsConf and ready to start moving again. I'm going to merge this and then just roll forward with any changes needed.


Nightly Flake Failures for 7/28

Going to look at startup and shutdown. Widgets look to be intermittently flakey tests. If I don't see anything obvious with widgets will disable them.

Validated.


fail transfer

Tqvault does not transfer items from a safe to a character with the steam anniversary edition.
When i put an item in my vault and retransfer it on another character, it doesn't work at all : my character still doesn't have the item.

Is the game open while doing the operation? Is so, this is not supported (see the answer to "Q. Can I use TQVaultAE while playing the game?"): close the game before opening the tool (and vice-versa). Closing the vault is when the inventories are saved.


[semver:minor] added Jira integration

Requires Jira integration to be enabled in your CircleCI project settings, as well as the hmpps-common-vars context. See https://dsdmoj.atlassian.net/wiki/spaces/PINT/pages/3887562955/Integrate+Jira+with+CircleCI

Given that some teams use Trello and other JIRA, would this be something you implement at the team repo level ?


Fix typo and simplified error string

Minor typo fix and simplified error string

used in pools_update2. I am closing this PR


Drop empty diffs

We might want to go a little further and not apply when merged is empty (i.e. everything went into rejected).