4 new dialog problems

For a while, I've been using tensor2tensor to do dialog modeling research, and I integrated some dialog problems in my own repo, but maybe others could also benefit from these if they were in the official tensor2tensor repo.
All problems are set up in a single-turn fashion without any additional inputs other than the utterances, so basically like a translate problem. I trained a transformer model on all problems and got decent results (see my repo for more details).
The PR contains 5 new files:

dialog_abstract: An abstract base class for all my other dialog problems, since a lot of functionality is shared. I believe this can also be used for other dialog datasets or problems. Subclasses should implement the preprocess_data function, which just sets up the names of data files and download url, and the create_data function, where the vocab file is built, the dataset is preprocessed and converted to be further processed by t2t-datagen.
dialog_cornell: Implements the Cornell Movie-Dialogs Corpus.
dialog_opensubtitles: Implements the Opensubtitles corpus. There are separate classes for different versions of this dataset (denoted by the year), e.g. dialog_opensubtitles64k2012.
dialog_dailydialog: Implements the DailyDialog dataset.
dialog_personachat: Implements the original Persona-Chat dataset.

All problems have basic preprocessing before data generation (lowering, etc.).
At the end of each registered problem name, I put the size of the vocab (e.g. dialog_dailydialog16k).
These problems depend on the clint package to visualize data downloading progress, should this be added to the setup.py script? (as I didn't see it there)

Can I get an update on this PR, what is required on my part to be able to merge it?
If there is no desire to add these problems to tensor2tensor, then let's close it, but I would like to get some feedback on what we should do with it because it just sits here since July.

Hi @ricsinaruto -- We're still working on our end to merge in this PR.
Why this is getting delayed:

the pylint we use internally doesn't like the formatting/doc format etc so that needs to be done.

You can run pylint using the pylintrc we're using by running pylint -j 16 tensor2tensor

internally we write .bazel build rules for new files added.
We'll probably skip clint since we don't want to add too many dependencies (should be a small change)

Really appreciate your contribution and sorry for the late update!

Awesome, thanks for the clarification! Let me know if I can do anything 😄

The only thing you can probably do, and would be a great help for me, is to run pylint with our pylintrc file https://github.com/tensorflow/tensor2tensor/blob/master/pylintrc on the newly added files and fix those and push it?
The other thing I forgot to write (and I'm not sure applies here) is that for a brief bit of time we need to be both py2 and py3 compliant.

I think I fixed everything

Thanks, let me take a look soon, we're busy for ICLR deadline so maybe tied
up a little.
On Tue, Sep 10, 2019, 7:05 AM Richard Csaky<EMAIL_ADDRESS>wrote:

I think I fixed everything
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensor2tensor/pull/1642?email_source=notifications&email_token=AAIUEFRKS2WXEADKOJDHKK3QI6ST3A5CNFSM4IGPYJD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6LGVAY#issuecomment-529951363,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIUEFREP4YDEGZHVVPE7ULQI6ST3ANCNFSM4IGPYJDQ
.

I tried to fix most of the things that you pasted above.

Thanks a ton Richard ! Let me give it another go.
On Fri, Sep 13, 2019 at 2:06 AM Richard Csaky<EMAIL_ADDRESS>wrote:

I tried to fix most of the things that you pasted above.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensor2tensor/pull/1642?email_source=notifications&email_token=AAIUEFTEHQYWHFW3CL4XESDQJNJZ3A5CNFSM4IGPYJD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6UOF5Y#issuecomment-531161847,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIUEFSUL3U2Z2G6INPKRPLQJNJZ3ANCNFSM4IGPYJDQ
.

--
We are here on earth to do good for others. What the others are here for, I
don't know.
Afroz Mohiuddin

Thanks a lot @ricsinaruto ! That really fixed almost everything, now running the tests internally and will merge in shortly -- many thanks for patiently working on fixing everything -- and a huge thanks for contributing the problems in the first place!

Awesome, I'm happy to contribute, and that they will finally make a part of tensor2tensor :)

Done! Thanks a lot for the work!!


plugins/compiler: init

Resolves https://github.com/nix-community/nixvim/issues/1527
From what I could tell, there's not much to setup with this?

Maybe we should add this warning to our documentation?

Some languages require you manually install their compilers in your machine, so compiler.nvim is able to call them. Please check here, as the packages will be different depending your operative system.

Maybe we should add this warning to our documentation?

Some languages require you manually install their compilers in your machine, so compiler.nvim is able to call them. Please check here, as the packages will be different depending your operative system.

Tried adding the note, is that how you guys will be using the new admonitions?

Alright, fixed the admonition.

@mergifyio queue


Monorepo structure

Following on from #16 and the latest engineering call I wanted to kick off some discussion about the structure of the potential monorepo. @nick had some ideas about the directory structure  which were great and I wanted to propose a similar structure that I think would work well with origin-box and docker-compose:
- dockerfiles
- envfiles
- origin-dapp
- origin-js
- origin-bridge
- origin-docs
- tests
- docker-compose.yml
- docker-compose-test.yml
- README
- ...

This is basically the directory structure used by origin-box at the moment. It makes sense for the Docker stuff to be at the top level if it is managing things. Some files ancillary to origin-box like the readme will need to find a new home as the top level readme should be for Origin as a whole.
The tests directory will eventually contain tests for the whole stack, for example we can use a container to mock the Twilio Verify API and test the whole attestation flow using a headless browser.
Some thoughts/discussion points:

Unsure about the origin- prefix on all of the directory names. The main reason I've included it is because having a directory just called js is confusing. Perhaps dapp, bridge, and origin-js?
Other necessary directories? @nick mentioned playground, mobile and schemas. @franckc might need a new one for the Apollo server?
Haven't figured out the origin-website stuff. It should be a separate repo but I'm not sure if it's container still belongs here (it is part of origin-box).

Some future changes:

I think it makes sense for the contracts to be moved to a top level directory (bought up by @nick in #16). Another container could be used to manage Ganache.
Similarly the IPFS stuff should eventually live in a top level directory, origin-bridge and origin-js are now both doing IPFS things and it'd be great if a separate container could handle the projects IPFS requirements.

I mentioned in Discord that I'm a bit concerned about how invested we get in Docker. You can see above I'm proposing adding even more containers. :scream: I think the key for me is that if we go down the docker-compose road we might as well go all the way and have it managing everything, rather than having things spinning up their own bits and pieces (e.g IPFS servers). We definitely need some sort of solution to manage this complexity.

Thanks for getting this started, @tomlinton. For what it's worth, we only recently renamed all of the repos to have an "origin-" prefix. That will be less important once they are nested in a single repo. Back to #16, I had proposed using tags as the method by which we promise stability to newcomers. We would also use that tag in the box so that people wouldn't even need to know what commits to use unless they want to venture out. I guess we can still do that with the containers in the same repo?

I think it's fine to keep the "origin-" prefixes. It makes it obvious which are the packages/apps.

Tags should work fine, we just tell people to checkout a specific tag when getting started. We probably would have to be careful the active development branch doesn't get too far ahead.
The tests directory in the root was intended for integration tests involving multiple components, e.g. testing origin-dapp and origin-bridge together rather than each in isolation (we don't have any of these yet). Each component would still have its own tests in its directory as it does now. It's not really important yet but down the track it might make an appearance.

I edited the directory structure in the first post to include some new things at the top level:

indexing (the GraphQL server, currently in a branch of origin-js)
event-listener (currently in a branch of origin-js)
ipfs (anything IPFS related e.g. pinner, currently lives in origin-devops)
orbit-db (currently lives in origin-bridge)

I think in general all the things that live at the top level should be services in the stack that need to be deployed. There are a few extra things that aren't services but still belong at the top level, I don't think we have to be strict about it, as long as services aren't embedded deep in subdirectories.

Maybe we could call the "indexing" service "origin-discovery"?

@micahalcorn If you need to tag releases of other services, you could just include the name of service in the tag name along with the version number.

@DanielVF I like the name! Perhaps something we should adopt even if we don't make it to a monorepo. I've just been calling it the GraphQL server which isn't very meaningful.

I like origin-discovery ! My only slight hesitation is that in the future the GraphQL server may support other features than just browse/search. Like for example mobile notifications. But I can't really come up with a better name. Maybe origin-services ? ... not great. naming is so hard ! :)
Related to naming, origin-bridge could probably get renamed to origin-attestation since that's the only thing the current bridge server will do for V1.

I like origin-discovery too for indexing. I think if we have other GraphQL services such as messaging, we might consider having that in a separate dir (origin-messaging?) so we can keep the components opt-in. For example a partner might want messaging but not discovery, or vice versa.

I would keep all the configuration in one .env file. Since we are transitioning to mono-repo I don't see why not have one "master" configuration file. IMHO there are some benefits to this approach:

different containers can share a variable. (for example ethNetwork: localhost:8545)
should be less error prone, since you can inspect/set the whole deployment configuration from one file

@sparrowDom agreed it would be nice for convenience. We will lose a few things though:

Ability to configure independently, e.g. to swap out an IPFS gateway for the DApp only to diagnose an issue
Separate files provide some type of self-documentation since it is clear to developers what is configurable for each service

I'm now concerned that we are close enough to the public release that we shouldn't switch before it. Probably need to keep our focus on finishing and polishing.

I think you are right @DanielVF. There are a few simple things that we can do before the public release that would ease the transition and also help me out with deployments.
Right now origin-box is the closest thing to the directory structure we want for the monorepo so if we use that as a base and:

Move the GraphQL/event listener out of origin-js and into the top level of origin-box and do the rename to origin-discovery or origin-services.
Move the orbit-db messaging server out of origin-bridge and into the top level of origin-box. I can do this one because I have to do a bit of a restructure for deployment anyway.

If we have any additional services that are going to need deployment for the public release that don't really have a home (I know @franckc is working on a faucet, not sure where it lives at the moment) then it'd be great if they could be added to the top level of origin-box. I realise origin-box will then have a bunch of code in it rather than being just a development setup which is not ideal, but I think we can deal with it for a while.

@tomlinton See origin-js PR #381 for the faucet code. It is currently lives under origin-js:/token. The only dependency it has on origin-js is the contracts JSON files. I can easily move it to a directory at the root of origin box (maybe origin-token ?). Same thing for the origin-js:/indexing, I can easily move it to origin-box:/origin-discovery.

That'd be great @franckc. Maybe call the faucet origin-faucet just to avoid any confusion with the token contracts?

Doc with mono-repo migration proposal: https://docs.google.com/document/d/1QBER5SbEqPtcMqk-J9fy7jV4xDqHoIirX0HSgqaJWMA/edit#


Update client to angular 19

This PR updates the client to Angular 19 and migrates all components to standalone

Just noticed that the CI tests are failing due to the script being removed, we could either add it back or adjust the workflow


Updates github.com/choleraehyq/pid to v0.0.17 to support Go v1.21

This version support Go v1.21, see https://github.com/choleraehyq/pid/releases/tag/v0.0.17

@JoshVanL let's merge this after cutting the release branch. We have agreed not to use Go 1.21 for this particular release.

@berndverst sounds good! I’ve actually had quite a bit of unexpected pain with Go 1.21 with surprising things not compiling. This dep comes from Kitex which fails to compile with 1.21
Let’s close this for now, and revisit after the release branch to reduce noise.


Art52123103 patch 1

Hi - I'm closing this pull request unless there was something specific trying to be accomplished.


depot password

hi i dont know if you can help me.
my problem is that i have a password on my depot file and i cant copy and paste the password into the browser password field of PP. can you do something at these point?
Greetings
Patrick

i overseen the copy to clipboard at the top right


Weather station selection criteria should include guidance for site as well as zip code centroid and a fallback option in case the first weather station fails because of data sufficiency

Current CalTRACK methods underspecify weather station selection.
"Weather station mapping requires locating the station nearest to the project. Each project file should contain a zip code that allows matching weather stations to projects"
I propose amending the methods guidance as follows:

Weather station mapping should first attempt to find the weather station nearest to the location of the site within the same climate zone.
Weather station mapping should fall back to the next closest weather station nearest to the location of the site within the same climate zone.
If the precise location of the site is unknown, but the zip code is known, the weather station closest to the centroid of the zip code should be used within the same climate zone. As a fallback, the next closest weather station to the zip code centroid within the same climate zone should be used.

I curious what other weather station selection methods were considered. Given the limited number of weather stations listed in the specification, it may be best to use the nearest available station data, but from what I've read, spatial interpolating by Kriging could be appropriate. The interpolation methodology would have to be adapted to accommodate for occasionally missing data.

@roberthansen Do you have a link to the Kriging methodology? We hadn't considered it, but it may be particularly useful as we get into hourly savings methods and have more of a substantial need for complete hourly data.

Here's a thread on the topic, including links to examples in a few languages (R and Fortran); the given answer is a generalized case, but hopefully useful: https://stats.stackexchange.com/questions/157543/how-does-kriging-interpolation-work
ArcGIS also has some help documentation for using their implementations: http://resources.arcgis.com/en/help/main/10.1/index.html#/What_are_the_different_kriging_models/00310000003q000000/
There are of course other interpolation methods that may be worth considering, but Kriging seemed to be the preferred option for temperature in my quick research.

I do not support spatial interpolation of weather data values for CalTrack for three reasons:

This would complicate the process needed to obtain a weather value for a site and significantly increase the computing power requirements.
There is a significant amount of error in spatially interpolated values, so I'm not convinced that the degree of accuracy would be improved over simply using the weather value from the nearest station. You may in fact be layering on an additional source of error.
It is unclear how missing values at a particular station would be handled in the spatial interpolation scenario and it could compound the error introduced by missing values.

I support McGee's general approach to selecting a weather station for a site and using the data from that station. However, in the event of missing values in the time series, I would suggest the following:

For up to two consecutive missing values, interpolate the missing value with the mean of the non-missing neighboring values. This will cover the vast majority of missing weather values.
For more than two consecutive missing values, identify the next nearest station as a fallback, as suggested by McGee, and use the non-missing values from that station to fill in the time series.
If the fallback station also contains missing values during the target time period, then identify the next closest station to fallback to.

Another weather station selection issue to consider: what subset of weather stations should be considered as sources of weather data? Not all weather stations are created equal and they have varying levels of data quality and reliability. In particular, issues to consider are the frequency of readings, percent of missing values in the time series, length of the time series, and calibration/accuracy of sensors. We should evaluate station reliability and develop (or adopt from other work) some criteria to screen out stations deemed unreliable.

Proposed plan:

Document methods for identifying primary and fallback weather stations using site address and ZCTA
Report on average temperature difference between a sample of primary and fallback weather stations
Develop criteria that define acceptable source of weather data and share with the working group (start from @danrubado's suggestions above)
Establish guidelines for filling in missing values, if present

If the approach for selecting the weather station and retrieving/manipulating data require testing, visualization, formatting, or adjusting, the freely available, open source tool , Elements, may be of use. See https://bigladdersoftware.com/projects/elements/ for more information.

Suggest dealing with hourly data separately (ie. different sub-sections), as the considerations will be different (eg. if talking about "two consecutive missing data points"). Based on (limited) recent experience it can be harder to find good quality complete hourly weather data, so we need to think carefully how we set the minimum data sufficiency requirement. Also, on the question of moving to the next-nearest weather station if you find bad data, CalTrack should define method in case that you end up 50 miles away, on the other side of a mountain or on the coastline, ie. in a place where weather may be different. Hard to be prescriptive about it, but you could at least require that the energy modeler justify their choice in cases where they choose a weather station outside of a specified radius.

@eliotcrowe Your suggestion makes sense to me, as we'll have a number of similar questions regarding data sufficiency once we get to hourly. For now, we should focus this question on monthly and daily weather sufficiency.

@danrubado, thanks for pushing back on the interpolation methods. I'd like to continue this discussion before resolving on nearest-neighbor interpolation (or, for that matter, Kriging). I'll address your concerns, item by item:

This would complicate the process needed to obtain a weather value for a site and significantly increase the computing power requirements.

I disagree that complexity would necessarily be increased significantly. With nearest-neighbor interpolation, each weather reading would, at worst, check each weather station's location and time stamps (88 stations if we constrain to the NOAA set, assume top-of-list/no searching for most recent) before selecting the nearest, checking for and applying the nearest (temporally and spatially) available reading. An alternative to nearest-neighbor would be triangulation, which would use three readings, rather than one, and could provide substantially more realistic values when sensors are geographically sparse. Inverse distance weighting and polynomial interpolation could use a subset (up to the full set) of all sensors, and again provide improved results at project sites. In each of these cases, the computational complexity is a function of both the fixed number of stations and the potentially growing number of sites. Since number of stations is constant, and the number of sites will likely exceed the stations, the complexity is essentially O(n), where n is the number of sites. Interpolation methods involving regression (e.g., Kriging) may add complexity to each interpolation calculation by a function of the number of stations, but as this number is constant, overall computational complexity remains O(n). In any case, any interpolation method is well-suited for parallelization. No interpolation method we might consider is going to overwhelm the processing capability a modern computer.

There is a significant amount of error in spatially interpolated values, so I'm not convinced that the degree of accuracy would be improved over simply using the weather value from the nearest station. You may in fact be layering on an additional source of error.

This is a fair point, however it applies equally, if not more so, to the proposed nearest-neighbor interpolation. I have been looking for studies which either apply or recommend this method for weather data, but I haven't found any such endorsement. Authors acknowledge that it is available, and is useful for ease of calculation, but only produces somewhat realistic values unless weather stations are immediately adjacent to the location of interpolation. I believe applying this interpolation across the distances between the 88 NOAA stations will yield unsatisfactory approximations of the weather at project sites. If, however, we allow (or require) implementers to setup their own weather stations at each site, or use nearby, reliable stations within, say, 1 to 10 miles, depending on the terrain and climate zone, this method would be justifiable.
One advantage Kriging holds over other interpolation methods is that it gives estimations of the error in the interpolated value. All interpolation methods necessarily layer on additional errors to those intrinsic to the station data. With nearest-neighbor interpolation, it's clear that these errors are significant, but just much is an unknown.

It is unclear how missing values at a particular station would be handled in the spatial interpolation scenario and it could compound the error introduced by missing values.

Again, resorting to nearest-neighbor interpolation does not address this concern. Indeed, Kriging can be generalized into three (2 space + 1 time) or four (add elevation/adiabatic lapse rate), and interpolations between readings in space and time become automatic--missing data points are no more of a problem than a weather station that never existed.
Finally, here's another resource on several common interpolation methods used for meteorological data, including nearest-neighbor and several flavors of Kriging:
Interpolation Methods for Climate Data
Since CalTrack aims to be an open, repeatable methodology, I would propose considering the deterministic interpolation methods described in the link above, as anyone could apply the methodology to the same data and return identical results. Kriging, while potentially a more accurate method, is stochastic in nature, and thus won't return the same results on repeated analysis. Thus, I propose either the inverse distance weighting (easier to code) or linear regression (better results).

Regarding missing data, I recommend reviewing literature for best practices, as I've seen authors suggest different approaches depending on the sampling interval. I'm not sure how real-time the calculations are to be done, but if we're talking about intermittently missing historical data, it makes sense to interpolate between both nearby stations and the closest station before and after the outage. For real-time calculation, we wouldn't have the nearest reading after the outage, but could still use the reading before the outage.
More generally, however, I think we could drop the weather station paradigm entirely and deal with weather readings independently, so for each site, at each time of interest, we would interpolate among a time- and space- windowed set of readings. For nearest-neighbor interpolation, this set would always be one reading (alternatively, interpolating in time between nearest-station readings is another possibility, in which case we would be looking at consecutive readings from one station). For more advanced interpolation methods, the window can be expanded arbitrarily or based on geographical features (i.e., mountain ranges). In this case, we wouldn't need to specify data sufficiency, but would need to define minimum sensor quality, including proper setup. This would open up the possibility of incorporating temporary and mobile weather stations.

It might be helpful to run an out of sample test using a sample of homes in which we select weather data from the primary weather station and then for the same sample of homes we select the fallback weather station data. It may turn out that the difference is large enough to justify additional cleaning of weather data. It also might turn out to not matter very much.

One of the first cuts at weather data we looked at so far is how consistent the weather is across a state over the course of a year. Of course this will raise more questions than answers, but I thought others would find this interesting as we dig in.
These are plots of all the weather stations in several states (New York, Texas, Illinois, Colorado, and California). You can see that California actually has three weather patterns, but in the other states, the weather is remarkably consistent across the state. The next question will be, how much do the variations in the weather matter?

I used the following testing methodology to test methods for matching a site to a weather station.
To isolate differences in weather from differences in model fits due to the peculiarities of any particular site, this methodology avoids using meter data and focuses entirely on temperature data. Temperature models as described in the daily and billing methods assume that an accurate source of site temperature data is available. Weather station matching algorithms are developed to assist in the process of finding appropriate weather station from which to obtain this temperature data. But to evaluate the effectiveness of these weather mappings, we need a way to test them for accuracy.
What I'm proposing as a more general testing method for weather station matching algorithm is to bootstrap a scenario in which we have knowledge of weather at a site - in this case by matching weather stations sites to other weather stations. Because we have a clear picture of weather at each weather station site, we can use that to match it to other stations using metrics such as weather-to-weather cosine distance or rmse we can't normally use without “ground truth” temperature data (because few buildings have a reliable data source of outdoor temperature data on site). The assumption, of course, is that the most similar source of weather data will be the best one to use in modeling (I see no reason why this should not be the case - but I could imagine someone coming up with a different similarity metric that more heavily weight the similarity of the shape or mean. This method should work even in those scenarios.)
Weather similarity was determined in the following way. For all weather stations in california meeting the data quality pre-screen, we pulled hourly temperature data for Jan 2014 to Jan 2018. We combined 3 distance metrics (rmse, km, and cosine_dist) by adding each rank together with equal weight (like scoring a racing team). Using that metric there are 12 (of 115) stations which have "best" stations with scores of 10 or higher, meaning for the remaining stations the worst rank we see is a 6 in any category, showing some evidence of stability. Two stations have scores are above 20, max is 29. Ranks correspond to size on the map below. A rank with a larger number means that the “best” station was less convincingly dominant in the three ranked similarity scores.
Example:
Station 1: rmse rank 3, distance rank 4, cosine distance rank 5 -> score 12
Station 2: rmse rank 2, distance rank 6, cosine distance rank 3 -> score 11
Station 2 has a better ranked similarity than Station 1.
 (Considering the following standards for US sites and stations: (a) IECC Climate Zone, (b) IECC Moisture Regime, (c) Building America Climate Zones, (d) CEC California Building Climate Zone Areas)
Determine the climate zone inclusion of the site, i.e, for each considered climate zone standard, determine which climate zones contain the site.
Determine the (next) closest candidate station.
Determine the climate zone inclusion of the candidate station.
Reject the candidate station if any of the following are true:
a. The candidate station is further than 150 km of the site.
b. The candidate station climate zone inclusion does not match the site climate zone inclusion for all considered climate zone standards.
c. The candidate station does not have sufficient data quality when matched with site meter data.
If criteria in (6) are met, use the candidate station. If criteria are not met, continue to test other candidates by moving back at step 3.
If no stations meet the criteria above, use method B.

Method B:

Determine the candidate stations which will be considered. CalTRACK recommends considering all ISD stations. If Method A did not succeed, CalTRACK allows using weather data not obtained
Determine the (next) closest weather station.
Reject the candidate station if any of the following are true:
a. The candidate station does not have sufficient data quality when matched with site meter data.
b. If no stations meet the criteria above, consider the site unmatched.

For each method, we measured the percentage of station-sites for which the method found the “ground-truth” station and used this to “score” the method. These scores are reported below and were used to determine preferred order of the methods.
Method A: 56%
Method B: 53%
Therefore, Method A is recommended in favor of Method B. Method A scores 56% because 56% of the selections made by Method A (restricted to the test station set) exactly matched the ground truth ranked distance selection in the table above. There is likely room for improvement here if other methods are proposed, for example one using elevation (although using a method like that would require having elevation data at the sites in question, which may not always be practical). One could imagine using a scoring method similar to this (i.e., using bootstrapped weather stations as sites) to score the accuracy of various kriging methods, although I did not.

Another effort I made was to test the effect of using poorly matched weather data on the following two meters from the greater Philadelphia area (matched weather station USAF ID 725113, ~8km from site) , using the Caltrack 1 Daily model (hdd range 55-65, increment 1, cdd range 65-75, increment 1). Basic fits on two years of data show below. (Also performed fits on first year and second year alone and both showed little difference):

Then I picked two weather stations from each state, the idea being that the weather at these stations would diverge widely (but realistically) from the weather observed at the station closest to the ISD station.
usaf_ids = [
    '720265', '720363', '702710', '998224', '720644', '722786', '723419', '723450', '722956', '722931',
    '724769', '720535', '725046', '725086', '997281', '997694', '722069', '722067', '722137', '720712',
    '911820', '911900', '720322', '725866', '744662', '722075', '724387', '720575', '725480', '725477',
    '724585', '724580', '724238', '720447', '720346', '722312', '997707', '726196', '722218', '997781',
    '725107', '744104', '725373', '725408', '726583', '726564', '723320', '998239', '724460', '724459',
    '727750', '727690', '725540', '725625', '725825', '724770', '726165', '742078', '997687', '724094',
    '722710', '723658', '744989', '744864', '723165', '723156', '727675', '720863', '724298', '725210',
    '723560', '722091', '726945', '726985', '724080', '725434', '722151', '725070', '723119', '723119',
    '726625', '726605', '723340', '721031', '720315', '722533', '725755', '725720', '720493', '726140',
    '724053', '724053', '727985', '722208', '724176', '724273', '720327', '726430', '720345', '725776'
]

Then I fit the models using the same CalTRACK 1.0 daily method with the same meter data but different the weather data - once for each of these stations.
The results are charted below, with results for electricity on the left and gas on the right.
Commentary on each row of charts:

No surprises here. Temperature differences (rmse) increase with distance. Much variation seems to appears within the first 1000km. Outliers are AK, HI. Temperature RMSE is calculated on hourly frequency data.
Model r-squared tends to decrease (but not always!) with temperature. R-squared appears for this meter data to be fairly consistent out to 5-ish degrees F average difference in temperature.
Model in-sample prediction error increases mildly with stronger temperature differences. Model fit on first year of data and predicted over that same year.
Model out-of-sample prediction error increases mildly with stronger temperature differences. Model fit on first year of data and predicted on second year of data.
In-sample predictions are very consistent, even with high temperature differences. (this is expected, due to model fitting process).
Out-of-sample predictions are surprisingly consistent, even with high temperature differences. Models fit on first year of data and predicted on second year of data. The consistency of the annual totals is striking, and shows some resilience of the predicted totals to badly fitted weather data. (although consistency in disaggregated, monthly, daily, or other breakdowns has yet to be measured.
(and 8) Heating and cooling balance points (and other model parameters too, presumably) appear to change dramatically due to differences in weather data and this suggests that models are highly coupled to weather station/source, and thus a model which was fit with data from one weather station should not be used to predict with data from another weather station.

From the summary:

This indicates that the accuracy of weather data does not have a significant effect on annual energy savings predictions, even in extreme cases.

There are a number of possible explanations for this. I would suggest a more accurate conclusion is:

This indicates that the accuracy of weather data does not have a significant effect on [CalTRACK's existing] annual energy savings predictions [on this building data set], even in extreme cases.

That was the intended conclusion, thank you for the clarifying language.

To further clarify the language, maybe "accuracy of weather data" (open to
interpretation, and there are many kinds of inaccuracy) could be adjusted
to something like "location of weather station", with an added caveat that
the SAME weather station needs to be used for pre and post case.
This might be stating the obvious but it seems that the key element is that
all weather stations have similar high/low/shoulder seasonal
characteristics. Even if different states see different extremes, the model
just needs to associate the highest energy consumption with the highest
ambient temp, and it doesn't seem to matter much if the highest ambient
temp is 90F or 110F, so long as it's occurring at roughly the same time of
year(?).
On Tue, Mar 6, 2018 at 9:19 AM, Phil Ngo<EMAIL_ADDRESS>wrote:

That was the intended conclusion, thank you for the clarifying language.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CalTRACK-2/caltrack/issues/65#issuecomment-370858147,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AiYRAEKgYb4dJFrouuvjibMnwOzCqPrdks5tbsUlgaJpZM4R1I_1
.

--
Eliot Crowe
Program Manager
Building Technology and Urban Systems Division
Energy Technologies Area
Lawrence Berkeley National Laboratory
541-708-3034 (O)<PHONE_NUMBER> (C)

"Accuracy" is definitely the wrong language, as it necessitates having something to measure against - a ground truth. I did not measure or get access to that ground truth at this or most other sites, so the lack of a ground truth in this case makes my choice of the word "accuracy" totally meaningless. Oops!
I also didn't actually measure statistical significance, so I'm going to avoid using "significant" as well.
Let's go with this milder version:

This indicates that the choice of weather station did not have a visible effect on CalTRACK's existing annual energy savings predictions in this building data set, even in extreme cases.

I raised the issue of accuracy in issue 73. As described there, I believe you could use a best-in-class building energy simulation tool (like CBECC-Res) to run various tests with different building types and weather data to test CalTRACK methods. These modeling tools create highly accurate disaggregated energy data for these virtual buildings, with the benefit of having no real occupants to mess up the measurements. This may be the closest thing we can get to "ground truth data".

Right! It's good to remember that there are many different ways to test these methods, and anything that gives us more (or less!) confidence in these methods can be helpful. It's likely there is something to gain from using simulations as a proxy for true ground truth. I think that at the end of the day what matters is the extent to which CalTRACK methods are reliable under a variety of circumstances. The better we understand these methods and their characteristics the better equipped we are to make those decisions.

This update has been integrated in CalTRACK 2. Closing this issue. Added an issue for weather data averaging in Sandbox #91


Postgresql Agent

Why are these changes needed?

This is an alternative approach to run SQL queries on PostgreSQL database without using function calling.
Mostly copy-pasted from coding package and user proxy agent.
Related issue number

Serve as an example for issue #28
Answer discussion #1558
Checks

[x] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[x] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[x] I've made sure all auto checks have passed.

@microsoft-github-policy-service agree

Nice PR! @mdfahad999, @MihirDavada, @ChristianWeyer, @tyler-suard-parker, @Namec999, @tytung2020, @walker-null-byte, @nengisuls for awareness and it would be great if you could help testing and reviewing this PR!

@yonitjio could you resolve the conflict in setup.py. Thanks!

What happened? Did I make a mistake? The gitguardian points to a file from a commit from another PR but why the name is main? I tried to sync from my fork but it seems it has no effect.

What happened? Did I make a mistake? The gitguardian points to a file from a commit from another PR but why the name is main? I tried to sync from my fork but it seems it has no effect.

Hi @yonitjio, sorry for the confusion. It is not caused by this PR and it is a false alarm. Please ignore it.

What happened? Did I make a mistake? The gitguardian points to a file from a commit from another PR but why the name is main? I tried to sync from my fork but it seems it has no effect.

Hi @yonitjio, sorry for the confusion. It is not caused by this PR and it is a false alarm. Please ignore it.

Thanks for clarifying.

@yonitjio thanks for the PR!
I was wondering if this design could also work for other sql databases as well, not just postgres. Have you considered using sqlalchemy as well?

I have, but I didn't think I can complete a PR using sqlchemy or similar library (someone suggested using pandasai in issue #28).
I already used this method for my experiment using Autogen with Odoo which uses Postgresql, so I didn't really do much for this PR.
I believe the more flexible way to do it is to have the these line as a callback:
                with pg.connect(self.dsn, row_factory=dict_row) as cnn:
                    with cnn.cursor() as cr:
                        cr.execute(code)
                        cres = cr.fetchall()
                        res_json = json.dumps(cres, default=str)

Link: https://github.com/microsoft/autogen/blob/930474588ebfe5511caf6fcc77f1c6ca254caaab/autogen/agentchat/contrib/postgresql_agent.py#L70
This way the code becomes more generic and does not depend on any specific library, but one may argue this burdens the user with generic code.
Of course, using connector library as you suggest is also feasible.
Autogen's coding package is very flexible, this is just one example of it.

Thank you very much for the PR, @yonitjio ! I left a few comments, but I'm OK to see the sql support being added as a code language support in another PR.

@thinkall I could not agree more with you on this. This PR should not be merged to main branch as there are better ways for Autogen to support SQL with the coding package. I'm going to close this PR. I apologize for the inconvenience.
I think we should continue the discussion in issue #28 for better implementation.


User Management

This is indeed a wonderful work done. Thank you for that. But the Role based implementation is not done all correctly. Though the configuration is done in site.php but when I login as staff or when I login as admin, that makes no difference in the Side Menu. Also, if I type the URL in the address bar directly, there are no restrictions.
In case, these two implementation are done, this will really be great. :)

@waifung0207 any plans to implement Role Based Access for Admin Users?

This is a good call because I am also preparing to add this feature into the repo. Currently you can still use verify_login() and verify_auth() functions from MY_Controller to check with user login.
Please stay tuned and I will implement the role handling from site.php in near future. Thanks for the issue.

In latest commit (dd25bcce82414b4982243fa02f2051b4e8b7f6ba), I have implemented the feature aforementioned.
Now you will see from site.php, there is a page_auth array to define page restrictions for different groups of login users, which will also affect the items appearing on sidemenu. In the mean time, "groups" array under menu is removed since it is not necessary any more; and the "groups" array of useful_links is now renamed to "auth" due to consistency.
You may try it out and see if there is any problems.

sure Thanks @waifung0207 :) I'll check it out and revert to you

@waifung0207 What is the correct way of hiding login and signup buttons from logged in user. And also show logged in user account or profile link.

@cansanet Yes I will use $this->ion_auth->logged_in() to limit access to pages like Login / Sign Up.


Running installer for Ascender-only, followed by Ascender-Ledger does not add Ascender to Ledger

Steps to reproduce:

Run the installer with Ledger excluded.
Run the installer with Ledger included.

Version: 5ef2888294a839abe1ba1ce05c64180696f743fe

A sample custom.config.yml:
---
# This variable specificies which Kubernetes platform Ascender and its components will be installed on.
k8s_platform: k3s
# Boolean indicating whether to set up a new k3s cluster (true) or use an existing k3s cluster (false)
kube_install: false
# Offline Install - Whether to use local assets to complete the install
k8s_offline: false
# Boolean indicating whether or not the kubeconfig file needs to be downloaded to the Ansible controller
download_kubeconfig: false
# Determines whether to use HTTP or HTTPS for Ascender and Ledger.
# If set to https, you MUST provide certificate/key options for the Installer to use.
k8s_lb_protocol: https
# Routable IP address for the K3s Master/Worker node
# required for DNS and k3s install
k3s_master_node_ip: "<IP_ADDRESS>"
# Boolean indicating whether to use the local /etc/hosts file for DNS resolution to access Ascender
use_etc_hosts: false
# TLS Certificate file location on the local installing machine
tls_crt_path: "/etc/letsencrypt/live/ascender/fullchain.pem"
# TLS Private Key file location on the local installing machine
tls_key_path: "/etc/letsencrypt/live/ascender/privkey.pem"
# A directory in which to place both temporary artifacts
# and timestamped Kubernetes Manifests to make Ascender/Ledger easy
# to uninstall
tmp_dir: "{{ playbook_dir}}/../ascender_install_artifacts"
# DNS resolvable hostname for Ascender service. This is required for install.
ASCENDER_HOSTNAME: ascender-test.our.domain.com
# Namespace for Ascender Kubernetes objects
ASCENDER_NAMESPACE: ascender
# Administrator username for Ascender
ASCENDER_ADMIN_USER: admin
# Administrator password for Ascender
ASCENDER_ADMIN_PASSWORD: REDACTED
# The image tag indicating the version of Ascender you wish to install
ASCENDER_VERSION: 23.5.1
# The version of the AWX Operator used to install Ascender and its components
ANSIBLE_OPERATOR_VERSION: 2.9.0
# Determines whether to keep the secrets required to encrypt within Ascender (important when backing up)
ascender_garbage_collect_secrets: true
# External PostgreSQL database name used for Ascender (this DB must exist)
ascender_replicas: 1
# The Ascender web container image pull policy (If unsure, choose IfNotPresent)
image_pull_policy: IfNotPresent
# Determines whether or not Ledger will be installed
LEDGER_INSTALL: true
# DNS resolvable hostname for Ledger service. This is required for install
LEDGER_HOSTNAME: ledger-test.our.domain.com
# Number of replicas for the Ledger web container
ledger_web_replicas: 1
# Number of replicas for the Ledger Parser container
ledger_parser_replicas: 1
# The image tag indicating the version of Ledger you wish to install
LEDGER_VERSION: latest
# The Kubernetes namespace in which Ledger objects will live
LEDGER_NAMESPACE: ledger
# Admin password for Ledger (the username is admin by default)
LEDGER_ADMIN_PASSWORD: REDACTED
# Password for Ledger database
LEDGER_DB_PASSWORD: REDACTED


Bug: secret-forget-secrets does not work after yosys optimizer

@j2kun - I thought it would be nice to check the Yosys optimizer pass for correctness by chaining yosys-optimizer with secret-forget-secrets and then lowering to LLVM to execute (this would have caught the bug)
The issue here seems to be that the secret.cast that transforms the multi-bit secrets to the tensor of single-bits has no conversion pattern. We already have code that extracts bits from plaintext multi-bits so we can use that here for an input conversion.

I lost the train of thought on this bug. Is there a simple reproducer for the error?

Sure! Maybe the add_one example is the easiest. It hits this bug on anything with a multi-bit integer value.
bazel run -c dbg @heir//tools:heir-opt --  --yosys-optimizer --canonicalize --cse --secret-forget-secrets $(pwd)/tests/yosys_optimizer/arith_ops.mlir


Add message tracing over NATS using open tracing standard

FEATURE REQUEST

Is there an open issue addressing this request? If it does, please add a "+1" reaction to the
existing issue, otherwise proceed to step 2.
No.

Describe the feature you are requesting, as well as the possible use case(s) for it.
After adding open tracing support to the existing gRPC and HTTP endpoints in Users and Things service, we should add support for tracing async messages that are sent over Mainflux.

Indicate the importance of this feature to you (must-have, should-have, nice-to-have).
This is a must-have.

After doing the previous tracing issue, I figured I would grab this one as well.
I'm going to take a little time to read up on NATS before starting but I see that there is a NATS publisher in a number of services.  I could see the implementation of this being similar to the tracing middleware in users and things.

Great! Thanks @nwneisen.

@anovakovic01 I took another look at this based on the not.go repo.  I believe the tracing middleware I create was on the right track but I needed to transfer the span over a NATS message rather than a context.
The publish messages look like I can write the mainflux.RawMessage payload to a not.TraceMsg and save the result as the new payload.  The span's context is then injected into this not.TraceMsg.
The subscriber side will be trickier due to messages coming in over a channel.  I believe the tracing middleware should receive the message from this channel and extract the span context from this message.  The remainder of the not.TraceMsg will be the payload that can be passed to the NATS broker.
Does this sound more like what you had in mind?

@nwneisen You're right, you can write context data and raw msg payload into the not.TraceMsg. Think about maybe embedding this info in the RawMessage, and avoid using TraceMsg, because, if we at some point change broker, we'll have to replace this not package for something else too. mainflux.RawMessage already contains metadata related to the message.

@anovakovic01 I started taking a look at this again.  This draft PR adds tracing for publishing the way we described above.  I'm still trying to wrap my head around how this can be done for subscribing.

@nwneisen thank you for the PR. It looks to me like it's the step in the right direction.

@nwneisen We recently created a broker package that wee use in all services to publish and subscribe. I guess that now it should be simpler to resolve this one. Would you like to take this one again?


No intermidiate directories are created when minifying to an empty output directory

Need a line in MinifyAntTask:
private void minifyOneFile(Minifier minifier, File fromFile, File toFile) {
	if(shouldSkipMinification(fromFile, toFile)) {
		return;
	}

-->		toFile.getParentFile().mkdirs();

Thanks for the hint!


.File.Anything without backing file throws error instead of warning

Whether this is a bug or enhancement is unclear.
Example:
{{ where site.Pages "File.BaseFileName" "p1" }}

With v0.122.0 and earlier:

WARN .File.BaseFileName on zero object. Wrap it in if or with: {{ with .File }}{{ .BaseFileName }}{{ end }}

With v0.123.0 and later:

ERROR Rebuild failed: render: failed to render pages: render of "home" failed: "/home/user/project/layouts/_default/home.html:12:5": execute of template failed – Pages "File is nil; wrap it in if or with: {{ with where site.Pages "File }}{{ .where }}{{ end }}

It is obvious to me that site and theme authors have been ignoring the warnings for years, so I am not terribly sympathetic to their current woes. Having said, it may be in our best interest to warn instead of error; I have seen 4 or 5 cases in the last couple of weeks, each one generating noise either here or on the forum.
Note that this issue is about File.Anything, not just File.BaseFileName.
References:

https://discourse.gohugo.io/t/48497
https://discourse.gohugo.io/t/48576
https://github.com/gohugoio/hugo/issues/12174#issuecomment-1970929216

We have had the general discussion about this before, so I will not repeat what I said about it then.
But if we scope this to where, then I would say that this is an old bug that needs fixing, as there is no way the user could do nil checking inside where clauses.
So I would say that:
{{ where site.Pages "File.BaseFileName" "p1" }}

Should just silently not match on any expression that is nil on the left side.

That sounds good to me. I am just trying to reduce noise.

I believe we now have better error messages for these nilpointers, which is all we can (or want to spend time on) do.


Upgrade to Python 3.12

Describe the bug
Upgrading from Python 3.11 to 3.12 breaks.  Get    [!!!!!]  | Something is wrong with your internet connection error after copying files over to new location.
To Reproduce
Steps to reproduce the behavior:

Copy ./site-packages from /usr/lib/python3.11 to /usr/lib/python3.12
Run radio

Expected behavior
Radio main page opens.

No longer receiving the error, so it might have been a server-side issue.  The app still does need to be updated to 3.12; until then, files need to be manually copied over as above.


Pulser retention 2

Updates to quad channel 765 support and minor fix for digital 675

Seemingly was integrated, with one dangling commit will push to other PR


RaspberryPi Pico W support

Heya peeps! Leaving this link to similar work on TinyGo: https://github.com/tinygo-org/tinygo/issues/2947#issuecomment-1689225954
Got some info I wish I had known before starting this odyssey :smiley:
God speed!


can't install pycrypto package

I can't install pycrypto package.
I described my problem on this page.
But pycrypto's repository seems to be dead, and no one will ask on my question.
Can anyone help me?

Answer from Stack Overflow

@LotosikRa this issue isn't related to flask-user, I think you need to open it with your distros or pycryptos maintainer. Can you close this issue.

Sorry to bother, but I think I'll bump this issue a bit.
How about replacing long-dead pycrypto with pycryptodome (drop-in fork) or, possibly, something modern like cryptography (maybe even use its Fernet instead of AES+TimestampSigner, although this is would be backwards-incompatible)?


Implement Basic FlightSQL Client #1413

Which issue does this PR close?
Closes #1413

Client application replicates functionality in FlightSqlClientDemoAp and works against FlightSqlExample
cargo run --example=client --features="flight-sql-experimental" Execute "select * from app.inttable"

+----+--------------+-------+-----------+
| ID | KEYNAME      | VALUE | FOREIGNID |
+----+--------------+-------+-----------+
| 1  | one          | 1     | 1         |
| 2  | zero         | 0     | 1         |
| 3  | negative one | -1    | 1         |
+----+--------------+-------+-----------+
+----+---------+-------+-----------+
| ID | KEYNAME | VALUE | FOREIGNID |
+----+---------+-------+-----------+
+----+---------+-------+-----------+

Some cleanup/suggestions for cleanup are very welcome (I'm still very new to writing rust code, most definitely will want to make the error handling more rust idiomatic..)

Thanks for the PR!
Actually I think what we expected is a client API that can help user to build the example you showed in here. Such as we add a src/sql/client.rs to implement the API similar to cpp-client-api, and then use the client api in examples/

Got it. Will try to find some time to  factor out the (ipc/conversion) parts and see what would make sense in a client-api..

@timvw I pushed my WIP code for your reference, you can take a look if you are interested.
https://github.com/wangfenjin/arrow-rs/commit/7da031d7debef6ef683e28bc0961b63227dc4a5f

I hope to find time to review this tomorrow

@alamb @viirya Could you please help to trigger the CI processes?

Triggered.

Codecov Report

Merging #1616 (202e0ec) into master (22e9f95) will decrease coverage by 0.22%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1616      +/-   ##
==========================================
- Coverage   83.02%   82.79%   -0.23%
==========================================
  Files         193      194       +1
  Lines       55612    55764     +152
==========================================
- Hits        46174    46172       -2
- Misses       9438     9592     +154

Impacted Files
Coverage Δ

arrow-flight/src/sql/client.rs
0.00% <0.00%> (ø)

arrow-flight/src/sql/mod.rs
0.00% <ø> (ø)

arrow-flight/src/sql/server.rs
0.00% <0.00%> (ø)

arrow/src/error.rs
21.56% <0.00%> (-2.88%)
:arrow_down:

parquet_derive/src/parquet_field.rs
65.98% <0.00%> (-0.23%)
:arrow_down:

parquet/src/encodings/encoding.rs
93.37% <0.00%> (-0.19%)
:arrow_down:

arrow/src/array/transform/mod.rs
86.57% <0.00%> (-0.12%)
:arrow_down:

arrow/src/datatypes/datatype.rs
66.80% <0.00%> (+0.39%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22e9f95...202e0ec. Read the comment docs.

Thanks @timvw  -- I am sorry I haven't spent much time helping to review this code. If you think it is ready to do so I'll try and give it a thorough review later this week
Thanks again

It's still WIP, maybe we can mark it as draft for now @timvw

cc @avantgardnerio  here is some work related to making a FlightSQL client

Closing in favour of https://github.com/apache/arrow-rs/pull/3207


Utilizing gevent/pywsgi for socketio server and HTTP listeners

Empire Version
3.0-Beta
OS Information (Linux flavor, Python version)
Any
Expected behavior and description of the error, including any actions taken immediately prior to the error. The more detail the better.
N/A
Screenshot of error, embedded text output, or Pastebin link to the error
N/A
Any additional information
The current socketio server in 3.0-beta (as well as the HTTP listener) uses the Flask development server, which isn't recommended for production use and can suffer under heavier loads. This is particularly relevant for socketio, where significant amounts of data may be transferred to connected operators while connected. I've done some work and been able to integrate gevent into the 3.0-beta branch, utilizing it for the socketio server, as well as gevent's pywsgi server for all other Flask-utilizing listeners throughout the project. Prior to submitting a pull request, I just wanted to post and gather any thoughts on this from current project developers.

@bpengu1n I'm always open to improvements. Submit your PR whenever and I'll test it when I can. Thanks!


no downtime while moving database from/to elastic pool?

I would like to make sure the "Additional resources" (in the following sentence) includes 'database' - actually I got a question (any downtime while moving database from/to elastic pool?) from my customers (same question from a few different customers at about the same time) but I couldn't find any specific information (or I'm missing) except the following.  Thanks.
"Additional resources can be added to an existing pool with no database downtime, except that the databases may need to be moved to provide the additional compute resources for the new eDTU reservation."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 954fdc5f-1572-c6a5-eb19-22e890b1174f
Version Independent ID: b5776cca-f233-4c34-a649-9a61324a2edd
Content: Manage multiple SQL databases with elastic pools- Azure
Content Source: articles/sql-database/sql-database-elastic-pool.md
Service: sql-database
GitHub Login: @CarlRabeler
Microsoft Alias: ninarn

@kazshi Thanks for your feedback! We will investigate and update as appropriate.

@kazshi thanks for your question. Adding a note for more clarity. https://github.com/MicrosoftDocs/azure-docs-pr/pull/47627

@CarlRabeler Thank you for your help and @kazshi We appreciate your feedback. We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


API Gateway Custom Authorizer

Hey folks,
as mentioned in the issue #4 I have added a feature that helps creating Custom API Gateway Authorizers.
I'm looking forward to merge this PR.

@tdsis Actually it would be better to use the AWS SDK classes to build an IAM Policy...
My bad!
I'll try to change that and make a new PR.

Well, I couldn't find a simple way to serialize the AWS SDK Policy classes the way that API Gateway expects(key words starting with uppercase letters), so I kept the IAM Policy representation classes (AuthPolicy, PolicyDocument and PolicyStatement````) and added the enums PolicyActionandPolicyEffect```.
I guess this implementation fits a basic custom authorization flow. I'll look forward to enhance this feature.


EKS 1.14 and external DNS seems is not trying to use the Service Account policy

I have installed the latest Helmchart, which installs the 0.5.16 version
Even though my Env Variables and the token are readable:

Environment:
AWS_DEFAULT_REGION:           us-east-1
AWS_ROLE_ARN:                 arn:aws:iam::****:role/cm--dev-eks.kube-system.cs-external-dns
AWS_WEB_IDENTITY_TOKEN_FILE:  /var/run/secrets/eks.amazonaws.com/serviceaccount/token

The error I get is: time="2019-09-12T17:18:32Z" level=error msg="AccessDenied: User: arn:aws:sts::250***:assumed-role/cm-dev-eks.worker-node-role/i-06ecf82ba**** is not authorized to perform: route53:ListHostedZones\n\tstatus code: 403, request id: 44d1b36c-2eda-4a*****
which it is an instance role!
If I block access on pod level to AWS endpoint the error is

cs-external-dns-f4d8556f7-96sn4 external-dns time="2019-09-12T20:51:21Z" level=info msg="Created Kubernetes client https://<IP_ADDRESS>:443"
cs-external-dns-f4d8556f7-96sn4 external-dns time="2019-09-12T20:51:53Z" level=error msg="NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"

also the arguments i start this pod is:

Args:
  --log-level=info
  --policy=sync
  --provider=aws
  --registry=txt
  --interval=1m
  --source=service
  --source=ingress
  --aws-batch-change-size=1000

Using: helm chart: 2.6.1
pod: docker.io/bitnami/external-dns:0.5.16-debian-9-r8
Thanks,
Vassilis

I am experiencing the same issue...

You should use the latest master of ExternalDNS. Amazon SDK is updated very recently, so there is no official release that supports EKS Service Account policy:
https://github.com/kubernetes-incubator/external-dns/commit/0ba01198ba9d7aaab2c1e3353bdb900e5c24f792
Slightly related topic: https://github.com/kubernetes-incubator/external-dns/pull/1185

New release has been released, feel free to reopen the issue if it still doesn't work with the latest release.

Verified and everything works as expected with EKS 1.14.


docs: fix link to homepage

I think this was a replacement mistake when moving over. This the the homepage for sp-repo-review, so linking to the GitHub page seems to make sense.
Fix #331.
Not sure what the best way to check for this might be, might be something a link check tool could support?

Hi @henryiii.
If there is no linkchecker for this sort of check, maybe a custom script could help?
Something like this? Not sure when or where to run this. It could be a pre-commit-hook, but if it talks to the network on every commit, this might be slowing things down. Mabye run this in CI?
import logging
import requests
import toml

logger = logging.getLogger(__name__)

with open("pyproject.toml") as f:
    data = toml.load(f)

urls = data.get("project", {}).get("urls", {})

with requests.Session() as session:
    for label, url in urls.items():
        try:
            response = session.get(url)
            response.raise_for_status()
            logger.debug(f"URL for '{label}' is valid: {url}")
        except requests.exceptions.HTTPError:
            logger.warning(f"URL for '{label}' returned a non-success status code: {url}")
        except requests.exceptions.RequestException as e:
            logger.error(f"Error checking URL for '{label}': {url}\nError: {e}")

Or this is stupid?

I think you could use Linkchecker, though it doesn't have an appropriate plugin AFAIK (https://linkchecker.github.io/linkchecker/man/linkchecker.html#plugins), you can just run it on the URLs, I think.
(PS: use tomllib or tomli, toml has been dead for years)


Simple Objects - Error appears when trying save to templates simple objects on canvas

Steps to Reproduce

Open Ketcher.
Draw some Ellipses, Rectangles, Lines using 'Shape Ellipse', 'Shape Rectangle', 'Shape Line' buttons.
Click the 'Save as' button.
Click the 'Save to templates' button.

Expected behavior
There's no error in console.
Actual behavior
Error appears
https://user-images.githubusercontent.com/90637536/134478939-19eb9140-a6ba-47e0-b9de-7aef46e55a8b.mp4
Desktop:
OS: Windows
Browser: Chrome Version 93.0.4577.63, Firefox Version 92.0, Microsoft Edge Version 93.0.961.47
Additional context
Reproduce in version 2.3
https://lifescience.opensource.epam.com/ketcher/demo/ketcher_2_3_remote.html
https://lifescience.opensource.epam.com/ketcher/demo/ketcher_2_3_standalone.html

Release/2.4 Closed as fixed.


Add HEIC and WEBP support

Could you add support for the HEIC image format (image/heic)?
Would it be enough to just add it here?

Can you please tell me where the support is failing according to you? The source you are referring to is for dynamically resolving embedded images from classpath.

Sorry it seems like I had a version mismatch. It works now! Thanks for your fast response. It's a great project with great support :)

Still, I've added .heic support for dynamically resolving embedded images from classpath  :)


Example doesn't work in Scala 3 with latest versions and sbt

Hi there - I copied your example from cask/scalatags but used the latest versions and Scala 3 and I'm getting a spurious error:
Found:    scalatags.Text.all.html.Self
Required: scalatags.text.Frag

The following import might make progress towards fixing the problem:

  import sourcecode.Text.generate

      html(

build.sbt:
val scala3Version = "3.1.0"

lazy val root = project
  .in(file("."))
  .settings(
    name := "Pride and Joy",
    version := "0.1.0-SNAPSHOT",

    scalaVersion := scala3Version,

    libraryDependencies += "com.lihaoyi" % "cask_3" % "0.8.0",
    libraryDependencies += "com.lihaoyi" % "requests_3" % "0.7.0",
    libraryDependencies += "com.lihaoyi" % "scalatags_3" % "0.11.0",
    libraryDependencies += "com.lihaoyi" % "utest_3" % "0.7.10" % "test",
    libraryDependencies += "com.novocode" % "junit-interface" % "0.11" % "test",
  )

src/main/scala/Main:
//package app
import scalatags.Text.all._

object Main extends cask.MainRoutes{
  @cask.get("/")
  def hello() = {
    doctype("html")(
      html(
        body(
          h1("Hello World"),
          p("I am cow")
        )
      )
    )
  }

  initialize()
}

Is this something obvious, or should I stick with the older versions used in the demo?
Thanks!

I have also encountered this problem and it seems to be a regression in scalatags because it works fine on 2.13.x versions but not on 3.x.x versions. It seems that the protected[this] qualifier is no longer permitted in Scala 3 which might be related to this problem. From my admittedly limited time of debugging, the Self type inside scalatags.Text.TypedTag has issues with variance which is getting ignored by the @uncheckedVariance annotation. For Scala 2, the scalatags.text.TagFactory#tag function used to create tags such as html, a and so forth refines to ConcreteHtmlTag[String] but in Scala 3, it refines into <tag-name>.Self where <tag-name> is the name of a tag like html or a. This results in the type checking error that you have seen where Self cannot be resolved to text.Frag.
Since doctype is only present in the text packages, I do not think that this issue occurs in the js-dom packages however everything else related to variance most likely does...but I am not fully sure. Here is a slightly smaller reproducible example using mill as the build tool:
build.sc
import mill._
import mill.scalalib._

object example extends ScalaModule {
  // Replace this with Scala 2 or Scala 3 to view the difference.
  override def scalaVersion = "3.0.2"
  override def scalacOptions = T {
    super.scalacOptions() ++
      if (isScala3(scalaVersion)) {
        Seq("-explain")
      } else {
        Seq()
      }
  }
  override def ivyDeps = T {
    super.ivyDeps() ++ Agg(
      ivy"com.lihaoyi::scalatags:0.11.0"
    )
  }
}

example/src/Example.scala
package example

import scalatags.Text.all._

object Example {
  def main(args: Array[String]): Unit =
    println("hello world")

  // Fails in Scala 3.
  def example() =
    doctype("html")(
      html()
    )
}

(nice reduction and investigation - thanks)

Could you work around it by using asInstanceOf?

@edwardcwang sorry for the late reply but from what I remember, asInstanceOf worked in some places but it does not work as general solution. I only managed to realise that annotating the types explicitly worked because of some trial and error.


Fix trezor T multisig and tests

Fixes the Trezor T multisig tests. Part of this requires fixes to displayaddress and the Trezor implementation. The majority of the changes here are to make the multisig derivation paths conform to Trezor's purpose 48.
Also fixes the Coldcard multisig tests. Coldcard now disallows the same fingerprint being used multiple times in a multisig. We add a patch which enables this behavior so our tests work. Additionally the patch which sets the default multisigs is updated to use the new derivation paths that we need to use.

Merging as all of the tests that this is supposed to fix have been fixed.

Merging as all of the tests that this is supposed to fix have been fixed.


[MNT] Add new dependency: typing-extensions

Reference Issues/PRs

What does this implement/fix? Explain your changes.

This PR proposes to add a new dependency typing-extensions.
Does your contribution introduce a new dependency? If yes, which one?
This PR adds typing-extensions: https://pypi.org/project/typing-extensions/.
"The typing_extensions module serves two related purposes:
Enable use of new type system features on older Python versions. For example, typing.TypeGuard is new in Python 3.10, but typing_extensions allows users on previous Python versions to use it too.
Enable experimentation with new type system PEPs before they are accepted and added to the typing module.
typing_extensions is treated specially by static type checkers such as mypy and pyright. Objects defined in typing_extensions are treated the same way as equivalent forms in typing."
This dependency is maintained by the Python core team and can be found at: https://github.com/python/typing_extensions

Any other comments?
The reason I would like to add this as a core dependency is it will allow us to type our modules much better and will be able to use all the typing features that aren't avaliable in earlier versions of python.

PR checklist

For all contributions

[x] I've added myself to the list of contributors. Alternatively, you can use the @all-contributors bot to do this for you.
[x] The PR title starts with either [ENH], [MNT], [DOC], [BUG], [REF], [DEP] or [GOV] indicating whether the PR topic is related to enhancement, maintenance, documentation, bugs, refactoring, deprecation or governance.

For new estimators and functions

[x] I've added the estimator to the online API documentation.
[x] (OPTIONAL) I've added myself as a __maintainer__ at the top of relevant files and want to be contacted regarding its maintenance. Unmaintained files may be removed.

For developers with write access

[x] (OPTIONAL) I've updated aeon's CODEOWNERS to receive notifications about future changes to these files.

I also should note the addition to the .gitignore is intentional and is a config for a static type checker I run locally that can now make use of the new types


[4.x] Problem with displaying menu tips (Tooltip)

Tested versions
Reproducible in all versions starting from 4.0+, the problem is not observed in 3.x+
System information
Windows 10, Radeon RX 560 Series
Issue description
When we hover the mouse over the menu, a tooltip window appears; after the cursor leaves the button in the menu, the tooltip does not disappear until we leave the entire open menu area

Steps to reproduce

Click on the Debug menu at the top of the screen
Point your cursor at any menu where there is a hint, and wait until the hint appears
remove the cursor from the menu item

Minimal reproduction project (MRP)
Not required, too easy to test

Sounds like the problem that is fixed by https://github.com/godotengine/godot/pull/88392

Looks like a duplicate of https://github.com/godotengine/godot/issues/73926

Sounds like the problem that is fixed by #88392

no, in 4.3 dev4 this problem is still present

@akien-mga I think the topic should be opened, since in 4.3 dev4 this problem is still relevant

no, in 4.3 dev4 this problem is still present

Because the issue is not fixed yet, the PR is only open, not merged.

@akien-mga I think the topic should be opened, since in 4.3 dev4 this problem is still relevant

See:

Looks like a duplicate of #73926

We close duplicates of the prior issue to keep track of the bug and its resolution in a single place. #73926 is still open, you should subscribe to that issue for updates.


fixed react 18 types

children is no longer implicit. see https://github.com/DefinitelyTyped/DefinitelyTyped/pull/56210

Thank you!


Add user authentication and profile creation and maintenance

Add the ability for user to authenticate (log-in) to the chatbot so that we can track credit usage and also save and restore chat history.
See #11  for HF specific authentication.
Chainlit also supports authentication.

Addressed in #36 and SCRUM-18


*: highlight breaking change in compaction retention

Uber actually uses YAML config file, and this was an undocumented breaking change.
@fanminshi @joelegasse

lgtm after nits.


Improve docs for tasks that take toolsmiths-env argument

Describe how bbl-state input and BBL_STATE_DIR params interact with toolsmiths-env input
[#169967477]
What is this change about?
This change makes the documentation for tasks that take toolsmiths-env as input less confusing. Specifically, there is a note under the BBL_STATE_DIR param saying that if toolsmiths-env is specified as an input, BBL_STATE_DIR will be ignored.
Please provide contextual information.
https://www.pivotaltracker.com/story/show/169967477
Please check all that apply for this PR:

[ ] introduces a new task
[ ] changes an existing task
[ ] changes the Dockerfile
[ ] introduces a breaking change (other users will need to make manual changes when this is released)

Did you update the README as appropriate for this change?

[ ] YES
[X] N/A

How should this change be described in release notes?
Improve docs for tasks that take toolsmiths-env argument
What is the level of urgency for publishing this change?

[ ] Urgent - unblocks current or future work
[X] Slightly Less than Urgent

Tag your pair, your PM, and/or team!
@cloudfoundry/pcf-toolsmiths

LGTM - thanks!


[feat] Use PersistentVolumeClaims for consumption, export and trash directories of paperless-ngx

Is your feature request related to a problem? Please describe.
The consumption, export and trash directories currently only allow to be mounted to host paths:
consumption:
  # -- Enable the volume mount of a [consumption directory](https://paperless-ngx.readthedocs.io/en/latest/configuration.html#paths-and-folders).
  enabled: false
  # -- Mount path of the consumption directory inside the container.
  mountPath: /consumption
  # -- Host path of the consumption directory outside the container.
  hostPath: ''

This is not practical for Kubernetes because when using NAS etc. for backups, the directories first need to be mounted on all nodes.
Describe the solution you'd like
I would like an option to specify an existing PVC to mount these at, like:
consumption:
  # -- Enable the volume mount of a [consumption directory](https://paperless-ngx.readthedocs.io/en/latest/configuration.html#paths-and-folders).
  enabled: true
  # -- Mount path of the consumption directory inside the container.
  mountPath: /consumption
  # -- Host path of the consumption directory outside the container.
  hostPath: ''
  # -- PVC of the consumption directory
  persistentVolumeClaim: 'my-consumption-claim'

Describe alternatives you've considered
Using some automation technology (like Ansible) to mount the directory to each node, but this opposes the idea of Kubernetes.

Hi @rabejens,
Thanks for opening an issue! I will consider adding this feature.
But I'm unsure if this really is necessary or helpful as the consumption / trash directories are short-lived temporary directories (at least in my opinion?) which are used from external, e.g. SFTP to the cluster node.
Also I don't see an easy way where someone would use these directories through PVCs, but please correct me if I'm wrong. 😊
BR, Pascal

The consumption / export dirs could be on a NAS. The clean K8S way would be, to my understanding, to have K8s manage mounting of these dirs instead of having it to do beforehand with methods like Ansible to have them available on every node at the same path.
I was not clear enough, it doesn't have to be a PVC. It could also be a Kubernetes volume specification as it is done when creating Pods under the volumes section.
So, to clarify things, I'd like an option to not only specify a host path for these directories, but also to specify a complete volume so I could be able to mount NAS shares.


Doesn't compile with Chia 2.3.0

Hello,
The reference pool doesn't compile anymore with the last version of chia (2.3.0) because the chia.types.Announcement no longer exists.
It's used in the create_absorb_transaction method in singleton.py.
I tried to simply remove the announcements, and directly pass the coin list in create_signed_transaction, but the pool claim then fail with an error ("No key for puzzle hash", I attach the log).
Any hint how to fix the singleton.py  for chia 2.3.0?
err.txt

Hi @mariano54,
Quick question on an update pool from 2.1.x to 2.3.x
In this code, need to replace Announcement by CreateCoinAccouncement?
transaction = await wallet['rpc_client'].create_signed_transaction(
            additions=[{'puzzle_hash': wallet['puzzle_hash'], 'amount': 0}],
            tx_config=DEFAULT_TX_CONFIG,
            coins=[spend_coin],
            coin_announcements=[Announcement(p2_coin.name(), b"$")],
            fee=uint64(1),
)

Same here?
transaction = await wallet['rpc_client'].create_signed_transaction(
            additions=[{'puzzle_hash': wallet['puzzle_hash'], 'amount': spend_coin.amount - fee}],
            tx_config=DEFAULT_TX_CONFIG,
            coins=[spend_coin],
            coin_announcements=[Announcement(p2_coin.name(), b"$")],
            fee=uint64(fee),
)

And here, it's AssertCoinAnnouncement?
solution: Program = w.make_solution(
        primaries=primaries,
        coin_announcements={message},
        fee=fee,
        coin_announcements_to_assert={Announcement(p2_coin.name(), bytes(b"$")).name()},
)

Thanks


OS X Big Sur - Minor issues

The new rounded windows cut off at the bottom corners of a terminal, making the corner tiles partially rendered.
Getting reports of (apparently harmless) log spam:
2020-11-19 15:56:29.649 dungeoncrawl[23603:3643367] NSKeyBindingManager: Bad key binding atom for '$\UF72B' = 'moveToEndOfLineAndModifySelection:'
2020-11-19 15:56:29.650 dungeoncrawl[23603:3643367] NSKeyBindingManager: Bad key binding atom for '$\UF729' = 'moveToBeginningOfLineAndModifySelection:'
2020-11-19 15:56:29.650 dungeoncrawl[23603:3643367] NSKeyBindingManager: Bad key binding atom for '\UF72B' = 'moveToEndOfLine:'
2020-11-19 15:56:29.650 dungeoncrawl[23603:3643367] NSKeyBindingManager: Bad key binding atom for '\UF729' = 'moveToBeginningOfLine:'
2020-11-19 15:56:29.650 dungeoncrawl[23603:3643367] NSKeyBindingManager: Bad key binding atom for '\UF72C' = 'pageUp:'
2020-11-19 15:56:29.650 dungeoncrawl[23603:3643367] NSKeyBindingManager: Bad key binding atom for '\UF72D' = 'pageDown:'

The latter is probably a winit update. The former may require adding a gutter.

A friend tells me that this working now.

(Sorry, commented on wrong issue)

#239 now implements a gutter system. It defaults to 8 pixels (enough to fix the rounding issue on my Macbook Air), and adds the with_gutter command to initialization if you need a different number. So this should be fixed now.


Two ingests have failed to store: b21330724 v15 and v16

We had two ingests fail to store last week; we should investigate and work out what went wrong, and how to store them successfully.

57b406bb-4367-49fb-9131-3e8737d20f84 – digitised/b21330724/v15
3a1f9d56-d2f4-403b-9e6a-783b7661011d – digitised/b21330724/v16

The "jump to dev logs" link was broken – it was still pointing at the old Firelens index. I've fixed it here: https://github.com/wellcomecollection/ingest-inspector/commit/1d3ecf1339507526b4b1b238c55480904f1c64f1

Successfully stored!


Package request: Watir

http://watir.github.io/
An open source Ruby library for automating tests. Watir interacts with a browser the same way people do: clicking links, filling out forms and validating text.

It appears that Watir is also published as a ruby gem and is supposed to be used as a DSL. Are you familiar with the ruby ecosystem?
You could package it yourself locally with the following few steps.

Create a Gemfile containing

source "https://rubygems.org"
gem "watir"

Resolve all ruby dependencies with bundler. This generates a Gemfile.lock file

`nix-build '<nixpkgs>' -A bundler`/bin/bundler lock

Run bundix to create a gemset.nix file from Gemfile.lock

`nix-build '<nixpkgs>' -A bundix`/bin/bundix

Use it in a nix expression (skeleton provided below)

with import <nixpkgs> {};
{
  env = bundlerEnv {
    name = "watir-env";
    gemfile = ./Gemfile;
    lockfile = ./Gemfile.lock;
    gemset = ./gemset.nix;
  };
}

For a walkthrough see e.g. http://blog.arkency.com/2016/04/packaging-ruby-programs-in-nixos/

@0xABAB if you need some help to get this done, I'd be willing to go throuh this together.

Thanks for your request. If you like to get a package in, then the best approach would be to roll up your sleeves and open a PR! :) People on this tracker and the NixOS IRC channel are typically quite willing to help out, and especially when discussing a PR.


Updated zh-CN translation text

Supplement and translate the missing KEY, and polish part of the Chinese translation.

Thanks for the contribution!
@all-contributors please add @jimcat8 for translation.


Consider making take, drop, ... curried on streams

Hi and thanks for sharing this library, it looks awesome ...
I was giving it a try and found that stream.take function  (and others) take 2 arguments, so id doesn't look too nice with pipe:
import { effect as T, stream as S } from "@matechs/effect"
import { pipe } from "fp-ts/lib/pipeable"

pipe(
  S.periodically(100),
  S.chain(n => S.encaseEffect(T.pure(n))),
  s => S.take(s, 100) // -> S.take(100)
)

Did you consider changing signature to (n: number) => <S>(stream: S) => ...? If so - what are the drawbacks of doing so?
Thanks!

Thanks for spotting it, they should definately be curried

released in v3.0.0 (major simply because non backward compatible at the api level)

Yeah, definitely a breaking change ..
Do you keep tracking of such issues in other place? It would be nice to be able to monitor the progress in a public place (a github milestone or something ..)


Investigate string function

This function needed a cast in order for master to compile, there was a PR for introducing this. @karroffel mentioned he wanted a closer look so just raising this to make sure it's not forgotten.
core/String.cpp
wchar_t &String::operator[](const int idx) {
	return *const_cast<wchar_t *>(godot::api->godot_string_operator_index(&_godot_string, idx));
}

In the last PR I mentioned that we could scipp the const_cast if we changed the return type to const wchar_t &
const wchar_t &String::operator[](const int idx) { 	return godot::api->godot_string_operator_index(&_godot_string, idx); }
Might be better. I have not had any issues. I do not do mush string manipulations though.


JWT public key

Improves documentation about JWT in Zowe APIML and adds information about the new public key endpoint.
Is there a related issue for this PR?
Issue number: https://github.com/zowe/api-layer/pull/561
Developer's Certificate of Origin (DCO)
Signed-off-by: Petr Plavjanik<EMAIL_ADDRESS>

@JirkaAichler, you suggestion has been applied. I have merged it with current doc-staging. @janan07, please review and merge into docs-staging. The functionality is already implemented.


Do not filter duplicates

Fixes #37
removes filtering of duplicates with extended advertisement format (like with the normal advertisements)

Sorry. I was really busy. I was planning to look into this WE. Thanks for the PR. That make my life easier. Will check it before I push a new version.


Tighten text contrast bounds, fall back to larger bounds

Fixes the issues outlined in https://github.com/flutter/flutter/pull/100267
Historically, we implemented text contrast checking assuming that we were only really looking at text color in comparison to a larger background surface. Thus it made sense to expand the area sampled from slightly to ensure that we picked up enough of this color - especially considering that flutter test will use ahem by default which is quite large and blocky.
Nevertheless, there are some cases where this doesn't work well -  such as through the usage of the text background color which is extremely tight to the text bounds themselves.
To adjust for this, we can expand the heuristic to first consider tighter bounds, and then if it fails to find multiple colors, fall back to the historical larger bounds.
This also takes an opportunity to correct two instances of > that should be >= when determining minimum text size

This may have difficulty rolling into google3, if we were incorrectly passing some instances before. Nevertheless, I'll have to investigate these failures since it could just be failures or the new heuristic.

actually this won't quite work with text-aa. Maybe instead I should check if there is a background color on the text object (since there is no other way for that to get painted) and select the rect based on that

(actually investigating more, this may be working correctly)....


[QUESTION] Use class members in AfterImport function

Prerequisites

[ x] Have read and understood: https://github.com/Maatwebsite/Laravel-Excel/blob/3.1/.github/SUPPORT.md
[ x] Checked if your Laravel Excel version is still supported: https://docs.laravel-excel.com/3.1/getting-started/support.html#supported-versions
[x ] Checked that your question isn't already asked before.
[x ] Filled in the entire issue template

Versions

PHP version: 7.4.6
Laravel version: 7.10.3
Package version: ^3.1

Description
I need to send an email with new records inserted after the import.
I can not do that cause the afterImport method is static and it can not access members of the class.
This is the class I use.
<?php
namespace App\Imports;

use Maatwebsite\Excel\Concerns\ToModel;
use Maatwebsite\Excel\Concerns\WithEvents;
use Maatwebsite\Excel\Concerns\RegistersEventListeners;

class MyImport implements ToModel, WithEvents
{
    use RegistersEventListeners;

    private $emails;

    public static function afterImport(AfterImport $event)
    {
        // Here I need to send email to $emails but I can not access to $emails member cause afterImport is static
    }

    /**
    * @param array $row
    *
    * @return \Illuminate\Database\Eloquent\Model|null
    */
    public function model(array $row)
    {
        // I insert my model

        $this->emails[] =<EMAIL_ADDRESS>    }
}

You can access the import via the event. Do a dd($event) and see the information you get


az image create documentation insufficient/out of date

Description

Documentation for az image create (https://docs.microsoft.com/en-us/cli/azure/image) and its help strings do not provide sufficient context to enable users to decide when to use the parameters (e.g. how does --source and --data-disk-sources differ?
Examples reference non-existent parameter --data-snapshot
Only -n and -g are marked as required flags, some invocations fail with NoneType error e.g. az image create -n foo -g bar --data-disk-sources mdname --os-type windows:

usage: az image create [-h] [--output {json,tsv,table,jsonc}] [--verbose]
                       [--debug] [--query JMESPATH] --resource-group
                       RESOURCE_GROUP_NAME --name NAME
                       [--os-type {Windows,Linux}] [--location LOCATION]
                       [--source SOURCE]
                       [--data-disk-sources DATA_DISK_SOURCES [DATA_DISK_SOURCES ...]]
                       [--tags [TAGS [TAGS ...]]]
az image create: error: 'NoneType' object has no attribute 'lower'

Providing --source appears to suppress the error.

Environment summary
Install Method: How did you install the CLI? (e.g. pip, interactive script, apt-get, Docker, MSI, nightly)
Answer here: pip3
CLI Version: What version of the CLI and modules are installed? (Use az --version)
Answer here:
azure-cli (2.0.7)

acr (2.0.5)
acs (2.0.7)
appservice (0.1.7)
batch (3.0.0)
billing (0.1.0)
cdn (0.0.3)
cloud (2.0.3)
cognitiveservices (0.1.3)
command-modules-nspkg (2.0.0)
component (2.0.5)
configure (2.0.7)
consumption (0.1.0)
core (2.0.7)
cosmosdb (0.1.7)
dla (0.0.7)
dls (0.0.7)
documentdb (0.1.5)
feedback (2.0.3)
find (0.2.3)
interactive (0.3.3)
iot (0.1.6)
keyvault (2.0.5)
lab (0.0.5)
monitor (0.0.5)
network (2.0.7)
nspkg (3.0.0)
profile (2.0.5)
rdbms (0.0.2)
redis (0.2.4)
resource (2.0.7)
role (2.0.5)
sf (1.0.2)
sql (2.0.4)
storage (2.0.7)
vm (2.0.7)

Python (Darwin) 3.6.1 (default, Apr  4 2017, 09:40:21)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)]

Python location '/usr/local/opt/python3/bin/python3.6'

OS Version: What OS and version are you using?
Answer here: MacOS 10.12.4
Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
Answer here: Bash

From your output, there is a validation bug--not just a documentation problem.

Documentation on az disk faces similar issues, notably both image and disk state that --source can be a SAS URI, but when I supply the HTTPS SAS URI generated for a Managed Disk, it fails to parse.

Another issue, trying to migrate a snapshot from one RG to another in the same sub by creating a new snapshot pointing to the same disk:
az snapshot create -g stad-redblueDeploy1 -n $snapshotName --source $snapshotId

just returns "The entity was not found." despite $snapshotId being a snapshot ID (e.g. /subscriptions/GUID/resourceGroups/RG_NAME/providers/Microsoft.Compute/snapshots/SNAPSHOT_NAME) per current docs.

Running the same command line but with az image:
az image create -g rg_name -n $snapshotName --source $snapshotId --os-type windows

just yeilds
An internal execution error occurred.

Documentation on az disk faces similar issues, notably both image and disk state that --source can be a SAS URI, but when I supply the HTTPS SAS URI generated for a Managed Disk, it fails to parse.

I will update the help to be unmanaged blob uri, SAS is not a right term

Another issue, trying to migrate a snapshot from one RG to another in the same sub by creating a new snapshot pointing to the same disk:

You have to supply the argument value of:
--source-storage-account-id   : Used when source blob is in a different subscription.

An internal execution error occurred.
It is a known bad server error for unmatched regions.  Let me contact CRP team for the progress/plan to fix this


Link to Visual Studio 2022 Remote toools broken

Unable to download Remote Tools for VS2022

@heretic13 -- Yaroslav, please try this link:

Download and Install the remote tools

I am closing this issue now.  You are welcome to @ mention me for any followup.  We hope to hear from you again.

This link not work. It's redirect me to https://visualstudio.microsoft.com/ru/
Tested on Chromium, Edge, Tor on two computers.

Worked after switching Tor to English.
Blocking the download based on the language in the browser?
Interesting. Is this an official Microsoft policy???

@heretic13 -- Yaroslav, I'm glad it is working for you.  I changed the language identifier in my URL from en-us to ru-ru and got the correct download page in Russian:

https://learn.microsoft.com/ru-ru/visualstudio/debugger/remote-debugging?view=vs-2022

This page (https://learn.microsoft.com/ru-ru/visualstudio/debugger/remote-debugging?view=vs-2022) contains a description of the tools and a lot of other documentation.
She's fine.
It contains the same download link - https://visualstudio.microsoft.com/downloads/#remote-tools-for-visual-studio-2022
And this page makes a redirect, provided that the language in the browser is Russian (not the URL, but the language of the browser).
I just changed the page priority in Edge. I made it so that pages in English are shown first, and only then pages in Russian. After that, the tool download page opened normally.
However, the fact of discrimination based on language has been proven (at least for me). :)
Perhaps another department (not the documentation department) deals with problems with Microsoft download servers, but you could send a bug report to them.


Replace go-swagger with gin-gonic for backend server.

Go-swagger, although a great tool, adds too much code bloat and confines all the business logic to the configure_xxx.go file.
Creating new flag options is also much harder with go-swagger.
Switch to gin-gonic as the backend framework. Gin gonic should provide more control over the server logic and allow easy integration.

Tracked in branch https://github.com/nitishm/vegeta-server/tree/gin-server


DM-42147 (cont.): Bump yaml-cpp version.

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

This should resolve a conflict with libmamba brought in by rubin-env-rsp.

@conda-forge-admin, please rerender


add Lesley's details

Hopefully the speaker images show up this time, please have a look to see if they do.

@om-henners, please merge this pull request. Also, please check to see if the image links are working...I think they are.

merged!

Thanks @FidelChe. The speaker photos are finally showing up.

Yes, tonnes of thanks @jonah-sullivan


Flow can't find module

I tried to resolve module with Trouble Shooting. But it absolutely doesn't work.
./src/.flowconfig has option:
[include]
../node_modules/

I installed sequelize module and test flow:

I saw the module is installed correctly. My environment was OSX El Cap latest.

I believe the issue here is that Flow is not able to find node modules above the "root" where the .flowconfig file is. Can you move your .flowconfig up one directory?

I moved my .flowconfig file up one directory.
[2016-04-17 11:09:51] Initializing Server (This might take some time)
[2016-04-17 11:09:51] executable=/usr/local/Cellar/flow/0.22.0/bin/flow
[2016-04-17 11:09:51] version=0.22.0
[2016-04-17 11:09:51] Parsing
/Users/maniax/GameRecommend/node_modules/ No such file or directory
[2016-04-17 11:09:51] Building package heap
[2016-04-17 11:09:51] Running local inference
[2016-04-17 11:09:52] Re-resolving directly dependent files
[2016-04-17 11:09:52] Calculating dependencies
[2016-04-17 11:09:52] Merging
[2016-04-17 11:09:52] Done
[2016-04-17 11:09:52] Server is READY
[2016-04-17 11:09:52] Took 0.546391 seconds to initialize.

I removed the relative path ../node_modules -> ./node_modules. Then it worked well, but not able to resolve node_modules.
[2016-04-17 11:10:57] Initializing Server (This might take some time)
[2016-04-17 11:10:57] executable=/usr/local/Cellar/flow/0.22.0/bin/flow
[2016-04-17 11:10:57] version=0.22.0
[2016-04-17 11:10:57] Parsing
[2016-04-17 11:10:57] Building package heap
[2016-04-17 11:10:57] Running local inference
[2016-04-17 11:10:57] Re-resolving directly dependent files
[2016-04-17 11:10:57] Calculating dependencies
[2016-04-17 11:10:57] Merging
[2016-04-17 11:10:57] Done
[2016-04-17 11:10:57] Server is READY
[2016-04-17 11:10:57] Took 0.594378 seconds to initialize.
[2016-04-17 11:10:57] Status: Error
  2: import Sequelize from 'sequelize';
                           ^^^^^^^^^^^ sequelize. Required module not found

sequelize module is installed correctly. I don't know what's wrong.

@samwgoldman I think time doesn't solve this.
Let me brief my situationmore detail to you.
I installed<EMAIL_ADDRESS>globally via npm.
Directory map of my project is:
.
├── LICENSE
├── README.md
├── package.json
├── src
│   ├── config.js
│   ├── constants.js
│   ├── core.js
│   └── index.js
├── tests
│   ├── helper.js
│   └── test.spec.js
└── tools
    ├── config.js
    ├── run.js
    ├── tasks
    │   ├── build.js
    │   ├── clean.js
    │   └── compile.js
    └── util.js

dependencies of package.json:
{
 ...,
  "dependencies": {
    "babel-polyfill": "^6.7.4",
    "pg": "^4.5.3",
    "pg-hstore": "^2.3.2",
    "pg-native": "^1.10.0",
    "sequelize": "^3.21.0",
    "source-map-support": "^0.4.0"
  }

My .flowconfig is:
[include]
./node_modules/

[ignore]
.*/node_modules/.*
.*/lib/.*
.*/tests/.*

[options]
module.ignore_non_literal_requires=true

I installed npm dependencies correctly.
And it still spits:
src/core.js:3
  3: import Sequelize from 'sequelize';
                           ^^^^^^^^^^^ sequelize. Required module not found

What did I set wrong?

I think I found my fault.

@Beingbook so, what was it?

@rmakarov94 I just added node_modules to ignore. It should place include. That was my mistake.


Partially parsing of large GPL file

Hi,
I sometimes want to parse large GPL files (e.g., GPL570), but my PC does not work due to out of memory. So, I'd like to be able to parse the GPL file partially by specifying the GSM samples to parse from the GPL. If you agree with my idea, I will make a pull request for this feature.
Thanks

Hi, I was on holidays with limited access to net. Yes, this could be a good feature and if you would like to do it - that's even more awesome.

Hi,
Thank you for your reply despite you are on holidays. I will make a pull request for the feature soon.
Enjoy your vacation!