Sample code: Player Points & Recent Activities

As I mentioned in my other recent post about free agents, last year I created a local copy and made some additions. Dusting it off for this season and just came across this set of code. Thought you might like to see it. Ultimately, I think the most valuable aspect of this is the ability to go from: string to Player Object to Player Points. Is there any other way to hit the api with a string and return the respective players points week-over-week? Let me know if you have any questions on all this and feel free to incorporate as you'd like. Would also love to know if there's a better way to go about what I did. Thanks!
Sample Outcome:

recent_activity returns actions where player is not a string, but a Player Object
The player Object includes a points attribute which is derived from a fetch_player_points method
player.points shows current week points and last week points (there's probably more to unpack here)

Activity Class: includes a Player Object in actions, where Player includes points as derived from fetch_player_points
class Activity(object):
    def __init__(self, data, player_map, get_team_data, get_player_data, fetch_player_points):
        self.actions = []  # List of tuples (Team, action, player)
        for msg in data['messages']:
            team = ''
            action = 'UNKNOWN'
            player = ''
            msg_id = msg['messageTypeId']
            if msg_id == 244:
                team = get_team_data(msg['from'])
            elif msg_id == 239:
                team = get_team_data(msg['for'])
            else:
                team = get_team_data(msg['to'])
            if msg_id in ACTIVITY_MAP:
                action = ACTIVITY_MAP[msg_id]
            if msg['targetId'] in player_map:
                player = player_map[msg['targetId']]
            self.date = msg['date']
            player = Player(get_player_data(msg['targetId']),fetch_player_points)
            self.actions.append((team, action, player))

    def __repr__(self):
        return 'Activity(' + ' '.join("(%s,%s,%s)" % tup for tup in self.actions) + ')'

Player Class: fetch_player_points as seen in Sample outcome
def fetch_player_points(self, playerId: int):
        """Retrieves a player's points"""

        params = {
            'scoringPeriodId': 5,
            'view': 'kona_player_info',

        }
        filters = {"players": {
            "filterIds": {"value": [playerId]},
            "sortPercOwned": {"sortPriority": 2, "sortAsc": False},
            "filterStatsForTopScoringPeriodIds": {"value": 3,
                                                  "additionalValue": ["002019", "102019", "002018", "1120195",
                                                                      "022019"]}}}
        headers = {'x-fantasy-filter': json.dumps(filters)}
        r = requests.get(self.ENDPOINT, params=params, cookies=self.cookies, headers=headers)
        data = r.json()

        stats = data['players'][0]['player']['stats']
        player_points = {}
        for stat in stats:

            if stat['id'] == str('00') + str(self.year):
                player_points['points_season_total'] = stat['appliedTotal']
                player_points['points_season_average'] = stat['appliedAverage']
            if stat['id'] == str('00') + str(self.year-1):
                player_points['points_season_last'] = stat['appliedTotal']
            if stat['scoringPeriodId'] == self.current_week:
                player_points['points_week_current'] = stat['appliedTotal']
            if stat['scoringPeriodId'] == self.current_week - 1:
                player_points['points_week_last'] = stat['appliedTotal']

        return player_points

class Player(object):
    """Player are part of team"""

    def __init__(self, data, fetch_player_points):
        self.name = json_parsing(data, 'fullName')
        self.playerId = json_parsing(data, 'id')
        self.posRank = json_parsing(data, 'positionalRanking')
        self.eligibleSlots = [POSITION_MAP[pos] for pos in json_parsing(data, 'eligibleSlots')]
        self.acquisitionType = json_parsing(data, 'acquisitionType')
        self.proTeam = PRO_TEAM_MAP[json_parsing(data, 'proTeamId')]
        self.positionalRanking = json_parsing(data, 'positionalRanking')
        self.totalRanking = json_parsing(data, 'totalRanking')
        self.totalRating = json_parsing(data, 'totalRating')

        self.fetch_player_points = fetch_player_points
        # Get player's season points
        self.points = fetch_player_points(self.playerId)

League Class:
recent_activity
activity = [Activity(topic, self.player_map, self.get_team_data,self.get_player_data,self.fetch_player_points) for topic in data]
def get_player_data(self, playerId: int):
        """Retrieves a player data from api"""

        params = {
            'scoringPeriodId': 5,
            'view': 'kona_player_info',

        }
        filters = {"players": {
            "filterIds": {"value": [playerId]},
            "sortPercOwned": {"sortPriority": 2, "sortAsc": False},
            "filterStatsForTopScoringPeriodIds": {"value": 3,
                                                  "additionalValue": ["002019", "102019", "002018", "1120195",
                                                                      "022019"]}}}
        headers = {'x-fantasy-filter': json.dumps(filters)}
        r = requests.get(self.ENDPOINT, params=params, cookies=self.cookies, headers=headers)
        data = r.json()['players'][0]['player']

        return data

This is great, thanks for posting your updates! I will definitely look into adding player object to recent_activity and see what data can be pulled!

@rysull23 I just asked a very similar question in another issue; trying to get the total player points for the season.
Would it be possible for you to share the full player.py?

Currently seeing the following error which I think is basically related to me not knowing exactly where to place the fetch_player_points def
Traceback (most recent call last):
  File "espn_ff.py", line 12, in <module>
    league = League(league_id=league_id, year=year, espn_s2=espn_s2, swid=swid)
  File "/usr/local/lib/python3.8/site-packages/espn_api/football/league.py", line 21, in __init__
    self._fetch_league()
  File "/usr/local/lib/python3.8/site-packages/espn_api/football/league.py", line 28, in _fetch_league
    self._fetch_teams(data)
  File "/usr/local/lib/python3.8/site-packages/espn_api/football/league.py", line 33, in _fetch_teams
    super()._fetch_teams(data, TeamClass=Team)
  File "/usr/local/lib/python3.8/site-packages/espn_api/base_league.py", line 63, in _fetch_teams
    self.teams.append(TeamClass(team, roster=roster, member=member, schedule=schedule))
  File "/usr/local/lib/python3.8/site-packages/espn_api/football/team.py", line 33, in __init__
    self._fetch_roster(roster)
  File "/usr/local/lib/python3.8/site-packages/espn_api/football/team.py", line 44, in _fetch_roster
    self.roster.append(Player(player))
TypeError: __init__() missing 1 required positional argument: 'fetch_player_points'

In the latest package (0.8.0) I added total_points and projected_total_points to Player Class and now recent_activity returns Player Object instead of string

Hey @glennrfriedman - Looks like I stuck the fetch_player_points method as a required input variable into the Player Object init function. You can see below, for example, that in the get_free_agents call, the method is passed into the Player object. It would also seem I stuck it into get_teams and get_roster in the Team object...I seem to have forced it in all over the place!
Let's see how @cwendt94 added this, as I'm sure he found a far more elegant way. Thanks for adding! Super cool. Any luck with the other stats, like last_week_points, etc? It was a bit cryptic but seemed to be a lot of stats you can extract.
from .constant import POSITION_MAP, PRO_TEAM_MAP
from .utils import json_parsing
from .tiers import PLAYER_TIER_MAP
from .league import requests, json

class Player(object):
    """Player are part of team"""

    def __init__(self, data, fetch_player_points):
        self.name = json_parsing(data, 'fullName')
        self.playerId = json_parsing(data, 'id')
        self.posRank = json_parsing(data, 'positionalRanking')
        self.eligibleSlots = [POSITION_MAP[pos] for pos in json_parsing(data, 'eligibleSlots')]
        self.acquisitionType = json_parsing(data, 'acquisitionType')
        self.proTeam = PRO_TEAM_MAP[json_parsing(data, 'proTeamId')]
        self.positionalRanking = json_parsing(data, 'positionalRanking')
        self.totalRanking = json_parsing(data, 'totalRanking')
        self.totalRating = json_parsing(data, 'totalRating')

        self.fetch_player_points = fetch_player_points

        # Get players main position
        for pos in json_parsing(data, 'eligibleSlots'):
            if (pos != 25 and '/' not in POSITION_MAP[pos]) or '/' in self.name:
                self.position = POSITION_MAP[pos]
                break

        # Get player's season points
        self.points = fetch_player_points(self.playerId)

    def __repr__(self):
        return 'Player(%s|%s)' % (self.name, self.position)

def get_free_agents(self, week: int=None, size: int=50, position: str=None) -> List[Player]:
        """Returns a List of Free Agents for a Given Week"""

        if self.year < 2018:
            raise Exception('Cant use free agents before 2018')
        if not week:
            week = self.current_week

        slot_filter = []
        if position and position in POSITION_MAP:
            slot_filter = [POSITION_MAP[position]]

        params = {
            'view': 'kona_player_info',
            'scoringPeriodId': week,
        }
        filters = {"players":{"filterStatus":{"value":["FREEAGENT","WAIVERS"]},"filterSlotIds":{"value":slot_filter},"limit":size,"sortPercOwned":{"sortPriority":1,"sortAsc":False},"sortDraftRanks":{"sortPriority":100,"sortAsc":True,"value":"STANDARD"}}}
        headers = {'x-fantasy-filter': json.dumps(filters)}

        r = requests.get(self.ENDPOINT, params=params, cookies=self.cookies, headers=headers)
        self.status = r.status_code
        checkRequestStatus(self.status)

        players = r.json()['players']

        return [Player(player,self.fetch_player_points) for player in players]

def _fetch_teams(self):
        """Fetch teams in league"""

        ...

        for team in teams:
            for member in members:
                # For league that is not full the team will not have a owner field
                if 'owners' not in team or not team['owners']:
                    member = None
                    break
                elif member['id'] == team['owners'][0]:
                    break
            roster = team_roster[team['id']]
            self.teams.append(Team(team, roster, member, schedule,self.fetch_player_points))

       ...

class Team(object):
    """Teams are part of the league"""
    def __init__(self, data, roster, member, schedule,fetch_player_points):

    def _fetch_roster(self, data):
        '''Fetch teams roster'''
        self.roster.clear()
        roster = data['entries']

        for player in roster:
            self.roster.append(Player(player, self.fetch_player_points))

I like the idea of adding totalRanking and totalRating to the player class as well. The last week points can be found in the stats field for the Player. I was thinking keeping week points general and just having it be keyed in the stats dict or can always get box scores for a specific week. So you can always do something like
stats = league.teams[0].roster[0].stats
stats[league.current_week - 1]
# or
box_scores = league.box_scores(week=2)

One other field I was thinking of adding to the Player class and might be useful is having last_year_points and last_year_projected_points. I will have to double check but I believe those values are usually always in the API data returned.


Beep sound is back! on selected text regions.

Steps to reproduce:

Select a region of test (using shift, or dragging with the mouse).
Press any letter on the keyboard.

Effects:
The infamous system beep bug.
This happens even though #9 appears fixed here. Using 0.0.6 (latest release).

Yep, selecting the text does it for me. ⌘ + A doesn't trigger it though.

@laserbeam You're right. I based it on the first character, but when you select it in reverse it will have the opposite direction. Let me find a patch now :+1:


Clojure example for transaction execution is outdated

The example tries to match on the error string whether it contains "restart transaction", instead of checking if the error code is 40001 or not
ping @knz

Ok there are two issues here:

the doc is wrong: at https://www.cockroachlabs.com/docs/transactions.html#transaction-retries the error code is indeed 40001 (Serialization Failure) but the error message should be either restart transaction or retry txn. There is no message retry transaction. We really recommend checking if the message begins with "restart transaction:". For example a full message would be: ERROR: restart transaction: retry txn "sql/executor.go:465 sql txn" id=93a60bd3 ...

the Clojure example is using the error message (correctly) instead of the error code (which could be faster). Also it doesn't retry in the way recommended by the doc. So clearly the Clojure example can be improved.

I'll have a look at the Clojure code but in the meantime @kosii you can assume it's not completely wrong (it will actually do 80% of the right thing), and you can also compare with the other examples which are also OK.

My problem then is that my error message starts with restart transaction, but PSQLException.getErrorCode returns 0. The error message has the priority in such cases?

I am surprised the error code is 0. Would you be ok with sharing a small example that demonstrates this?

totally, but I need to reduce first the code needed to reproduce it to a manageable size

@knz I created a project to reproduce the exception with error code 0. To run type ./sbt run.
On the other hand in a real project it makes no sense to wrap the query in a Try, because it will just make impossible for DB.localTx to detect if a rollback is necessary by hiding the throw from the statement.

Thanks! We'll have a look.
I've logged https://github.com/cockroachdb/cockroach/issues/9317 to look into this as this is not strictly a Clojure issue any more.

thanks, I subscribed to the other issue too to be notified if more information is necessary

@knz, any updates here?

No update yet, but it's rather high on my to-do list still.

Update on the Clojure code: I wrote a piece of code which successfully uses the retry protocol but is actually detrimental to performance, see https://github.com/cockroachdb/cockroach/issues/5935.
Until the performance issue is investigated I'd recommend not advertising this method for Clojure yet.

@knz we do expect worse performance with the retries, although it shouldn't be too significant. The point of the retries isn't higher performance, it's to prevent starvation. Let's get these issues closed.


Update README.md

Fully updated readme.md to its completely new version as I stated
Closes #275

@Kalivarapubindusree have a look on this pr , if any changes needed please let me know


Updating alert text for clarity

@geurts that visual-diff test that failed is flakey. I suggest just rerunning the visual-diff tests.


Saved Searches Error

When trying to save a "Saved Search" there is an error message.
Question: do saved searches have to be from the public url to be viewable to the public?
Another Question: The field I searched on for the saved search should not be viewable to the public - is that going to pose an issue?

@emjohnst - Can you please send me the information for the search you were trying to create by email (URL, description, image if you're trying to use one)

@bferguso Here is the link to the search: https://apps.nrs.gov.bc.ca/int/bcrhp/search?paging-filter=1&tiles=true&format=tilecsv&reportlink=false&precision=6&total=56&advanced-search=[{"op"%3A"and"%2C"6cc30f6e-0d06-11ed-8804-5254008afee6"%3A{"op"%3A"eq"%2C"val"%3A""}%2C"6cc30d5c-0d06-11ed-8804-5254008afee6"%3A{"op"%3A"eq"%2C"val"%3A""}%2C"1f28339e-3b93-11ee-b4c5-080027b7463b"%3A{"op"%3A""%2C"val"%3A""}%2C"50c26f28-3b93-11ee-ac05-080027b7463b"%3A{"op"%3A""%2C"val"%3A""}%2C"6cc30b36-0d06-11ed-8804-5254008afee6"%3A{"op"%3A"~"%2C"lang"%3A"en"%2C"val"%3A""}%2C"6cc30c44-0d06-11ed-8804-5254008afee6"%3A{"op"%3A"~"%2C"lang"%3A"en"%2C"val"%3A"japanese canadian recognition project"}}]

Description: Japanese Canadian Historic Places
The 56 places received provincial recognition through the Province of British Columbia’s “Japanese Canadian Historic Places Recognition Project”. They were selected by community and sector experts for their importance to the Japanese Canadian community and contributions to the history and development of the province. Together, the sites help tell the story of Japanese Canadians across the province, and give insight into significant moments in BC’s history.

This appears to be related to adding the image

This was caused by the fact that system settings graph doesn't have a graph slug. Setting null graph slug to system_settings so the images can be saved properly in s3.

@bferguso I believe this is complete now?


Add support for coming spread syntax

Bug Report

TSLint version: 4.0.0-dev.1
TypeScript version: 2.2.0-dev.20161113
Running TSLint via: vim

TypeScript code being linted
const Component: React.StatelessComponent<Props> = ({ ...rest }) => <div {...rest } />;

Actual behavior
Missing semicolon (semicolon) [typescript/tslint]
Expected behavior
Pass

Oops I figured that my vim used a different tslint version. Sorry for the noise. Closing this for now.


Do not rely on implicit NULLCMD=cat

Thank you! I have cherry-picked onto develop branch as 8ae0283


when I completed the process of building Project, I can run it on my device, but there is a error

Error:Unable to find module with Gradle path 'plugins/edx-app-gradle-plugin' (needed by module 'buildSrc'.)

AFAIK this has existed for a long time but we haven't run into any problems with it.

Looks like #513 tried to address this.

when i run android studio, during the build i get this error:
Error:Execution failed for task ':buildSrc:plugins/edx-app-gradle-plugin:compileGroovy'
ANY HELP ??

I was able to get the answer from the closed threads but after that i started getting a different error
ERROR
Error:Execution failed for task ':OpenEdXMobile:validateReleasekeySigning'.

Keystore file /home/arkangel/Documents/edx-app-android/OpenEdXMobile/edx-android-release-key.keystore not found for signing config 'releasekey'.

Closing this in favor of https://github.com/edx/edx-app-android/issues/933

How to resolve this issue?

@ankushgupta1393 You can ignore it. This error won't block or affect anything related to the building/deployment of app.

@miankhalid I'm also getting another exception -java.lang.IllegalArgumentException: Multiple entries with same key: supportsRtl=false and supportsRtl=true

@ankushgupta1393 you can create a new issue and share all the details there for a followup.

i am also getting error : Error:Unable to find module with Gradle path 'plugins/edx-app-gradle-plugin' (needed by module 'buildSrc'.) and i cant able to run app . Gradle sync failed

@rizwans-cuelogic That error shouldn't be causing any issues.
I believe we haven't added support for Android Studio 3 yet and still using Android 2.3

so do i want to downgrade gradle to 2.3

@rizwans-cuelogic not gradle, Android Studio's version 2.3.

@miankhalid is it worked on Android Studio's version 2.3.2

yes

thank you. it worked.  @miankhalid

You're welcome. Great!


Fix tests for size your images challenge

Pre-Submission Checklist

[x] Your pull request targets the staging branch of FreeCodeCamp.
[x] Branch starts with either fix/, feature/, or translate/ (e.g. fix/signin-issue)
[x] You have only one commit (if not, squash them into one commit).
[x] All new and existing tests pass the command npm run test-challenges. Use git commit --amend to amend any fixes.

Type of Change

[x] Small bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would change existing functionality)
[ ] Add new translation (feature adding new translations)

Checklist:

[x] Tested changes locally.
[x] Closes currently open issue (replace #9383 with an issue no): Closes #9383

Description

Added tests for checking whether a image element exists with class smaller-image and src https://bit.ly/fcc-relaxing-cat

The repetition is to check whether the quotes match. If the user didn't use quotes on both sides, it would be undefined and true since we are comparing the opening and closing quotes.
The first code fails because I expect a closing img tag. I have fixed the regex to pass the second code block you posted.
I think we should make sure to check closing tag exists. What do you think?

@arun1595, as discussed on gitter, #9408 solved this issue more cleanly. Sorry about that :disappointed:


Update cable metod put.

Environment

Python version:  3.6.6
NetBox version:  2.5.5

Create cable, Length length_unit do not fill.
If use class NetboxMapper in netboxapi and change termination.
Сalling the put method, returns an error that length_unit cannot be empty. Even if you first remove the attributes from the object. They are added to the to_dict functions class NetboxMapper. Here is an example of what is being transmitted in json:

{'id': 612, 'termination_a_type': 'dcim.interface', 'termination_a_id': 10488, 'termination_b_type': 'dcim.interface', 'termination_b_id': 8491, 'type': 3040, 'status': True, 'label': '', 'color': '', 'length': None, 'length_unit': None, 'termination_a': 10488, 'termination_b': 8492}

If I fill in the Length and length_unit and call the put method, then everything is fine, except that something strange happens on the device pages, I cannot describe it, but I can show:https://ibb.co/6B7kdwf

NetboxMapper is not a part of this project. Please raise an issues against that project if you believe it has a bug. We can only accept bug reports that can be reproduced without the use of a third-party tool.


Can´t load .IMA (Dicom from old siemens machine)

Hello,
I'm trying to save a 4D mri image (heart cines). The thing is they are .IMA instead of the usual .dcm this seems to be the way siemens call their dicoms, also I think the machine is a bit old (probably +12 years). When I use sitk.ImageSeriesReader.GetGDCMSeriesFileNames() it always return an empty tuple. Sometimes I also get a warning window saying:
WARNING: In D:\a\1\sitk-build\ITK\Modules\IO\GDCM\src\itkGDCMSeriesFileNames.cxx, line 75
GDCMSeriesFileNames (0000014A3B90D770): C:\Users\Manuel_LincBiotech\Desktop\Doctorado\Base datos clÃncio\283229 - copiacine\Nueva carpeta is not a directory
WARNING: In D:\a\1\sitk-build\ITK\Modules\IO\GDCM\src\itkGDCMSeriesFileNames.cxx, line 109
GDCMSeriesFileNames (0000014A3B90D770): No Series were found
WARNING: In D:\a\1\sitk-build\ITK\Modules\IO\GDCM\src\itkGDCMSeriesFileNames.cxx, line 75
GDCMSeriesFileNames (0000014A3B90CC80): C:\Users\Manuel_LincBiotech\Desktop\Doctorado\Base datos clÃncio\283229 - copiacine\Nueva carpeta is not a directory
WARNING: In D:\a\1\sitk-build\ITK\Modules\IO\GDCM\src\itkGDCMSeriesFileNames.cxx, line 122
GDCMSeriesFileNames (0000014A3B90CC80): No Series can be found, make sure your restrictions are not too strong
Not sure why I can´t load them, I tried changing manually the extension to .dcm, also tried with just one of the files with sitk.ImageFileReader() and it says the file doesn´t exist.
I know that the files are actually dicoms as I opened them with pydicom. I would prefer to stick with simpleitk to convert them directly to .nii but if it can´t read these maybe I´ll read the arrays with pydicom and then convert the arrays to sitk.

The first warning suggests to me that it's a problem with the path name.  I would try getting rid of all the spaces in all the file and directory names.

Ah that´s right. True, spaces are not something good in paths. Already changed it and it worked.
Thanks @dave3d


914 keep download product types

Fixes #914
Were any Django, import pipeline, table_schema, or dictionary files modified? Y

Database used: opus3_test_200107
All Django tests pass: Y

Were any JavaScript or CSS files modified? Y

JSHINT run on all affected files: Y
Tested on Chrome, Firefox, Safari, and iOS: Y
(Remember to test all browser sizes)
Tested for race conditions (with delayed API calls if applicable): Y

Description of changes:
-keep track of unselected product types; pass into the python/django code
-the django cart.html template modified to use new item in the dictionary that is the 'checked' state
-fixed a bug where the totals in the product type might go to 0 if unselected
-removed extra update of cart status after the reset that caused a 'flash' of inaccurate information
-changed the call to retrieve cart.html to a getJSON and return reqno w/that getJSON
-updated format test
-added a new test to check for unselect of product types and counts to be correct
Known problems:
no known problems

Database used: opus3_test_200107
All Django tests pass: Y
JSHINT run on all affected files: Y
Tested on Chrome, Firefox, Safari, and iOS: Y
(Remember to test all browser sizes)
Tested for race conditions (with delayed API calls if applicable): Y


Migrate subgraph metadata to relational storage

The metadata for subgraphs (in the subgraphs database schema) still uses JSONB storage; it needs to be moved to relational storage so we can eventually delete the JSONB storage code.
There's two big things that need to be done here:

write a database migration that creates a relational subgraphs schema and copies the data from the entities and entity_history tables into that schema
make it possible to not record history for certain entities/tables as only subgraph metadata related to dynamic data sources should be versioned, and at least for the case of SubgraphDeployment versioning the data is completely nonsensical and would lead to enormously bloated tables

We can only delete JSONB code after dropping support for subgraphs that were deployed under it, right? Or if we put those on a legacy system that stops getting updates.
Regarding the issue with only the dynamic data sources needing history, I wonder if we want to keep those in a shared table or move them into the per-subgraph schema.

Yes, we absolutely need to have all subgraphs use relational storage before we can delete the JSONB code. We will also need to come up with and communicate a deprecation plan to users (on their end, it will consist of "Redeploy your subgraph by this date/release")
In prod, we currently have 460 JSONB subgraphs and 322 relational ones, so it looks like the natural cycle of updating subgraphs is taking care of a lot of this; but there will be a point where we need to identify inactive JSONB subgraphs and come up with a plan for how to move them forward or delete them with the owner's consent.
Either way, as long as subgraph metadata uses JSONB storage, it won't be possible to move away from JSONB.
As for dynamic data sources: I was going to leave them in shared tables, simply because it will reduce risk when making this change. Longer term, I think it's a worthwhile goal.


Can't set tty to false

Version - v0.1.1-rc1
Steps:

rio run -n ttysvc2 --tty false nginx
kubectl get po -n default
kubectl get -n default -o=yaml pod/"ttysvc2 po name"

Results:  tty is set to true.  I also tried --tty=false.  Still wouldn't change.

Tty is a bool flag. So to set it to true you just run rio run --tty svc. To set it to false do not present the flag. @tfiduccia


switch shared storage back to efs

shared storage must be read-writeable by all.
emptyDir volumes currently provide insufficient space for dealbot daemons which need to stage 32GB sectors

@ognots  They don't; they expect the directory to already exist. However, the file names they use are hashes.

so long as all dealbot daemons are able to account for their own directories/cids let's ship this to unblock retrieval. we can look at improving tree structure later


Bug: Modified spacing for Markdown lists in the Changelog

TL;DR: This action adds two extra spaces to each markdown-list element in the changelog.
Currently, when processing the following example:
## [Unreleased]

This release does one thing

### Added

- List with one space

The action produces the following output:
## [Unreleased]

## [v0.0.9] - 2021-04-03

This release does one thing

### Added

-   List with one space

However, the expected output is:
## [Unreleased]

## [v0.0.9] - 2021-04-03

This release does one thing

### Added

- List with one space

Literally, it adds two extra spaces for each list item.
It changes (_ denotes a space):
-_List with one space

into
-___List with one space

I think the list entries should remain unmodified due to the following reasons:

https://keepachangelog.com/en/1.0.0/ defines them with a single space
Using 3 spaces triggers markdown-lint warning: MD030/list-marker-space: Spaces after list markers [Expected: 1; Actual: 3] markdownlint (MD030)

It would be great if this could be fixed 🙂

Thanks for the report!
This is unfortunately a known issue without a solution at the moment. See https://github.com/thomaseizinger/keep-a-changelog-new-release/issues/7.

So I guess, the status here is won't fix, or?
Maybe it is worth to keep the bug report open and invite someone to give it a look?

Sure!
Happy to keep it open :)
Should probably also document it in the README as a known problem!

I had to stop using this action because of this issue. I need my changelog to be updated perfectly since it's hands-free. I do not get a chance to manually edit and fix these issues.

Are you enforcing the formatting of your markdown files as part of CI?
If yes, then I'd recommend to just run the formatter after this action. That is what we do in some of our repositories.
If no, then I don't see why the reformatting is such a big deal. You could simply adopt the new formatting.
I'd love for this action to not make any formatting changes but I currently don't have the bandwidth to deal with it. Given that it is easily fixable by running a formatter afterwards, it is also fairly low priority.

I also use this action as one of  the last elements in automated release chain. The code is not touched by a human afterwards (it theoretically could be corrected by human in the release-PR but this is annoying).
@rcdailey did you find other action that is not affected by this problem?

I was not able to find and formatters in github actions. If you have one in mind please let me know.


Test full round trip of RL agent

The latest EvalAI platform and challenge environment are live at:

http://challenges.rangl.org
http://submissions.rangl.org

We need to test a round trip submission.
The steps involved are:

Register for http://challenges.rangl.org and participate in the challenge.

Test the containerised submission locally using the instructions here: https://github.com/rangl-labs/netzerotc/tree/main/random_agent_submission#local-testing

Submit an agent using the code from the meaningful_agent branch:

git clone https://github.com/rangl-labs/netzerotc.git
git checkout meaningful_agent

# build the agent image
cd netzerotc/meaningful_agent_submission
make build

# submit the agent image
evalai host -sh http://submissions.rangl.org
evalai login
evalai push submission:v0.1.0 --phase nztc-open-loop-1

This has been completed.


⚠ SceneTime has degraded performance

In 459dbb7, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 4545 ms

Resolved: SceneTime performance has improved in 8d2ee4f after 6 minutes.


chore(deps): update terraform github.com/terraform-ibm-modules/terraform-ibm-powervs-infrastructure to v5.3.0

This PR contains the following updates:

Package
Type
Update
Change

github.com/terraform-ibm-modules/terraform-ibm-powervs-infrastructure
module
minor
v5.2.3 -> v5.3.0

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Renovate will not automatically rebase this PR, because other commits have been found.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, click this checkbox. ⚠ Warning: custom changes will be lost.

This PR has been generated by Renovate Bot.

@surajsbharadwaj @Beilmann Can you guys make the relevant code change required to consume this version of powerVS infra?
Pipeline is failing with:
Validation failed: examples/basic
╷
│ Error: Invalid value for input variable
│
│   on main.tf line 112, in module "power_infrastructure":
│  112:   nfs_config                  = var.nfs_config
│
│ The given value is not suitable for
│ module.power_infrastructure.var.nfs_config declared at
│ .terraform/modules/power_infrastructure/variables.tf:159,1-22: attribute
│ "nfs_file_system" is required.

(That probably means that the nfs change should have been a major version increase, but too late now. Please bare in mind for next time)

Autoclosing Skipped
This PR has been flagged for autoclosing. However, it is being skipped due to the branch being already modified. Please close/delete it manually or report a bug if you think this is in error.

:tada: This PR is included in version 4.0.0 :tada:
The release is available on:

GitHub release
v4.0.0

Your semantic-release bot :package::rocket:


handle err messages from broker and log them?

Should we provide a way for a consumer of this library to handle these errors in a pluggable way? (ie forward them onto honeybadger/appsignal/etc?)


[bug] Node.js 12 actions are deprecated

Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/checkout@v2, actions/setup-python@v2. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

Here is the solution: https://github.com/actions/setup-python
- uses: actions/checkout@v3
- uses: actions/setup-python@v4


Use set instead of put

Since we aren't returning the value on puts, use IMap.set(k, v) instead of put(k,v) so that the old value doesn’t have to be returned (and thus, no deserialization).
Signed-off-by: Dave Sinclair<EMAIL_ADDRESS>

Thanks !


Padding doesn't work in jitted mode

Hi,
Thanks for the work on the library.
For a project I'm needing to implement a padding on the last dimension of a tensor. Everything works fine when the function is not jitted, but I get a conversion error when I jit it. It seems to me like it is coming from a bug in the lax_numpy internals (this line), but maybe I'm missing something...
Adrien
The code is as follows:
from jax import jit as jjit
import jax.numpy as jnp
import numpy as np

vals = np.random.randn(50, 100)

def pad_last_dim(array, pad_size):
    ndim = jnp.ndim(array)
    npad = jnp.zeros((ndim, 2), dtype=jnp.int32)
    axis = ndim - 1
    npad = ops.index_update(npad, ops.index[axis, 1], pad_size)
    return jnp.pad(array, npad, 'constant', constant_values=0)

print(pad_last_dim(vals, 1))  # All good
print(jjit(pad_last_dim)(vals, 1))  # raises

Thanks for the question!
The issue is that the result array size depends on the value of the pad_size argument to the jitted function. This kind of value-dependent-shape isn't allowed in a jitted function. The reason is that we try to compile a single XLA HLO program for all possible values of the inputs of a given shape and dtype, but since XLA HLO programs have array shapes in their static type system (for performance reasons), that means we can't compile as a single program a function in which the shapes depend on the numerical values of inputs.
One thing you can do is have JAX re-compile new versions of the function for every value (rather than shape/dtype) of the pad_size argument. Here's how you would do it:
print(jjit(pad_last_dim, static_argnums=(1,))(vals, 1))

I want to improve the error message you get here so that it's crystal clear what's going wrong, hat the constraints are, and what the options are for fixing it. Do you have any suggestions for how this could have been made more clear?
(The recent #3603 is related.)

For posterity, here's the current error message (stack trace removed):
Exception: The numpy.ndarray conversion method __array__() was called on the JAX Tracer object Traced<ShapedArray(int32[2,2]):JaxprTrace(level=-1/1)>.

This error can occur when a JAX Tracer object is passed to a raw numpy function, or a method on a numpy.ndarray object. You might want to check that you are using `jnp` together with `import jax.numpy as jnp` rather than using `np` via `import numpy as np`. If this error arises on a line that involves array indexing, like `x[idx]`, it may be that the array being indexed `x` is a raw numpy.ndarray while the indices `idx` are a JAX Tracer instance; in that case, you can instead write `jax.device_put(x)[idx]`

One thing you can do now is have JAX re-compile new versions of the function for every value (rather than shape/dtype) of the pad_size argument. You can do that using static_argnums:
print(jjit(pad_last_dim, static_argnums=(1,))(vals, 1))

But this isn't a great idea; there are more issues here. The code treats npad as an array, creating it with jnp.zeros and then applying jax.ops.index_update to it, means that value will be computed on, say, your GPU. That's likely not where you want to do scalar integer shape arithmetic! It's better not to use jnp or jax.ops to do shape computations, and in the future this is likely to raise an error.
Let me know if you have thoughts on how to make the error message more clear! I have some I'm starting to play with.

After #3627, here's the new error message:
TypeError: jax.numpy.pad got an unexpected type for 'pad_width': got [[0 0]
 [0 1]] of type <class 'jax.interpreters.xla.DeviceArray'>.

Unlike numpy, jax.numpy requires the 'pad_width' argument to jax.numpy.pad to
be an int, single-element tuple/list with an int element, or tuple/list of int
pairs (each pair a tuple or list); in particular, 'pad_width' cannot be an
array.

Hrm, test failures on #3627 have made me think we should allow numpy.ndarrays, but not JAX arrays.

Thanks a lot for that Matt. Bit late for me but I'll come back to it tomorrow. I think you should forbid lists and ndarrays tbh. Would be consistent with tf where they consider lists to be a collection of some kind.
In the meantime I like your solution and I'll use it (I don't need the ndarray, just was easier to set the padding)

Hrm, test failures on #3627 have made me think we should allow numpy.ndarrays, but not JAX arrays.

I think these are just failures due to JAX's test harness, which uses np.random to make random padding sizes. This is much less common in actual user code.

Thanks for catching that, @shoyer!

After #3627, here's the new error message even with static_argnums=(1,):
TypeError: jax.numpy.pad got an unexpected type for 'pad_width': got [[0 0]
 [0 1]] of type <class 'jax.interpreters.xla.DeviceArray'>.

Unlike numpy, jax.numpy requires the 'pad_width' argument to jax.numpy.pad to
be an int, single-element tuple/list with an int element, or tuple/list of int
pairs (each pair a tuple or list); in particular, 'pad_width' cannot be an
array.

If you need to manipulate a pad argument with NumPy, as in the original code,I recommend doing it with np rather than jnp, perhaps like this:
from jax import jit as jjit
import jax.numpy as jnp
from jax import ops
import numpy as np

vals = np.random.randn(50, 100)

def pad_last_dim(array, pad_size):
    ndim = jnp.ndim(array)
    npad = np.zeros((ndim, 2), dtype=np.int32)
    axis = ndim - 1
    npad[axis, 1] = pad_size
    npad = list(map(list, npad))
    return jnp.pad(array, npad, 'constant', constant_values=0)

print(pad_last_dim(vals, 1))  # All good
print(jjit(pad_last_dim, static_argnums=(1,))(vals, 1))

However, it's probably better just to use lists/tuples here:
from jax import jit as jjit
import jax.numpy as jnp
from jax import ops
import numpy as np

vals = np.random.randn(50, 100)

def pad_last_dim(array, pad_size):
    ndim = jnp.ndim(array)
    npad = [[0, pad_size]] * ndim
    return jnp.pad(array, npad, 'constant', constant_values=0)

print(pad_last_dim(vals, 1))  # All good
print(jjit(pad_last_dim, static_argnums=(1,))(vals, 1))

Your solution worked for me, thanks a lot!

I think we should declare this issue fixed. We still allow arrays as pad_width arguments to jnp.pad, but I think that's likely a good thing. The error message is much better these days:
ConcretizationTypeError: Abstract tracer value encountered where concrete value is expected: traced array with shape int32[2,2].
pad_width argument of jnp.pad
The error occurred while tracing the function pad_last_dim at <ipython-input-1-ec65a8c978ae>:7 for jit. This concrete value was not available in Python because it depends on the value of the argument pad_size.

See https://jax.readthedocs.io/en/latest/errors.html#jax.errors.ConcretizationTypeError

and I think that's good enough.


The module 'Models' could not be loaded.

PS C:\Users\şule meşe\Downloads\mltu-main\mltu-main> Models\03_handwriting_recognition\202402160440\logs
Models\03_handwriting_recognition\202402160440\logs : The module 'Models' could not be loaded. For more information, run 'Import-Module Models'.
At line:1 char:1

Models\03_handwriting_recognition\202402160440\logs

  + CategoryInfo          : ObjectNotFound: (Models\03_handw...2402160440\logs:String) [], CommandNotFoundException
  + FullyQualifiedErrorId : CouldNotAutoLoadModule

Can you help me please :)

Hi, i had similar problem with some modules. i tried:
import sys sys.path.append('D:\\Projects\\python\\web\\Das Auto\\das auto\\captcha\\mltu-main')
in the train.py while using 02_captcha_to_text model

It worked ofr me to find the mltu, and other models in the folder

thank you for your helps <3

thank you for your helps <3

did it help?

simply use pip install mltu, and if you want to use my models, but it anywhere you want and give full path to it


Use OIDC publish

OpenID Connect (OIDC) provides a flexible, credential-free mechanism for delegating publishing authority for a PyPI package to a trusted third party service.

I think we should use OIDC instead of username and password.
https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect

Oh, sorry. I forgot adding permissions.id-token: write. I fixed it.


[2012-10-29] Fix tabsets in the backend

created by: @kmayo-ss (kmayo)
created at: 2012-10-29
original ticket: http://open.silverstripe.org/ticket/7982

The tabsets in SilverStripe use all different types like ss-tabset, tabset, cms-tabset etc this should be changed to use the one type.

SS4 is getting a CSS overhaul at the moment, that should cover this.


docker daemon fails silently due to incorrect iptables NAT kernel modules

Description of problem:
I am running docker daemon on Gentoo, on AWS. I am running Gentoo with a custom kernel. While configuring the kernel options looks like, by mistake, I had turned of the CONFIG_NF_NAT because of which the iptables_nat.ko module wouldn't load. Docker was failing to start cause when it launched the iptables -t NAT .... command iptables was failing and causing network connectivity issues to the instance on which it was running.
I was able to figure out the problem by running docker in debug mode (docker -d --log-level=debug). However, I would expect docker to be able to log this failure in /var/log/docker.log, which would saved me a lot of time (initially I started suspecting there was a problem with the btrfs backend driver I was using).
When starting docker daemon should dectect if iptable is NAT capable or not.
docker version:
Client version: 1.6.1
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 97cd073
OS/Arch (client): linux/amd64
Server version: 1.6.1
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 97cd073
OS/Arch (server): linux/amd64
docker info:
Containers: 0
Images: 0
Storage Driver: devicemapper
Pool Name: docker-202:1-924875-pool
Pool Blocksize: 65.54 kB
Backing Filesystem: extfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 305.7 MB
Data Space Total: 107.4 GB
Data Space Available: 47.35 GB
Metadata Space Used: 733.2 kB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.147 GB
Udev Sync Supported: true
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.88 (2014-08-05)
Execution Driver: native-0.2
Kernel Version: 3.14.14-gentoo
Operating System: Gentoo/Linux
CPUs: 2
Total Memory: 3.68 GiB
Name: stitchnetos-aws
ID: BJM2:HBTR:HS2Y:CHUD:BMUV:BM26:GB7J:K2FV:4FRK:22JN:VWCF:3DBM
uname -a:
Linux stitchnetos-aws 3.14.14-gentoo #3 SMP Sun Jul 12 23:31:41 PDT 2015 x86_64 Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz GenuineIntel GNU/Linux
Environment details (AWS, VirtualBox, physical, etc.):
Custom Gentoo EBS-backed AMI running on AWS. This is a PV-HVM AMI
How reproducible:
Should be reproducible every time.
Steps to Reproduce:

Make sure that CONFIG_NF_NAT is turned off. This will lead to a missing nf_nat.ko module which in turn will not allow iptable_nat.ko to be loaded.
Start docker. It will fail to start
Check the /var/log/docker.log, there won't be any ERROR messages indicating something went wrong with docker.
Try running docker in debug mode docker -d --log-level=debug and you should see docker going and getting stuck trying to install the NAT rules.

Actual Results:
No message in /var/log/docker.log
Expected Results:
Expect ERROR messages in /var/log/docker.log indicating the failures (or absence) of iptable NAT.
Additional info:
The missing KERNEL config can be checked using the check_config.sh script, but in a production environment if there is a missing kernel module it might be harder to debug the issue, without apriori knowing that it is a problem with the kernel modules. Hence raising this issue to see if any checks and logs can be added to the docker daemon.

@asridharan Could you provide message from debug level pls?

oddly, I am unable to reproduce the kernel config that caused this problem. It could be that I incorrectly edited the Kconfig leading to the issue. If I explicitly remove iptables NAT from the kernel, I can see iptables throwing and error and it being reflected in /var/log/docker.log:
ime="2015-07-13T18:45:47-07:00" level=info msg="+job init_networkdriver()"
time="2015-07-13T18:45:47-07:00" level=warning msg="Running modprobe bridge nf_nat failed with message: , error: exit status 1"
Unable to enable network bridge NAT: iptables failed: iptables --wait -t nat -I POSTROUTING -s <IP_ADDRESS>/16 ! -o docker0 -j MASQUERADE: iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
(exit status 3)
time="2015-07-13T18:45:47-07:00" level=info msg="-job init_networkdriver() = ERR (1)"
This is what I was expecting. Going to close this issue. Will re-open if I am able to reproduce this problem .


⚠ Bofrost.at has degraded performance

In 3685bc2, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2446 ms

Resolved: Bofrost.at performance has improved in 0eddcaf.


Do not ship assumed default values for a kafka cluster

Problem
For the control plane the release artifacts ship the kafka-broker-config CM. With this content:
apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-broker-config
  namespace: knative-eventing
  labels:
    kafka.eventing.knative.dev/release: "devel"
data:
  default.topic.partitions: "10"
  default.topic.replication.factor: "3"
  bootstrap.servers: "my-cluster-kafka-bootstrap.kafka:9092"

IMO this more handy for us, doing development, rather than in production.
Persona:
Which persona is this feature for?
Exit Criteria
A measurable (binary) test that would indicate that the problem has been resolved.
Time Estimate (optional):
How many developer-days do you think this may take to resolve?
Additional context (optional)
Add any other context about the feature request here.

This isn't only handy for development but is handy also for getting started and servers as self-explaining documentation, I'd prefer leaving these values as they are.

I understand the point here.
Does the documentation reflect this is for a better "getting started experience" ?

/close
let's leave them, they don't do much harm.
if we have real complaints, we can always than react


ITMS-90809: Deprecated API Usage - New apps that use UIWebView are no longer accepted. Instead, use WKWebView for improved security and reliability.

I'm using Cordova to build iOS apps. When I'm uploading a build to AppStoreConnect, I'm getting the well known deprecation message:
ITMS-90809: Deprecated API Usage - New apps that use UIWebView are no longer accepted. Instead, use WKWebView for improved security and reliability. Learn more (https://developer.apple.com/documentation/uikit/uiwebview).
The plugin is using UIWebView.
Please update the plugin removing all UIWebView usage and using WKWebView or please let me know if we can use some other alternative plugin.

I'm pretty sure the latest published version already no longer uses UIWebView.


Would you consider adding a formula for Homebrew?

I've already sent a PR 5days ago, so let's wait for it.

They just merged it.


Add --range-start and --range-end options to ruff format

Summary
This PR adds the --range-start=<CHAR OFFSET> and --range-end=<CHAR_OFFSET> options to the format command.
The new options allow users only to format a selected range rather than the entire document. The main use case is to enable range formatting in IDEs.
Closes #7233
Design Decisions

Range formatting is only supported when formatting a single file, which should be the main use case anyway
The CLI only supports a single range. We can explore supporting multiple ranges in the future. The main challenge is that overalaping ranges invalidate the offset of which ever range gets formatted last (starting from the back helps but doesn't prevent it). This especially a problem if the formatter has to extend the formatted range.
The range is specified in character offsets. The alternatives I considered are:

line numbers similar to Black but being able to specify the range exactly can help the formatter to narrow the range better
line:column This would be more consistent to our --output-format=json where we output row and column numbers and can be easier to determine. However, this is mainly a feature for editors or when integrating Ruff into other tooling where computing a character offset shouldn't be a concern. The main downside of line:column is that it is a more complicated value

I went with two options instead of one to avoid the need for a custom syntax like 4-5 that user need to figure out

Limitations
The current implementation doesn't support notebooks because it's unclear if the range is relative to the notebook content or the raw notebook.
I decided to not support notebooks for now because the main use case, range formatting in VS Code, doesn't require notebook support because it only formats the closest cell.
Test Plan

Added CLI tests
I used the debug build to develop and test the LSP range formatting functionality

Was "byte offset" an option, or is the LSP not able to provide that?

Was "byte offset" an option, or is the LSP not able to provide that?

I considered it but decided against it because I want to provide users with a safe interface without thinking about encodings. Byte offsets would mean that we, in addition to out of bound indices, would also need to error on inputs that fall between character boundaries.

I would still bias toward byte offsets I think. Or perhaps even better, provide a way for a user to enter either byte offsets or character offsets. Worrying about encoding is a good point, but I imagine the vast majority of all Python source files are UTF-8. (Is it possible for Python source files to be something other than UTF-8? I'm actually not sure.)
Separately from that, could we use the word "codepoint" instead of "character" here? The former has a more concrete and unambiguous definition. The downside I suppose is that "character" is probably a more accessible term. Still, using the word "codepoint" will be an extra clear sign-post that the input is not byte offsets.

@BurntSushi what's your reasoning for biasing towards byte offsets? The only reason we use byte offsets internally is because they're convenient (and fast) to slice strings.
I don't see performance being a key motivator for this use case. Lexing, parsing, and all the IO are so dominant that the offset conversion won't matter.
Allowing different formats is intriguing but I think I would than either allow row:col or code point offsets.

Ah no, it's definitely not about performance. I suppose it's more about what is easy for users to actually get. Users are unlikely to be counting characters or bytes to figure out the inputs to these flags. So they'll probably get them from elsewhere. And I feel like usually what you get are byte offsets, especially when dealing with files on Unix operating systems. For example, grep has a -b/--byte-offset flag but no corresponding character offset flag. Byte offsets are just in general easier to deal with because they don't require knowledge of encodings when you treat everything as conventionally UTF-8. (Which is the prevailing strategy on Unix. Files are just bytes and most happen to be UTF-8.)
This is also why I suggested offering multiple ways to provide the range. Even if you switched to byte offsets, in the case of a user with char offsets, converting to byte offsets will be pretty annoying. Similarly, if you have byte offsets but need to provide char offsets, that's annoying too. For the same reason, I'd also advocate supporting line/column inputs too (grep provides them as well).
I'm not sure what the typical use case for these flags are though, and I'm sure that would have an impact on what we accept.

@BurntSushi the tooling support is an interesting consideration, although I don't really know what the use cases are for using range formatting over the CLI other than from an editor integration. And even there, I would advocate using the LSP instead that supports UTF32, UTF16, and UTF8 offsets.
For today, I don't think I want to support multiple encodings because we aren't aware of any use case. However, it would be nice if the design supported different encodings:

One option is to use different syntaxes, e.g. row:col and codepoint. However, this doesn't really support UTF8 vs codepoint. We could extend it further with cell:row:col for notebook support
separate --range-encoding option that can either be bytes, codepoints where row:col would be changed to row:bytes when used with bytes. The benefit of this approach is that we don't need to ship it today.

I could see us do both to allow the most flexibility but I think it's something we can defer until we know of actual use cases needing a different encoding (and they cant use the LSP).

If we don't expect users to call this directly we should hide it from the CLI help menus

If we don't expect users to call this directly we should hide it from the CLI help menus

I feel conflicted on this. It is a public API regardless if we hide it or not. I rather have it documented even if I would prefer not having to expose it at all.
@BurntSushi it's not that I'm not anticipating other use cases. It's just that I want to focus on the use case at hand. What's important to me is that the design allows us to support other potential use cases in the future, without having to redesign all options. That's why your feedback is very valuable and we should explore alternative options more if you aren't convinced that the one that I outlined are sufficient.

If we don't expect users to call this directly we should hide it from the CLI help menus

I'm not convinced that hiding options solves the problem. It is a public API as soon as we add it, even if undocumented. That's why I prefer documenting the behaviour even if I would prefer not having to expose it at all.

Yeah I'd rather it be in --help too. Otherwise if someone finds themselves debugging editor integrations (or whatever) and don't see the flags in --help, I could imagine that being pretty disorienting.

@BurntSushi it's not that I'm not anticipating other use cases. It's just that I want to focus on the use case at hand. What's important to me is that the design allows us to support other potential use cases in the future, without having to redesign all options. That's why your feedback is very valuable and we should explore alternative options more if you aren't convinced that the one that I outlined are sufficient.

Probably "sufficient" isn't the right word. It's not that codepoint offsets won't work. They will. It's a sound approach. I'm mostly just making the argument that, in my experience, byte offsets tend to be easier to come by.
If we're just looking for a path forward here that doesn't requiring potentially redesigning everything, then @charliermarsh's idea seems okay. To be clear, I agree with you that these flags will probably only really be used by editor integrations and not by end users directly. So in that sense, being more flexible in what we accept is maybe not so important.
If you asked me what my ideal design was and we had a good reason to believe these flags might be used outside of editor integrations, then I think I'd add one flag called --range with its own little DSL for specifying ranges. e.g., --range 'codepoint(start-end)' or --range 'bytes(start-end)' or --range 'lines(start-end)' and do things that way. But that's a fair bit of work. And if we start with --range-start-codepoint/--range-end-codepoint, then we can always add a hypothetical --range flag later if there's user demand.

For reference,

black supports line ranges (--line-ranges)
stylua supports byte offsets (--range-start, --range-end)
prettier supports character offsets (--range-start, --range-end)
clang-format supports both line ranges and byte offsets (--offset with --length, --lines)

I'll reply in more depth but one thing to consider is that powershell's IndexOf method make it easy to find the index of a substring. Getting the UTF8 byte offset is more involved (also true in Python where you need to encode the string in UTF8 first).

I'm leaning toward changing the input to line:column because:

It's the most convenient for users: Open your editor and you can see the numbers right there
It's what we get in the LSP
it allows us to later add our own DSL for providing codepoint or byte offsets.
Using byte offsets over code point offsets favors UNIX users because getting the byte offset seems easier on UNIX but is more involved on Windows. Using code points is true the other way

The downside of this is that it may require users to convert from a byte offset to line/column number if they want to use this feature in an automated way. I think I'm fine with this as a compromise for now because some tools provide line/column number output and converting a byte offset to a line number can be done using wc or String-Content (the range would be slightly larger but than providing exact ranges but that's probably neglectable).
The only remaining question is if it should be single or multiple arguments. I think I'll go with a single argument because supporting a custom DSL where you specify the range type is more awkward with multiple options.

What’s the scoop with the benchmarks?

What’s the scoop with the benchmarks?

It's most likely that I need to rebase my changes


max-results

Hello Everyone,
Can you please help me, how to set "max-results" parameter with Google_Service_AnalyticsReporting
what is appropriate function to pass this parameter?
Thank you

You're going to have to be more specific. Analytics Reporting is an entire API. What are you trying to accomplish?

@bshaffer thanks for reply, actually I am using Google_Service_AnalyticsReporting API Client and want to apply limit on maximum results. how can i achieve this?
As for Page order I am using following function: setOrderBys()


Fix failing tests and unify testsuite

Fix #34
With OutputFormat being added to commands, the version tests started failing. I also wanted to reuse the logic from cmd/test/test.go for the OutputFormat tests, so I brought it under pkg/cli/testsuite and made it a bit more minimal (we can increase it's features as we build out more commands).
I also brought in https://github.com/stretchr/testify for testing to use it mainly as an assertion library.

About testify: the chat was more or less uni soon that "go test should be enough. I, for one, appreciate the use of testify assertions. However, I understand that it's quite a big dependency for just using assertions. Maybe @krisnova / @dmah42 have opinions?

Also +1 for "human readable" as default output.

About testify: the chat was more or less uni sono that "go test should be enough. Therefore I intentionally built the first test without testify.

Good to know. I'll update the PR.

Also +1 for "human readable" as default output.
https://discord.com/channels/1026578122969206796/1026613433220726834/1062503346226876446

What are your thoughts on not using --output for stuff like that (and using it only for structured output) and creating a logger interface for things like screenshot you posted in discord?
With kubectl, if you do a command like kubectl get -v 6 you'll get structured output determined by --output along with additional logging output which is ignorant to --output.

With kubectl, if you do a command like kubectl get -v 6 you'll get structured output determined by --output along with additional logging output which is ignorant to --output.
Edit: We could even just skip the logger and use fmt directly for unstructured output since we have your SetWriter code.

I agree. I think it will add benefit to add a logger.
Yes, we could start with fmt directly. However, whats wrong with directly go with a structured logger that is aware of loglevels?

With kubectl, if you do a command like kubectl get -v 6 you'll get structured output determined by --output along with additional logging output which is ignorant to --output.
Edit: We could even just skip the logger and use fmt directly for unstructured output since we have your SetWriter code.

I agree. I think it will add benefit to add a logger. Yes, we could start with fmt directly. However, whats wrong with directly go with a structured logger that is aware of loglevels?

I'm good with that too.

Bump. Can we resolve these conflicts and get this merged? Looks like we are in a good spot. Do we have any blockers right now?

This is rebased and ready for merge.


Greenkeeper/serverless domain manager 1.1.19

Coverage remained the same at 100.0% when pulling 857748912bee78e69b10c80245dbf14df2f16584 on greenkeeper/serverless-domain-manager-1.1.19 into 1560696d8b369e838fd8630776543992ed217290 on master.

Coverage remained the same at 100.0% when pulling 857748912bee78e69b10c80245dbf14df2f16584 on greenkeeper/serverless-domain-manager-1.1.19 into 1560696d8b369e838fd8630776543992ed217290 on master.


Not serialising contained entity collection correctly for insert

Hi
We're using Simple.OData.Client to try to create a top level entity with a contained collection, e.g. TimeSeries with a collection of TimeSeriesPoint objects, where TimeSeriesPoint is a contained entity type, i.e. it is not exposed as a top level entity set, but it is also not a complex type.
We're see the JSON body of the POST request contain<EMAIL_ADDRESS>with a collection of URLs in an array, i.e. links to the contained TimeSeriesPoint object.
We're wondering if this is a bug, since we cannot create TimeSeriesPoint entities before creating the TimeSeries entity, since TimeSeriesPoint is not a top level collection.
In the metadata, the TimeSeriesPoint collection is a navigation property, but it has ContainsTarget=true. Does Simple.OData.Client currently treat this differently?
We would expect to see Points: [ { serialised TimeSeriesPoint object }, ... ] rather than link entries.
Can you advise please?
Thanks for providing this excellent library!

Sorry for the late response. I am struggling with other projects and unfortunately wasn't able to look at this before. Can you confirm that what you are trying to achieve works if you post the payload manually, i.e. using Postman? If so, I would like to have an example of working JSON payload, the POST URL and the copy of your OData service schema. You can send this to<EMAIL_ADDRESS>


Javascript/React cannot find module (for custom modules defined within the project)

Hi guys,
I'm trying to use hydrogen with a project that has a lot of custom dependencies, and when importing them I get the following: "Error: cannot find module 'bundles/page/components/TrackedButton';
Here's a screenshot:
https://www.dropbox.com/s/i2we272w9mqmenu/Screenshot 2017-09-24 14.41.12.png?dl=0
Is there a way to use this with custom imports? I noticed that I get the same error if I try running the jp-babel kernel on jupyter and running that same line in the notebook. Might be something to do with how jp-babel is being initialized, here's the kernel specs:
{
"kernelspecs": {
"babel": {
"spec": {
"display_name": "jp-Babel",
"argv": [
"jp-babel-kernel",
"--hide-undefined",
"--protocol=5.0",
"{connection_file}"
],
"language": "Babel ES6 JavaScript"
}
}
}
}

Is bundles an npm package or is it a local package? If it's local, you'll need to import from ./bundles

It's a local package, the ./ worked with the caveat that i also needed to add the parent directory to bundles "static", but this can be omitted when running the app that's why it wasn't added before. I'll close this.
Thanks

Hey guys, I'm running into something similar. Can't get Hydrogen w/ jp-babel to successfully import modules.
Ideally, I'd love to be able to do:
import React, { Component } from "react";
import Button from "./components/button"

However, both of these throw the same cannot find module @giovannipbonin linked above. No amount of fiddling with paths, adding parent dirs in and everything seems to help.
Seems like pretty basic functionality. Have I messed something up with my install? Here's my kernel specs
{
 "kernelspecs": {
    "babel": {
      "resource_dir": "/home/dano/.local/share/jupyter/kernels/babel",
      "spec": {
        "argv": [
          "jp-babel-kernel",
          "--hide-undefined",
          "{connection_file}",
          "--protocol=5.0"
        ],
        "env": {},
        "display_name": "jp-Babel (Node.js)",
        "language": "babel"
      }
    }

Hi @jdpigeon, I'm not very familiar with the jp-babel kernel but here are some general notes:
The first time you run code Hydrogen starts a new kernel in the directory specified via the Hydrogen settings.
Depending on your configuration Hydrogen will start the kernel either in the first project's directory opened in the tree view (default), the project directory relative to the file or the current directory of the file.
Can you double check if the kernel is started in the correct directory to make the imports work?

Aha! That's it. Of course. I always set my root in this one project one directory above where all the node_modules are.
Now I seem to be in the situation giovanni was in, though. I can import local stuff, but I have to use the path from the project root rather than the local path. I can temporarily change things in order to test with Hydrogen, but do you think there's a way of having Hydrogen figure those local paths based on which file you're running it in?


Check for multiple outputs connected to a single input (not allowed)

Added in Parameter::IsCompatible.


runtime: heapdump16 and heapdump17 are missing

What version of Go are you using (go version)?
1.10
What did you do?
Trying to decode heap dumps. The godoc leads me to https://github.com/golang/go/wiki/heapdump15
, but heap dump version 1.6 and 1.7 exist as well:

1.7: https://github.com/golang/go/commit/30f93f09944c54147bec9e5c39631f17addd94c7
1.6: https://github.com/golang/go/commit/cedbbfaa45fac82a0088b3163c7137e814020a53 (but the code review in https://go-review.googlesource.com/c/go/+/18912 seems to indicate there was no actual change?)

It would be helpful to have the specification for heapdump16 and heapdump17 in the wiki.

/cc @randall77 @aclements

1.5->1.6 had no changes other than the header.
1.6->1.7 has one very minor change to how to tell whether an interface is direct.
I'll update the wiki.

Done.
Let me know if my changes are clear.

For posterity, the new link is: https://github.com/golang/go/wiki/heapdump15-through-heapdump17


Race condition when automatically assigning IPs to VMs

ISSUE TYPE

Bug Report

COMPONENT NAME

Core/Network

CLOUDSTACK VERSION

4.17.2

CONFIGURATION

N/A
OS / ENVIRONMENT

N/A
SUMMARY

When creating 2 virtual machines in the same guest network at the exact same second (in our case 23 Aug 2023 13:34:11) with automatic IP assignment, both VMs can reserve a single IP address by race condition.
STEPS TO REPRODUCE

With Terraform, create 2 VMs in the same Guest network
resource "cloudstack_instance" "terraform_vm" {
  name =  "vm-created-using-terraform"
  service_offering = "Dynamic"
  template = "Debian 11"
  network_id = resource.cloudstack_network.testnet.id
  keypair = "[name_of_ssh_keypair]"
  details = {"cpuNumber" = 4,"memory" = 2048}
  expunge = "true"
  zone = "PA4"
}

resource "cloudstack_instance" "terraform_vm_2" {
  name =  "vm-created-using-terraform-2"
  service_offering = "Dynamic"
  template = "Debian 11"
  network_id = resource.cloudstack_network.testnet.id
  keypair = "[name_of_ssh_keypair]"
  details = {"cpuNumber" = 4,"memory" = 2048}
  expunge = "true"
  zone = "PA4"
}

Then apply the manifest several times (until the race condition triggers), and check their IPs

EXPECTED RESULTS

The IPs are always different
ACTUAL RESULTS

In some iterations, here is what we saw (sensitive information redacted):

Both VMs have reserved the exact same IP, the same Zone and guest network
Here is the relevant entries from the cloud database:
> select * from nics where ip4_address='<IP_ADDRESS>' and state='Allocated' \G;
*************************** 1. row ***************************
            id: 7225
          uuid: eab5163e-b88b-43a0-b6a3-ac02fd5a6037
   instance_id: 5471
   mac_address: 02:00:7c:99:00:19
   ip4_address: <IP_ADDRESS>
       netmask: <IP_ADDRESS>
       gateway: <IP_ADDRESS>
       ip_type: Ip4
 broadcast_uri: NULL
    network_id: 627
          mode: Dhcp
         state: Allocated
      strategy: Start
 reserver_name: VxlanGuestNetworkGuru
reservation_id: a3a9b296-e962-49f4-8c8f-3a3eef9128da
     device_id: 0
   update_time: 2023-08-23 15:34:11
 isolation_uri: NULL
   ip6_address: NULL
   default_nic: 1
       vm_type: User
       created: 2023-08-23 13:34:11
       removed: NULL
   ip6_gateway: NULL
      ip6_cidr: NULL
  secondary_ip: 0
   display_nic: 1
*************************** 2. row ***************************
            id: 7227
          uuid: 49af630f-9075-40f3-ae95-3d5a28c19ec2
   instance_id: 5475
   mac_address: 02:00:6e:f1:00:1d
   ip4_address: <IP_ADDRESS>
       netmask: <IP_ADDRESS>
       gateway: <IP_ADDRESS>
       ip_type: Ip4
 broadcast_uri: NULL
    network_id: 627
          mode: Dhcp
         state: Allocated
      strategy: Start
 reserver_name: VxlanGuestNetworkGuru
reservation_id: 5dc0a9e8-b4c5-4f8e-a883-00178984c09a
     device_id: 0
   update_time: 2023-08-23 15:34:11
 isolation_uri: NULL
   ip6_address: NULL
   default_nic: 1
       vm_type: User
       created: 2023-08-23 13:34:11
       removed: NULL
   ip6_gateway: NULL
      ip6_cidr: NULL
  secondary_ip: 0
   display_nic: 1

there are some issues when deploy or start virtual machines in parellel.
for example #7202 #6644 (fixed by #6694)

It seems we may have two PRs for this @sureshanaparti @DaanHoogland @weizhouapache @abh1sar and others

It seems we may have two PRs for this @sureshanaparti @DaanHoogland @weizhouapache @abh1sar and others

I see only @abh1sar 's @rohityadavcloud . Can you point out the other one?

^rubs my eyes^ is this same/similar https://github.com/apache/cloudstack/pull/9234 @DaanHoogland ?

You are right @rohityadavcloud, it is the same. I didn't see that link. @abh1sar can you contact @hsato03 on his PR and discuss a (the best) way forward

You are right @rohityadavcloud, it is the same. I didn't see that link. @abh1sar can you contact @hsato03 on his PR and discuss a (the best) way forward

it looks this pr aims to fix same vm IP, while #9234 aims to fix same Public Ip.

I think Wei is right, they are different situations.

+1 to @weizhouapache and @hsato03 . #9234 fixes public IP race by taking lock before making changes to the IPAddressVO table.
For guest IPs the approach would be different #9240 .

Thanks for checking @weizhouapache @hsato03 @abh1sar

Fixed in #9240


Remove the use of Google ad tracking

As an end user, I'd like to know that my details aren't being sold to Google for ad tracking.
Remove the Google tracking code and replace it with something more specific that lets the use profiles be visible but doesn't require third party cookies to work.

Resolved


lit-html-server

Hi! I've started working on a server renderer for lit-html templates, and wanted to share it here in case anyone thinks it may be a good starting point for an official renderer. The lit-html-server package is still available on npm, but I've published it as a scoped package in case it doesn't fit with the current plans:

NPM: @popeindustries/lit-html-server
GitHub: lit-html-server

I've tried to be faithful to (most of) the lit-html semantics and behaviour, while also emphasising rendering performance in a production environment (ie. streams, limited object/instance creation, and no html parsing). Here is a list of completed and planned features:

[x] html and svg template functions
[x] render(template) function returning a Node.js Readable stream
[x] handle primitive value types (string, number, boolean, null, undefined)
[x] handle nested templates
[x] handle Arrays
[x] handle Promises
[x] handle element attribute bindings (?.@)
[x] handle element attribute value quoting
[x] escape values for html
[x] handle sync iterators
[x] port default lit-html directives
handle async iterators (will wait until no longer an experimental Node feature)
rehydration?? (needs some planning!)

Streaming
The streaming implementation is based on the stream-template library by @almost (of which I'm also a contributor). As a performance optimisation, however, instead of returning a Readable stream instance for every template, lit-html-server prefers strings over streams, and only deopts to streams for (nested) stream and Promise value types (render(template) will always return a stream even if template is a string, however). In general, this should result in a significant performance increase (see this issue).
Attributes
I've tried to handle attributes in a way that is logical for static html rendering, and that also conforms to existing lit-html behaviour. Attribute values missing quotes are wrapped in quotes, and attribute names that are prefixed with binding characters are handled as follows:

?: boolean bindings remove the ?name=${value} if falsey, and retain the name if truthy
.: property bindings remove the .name=${value}
@: event handler bindings remove the @name=${value}

Obviously, the current handling of attributes only makes sense in a static html world, and I haven't tried to address a strategy for re-hydration or any form for bootstrapping lit-html on the client.
Future plans
As mentioned above, my immediate plans are to add iterator support and begin porting directives, in addition to more tests and documentation. After that, I hope that others from the lit-html team/community may be able to begin planning a strategy for re-hydration/bootstrapping.
Please feel free to weigh in with ideas/comments here or at https://github.com/popeindustries/lit-html-server/issues

There's a lot to unpack here, but I just wanted to pop in to say that this is very exciting!
I have previously gotten most of lit-html itself working in node with a forked version of undom, but hadn't tackled rehydration. That experiment definitely made me think that a seperate server implementation could do some things better like streaming, and of course that property and event handler bindings only make sense if you are deeply rendering custom elements (which I'm somewhat skeptical of for various reasons).
Independently, I made a simple html tag for server rendered pages (for a blog system), which worked really well. lit-html-server seems like a much, much more complete version of that.
Thanks for posting this!

Added classMap/styleMap directives, and will now try and synchronize version numbers with the main lit-html project.

This is amazing!  Could we get an example repo to get us started?

Added this to my collection: https://github.com/web-padawan/awesome-lit-html

@aadamsx the only example available right now is probably this silly perf test (~4000 req/sec on my 3+ year old MacBook, in case you were curious 😉)
Maybe I should take a look at updating this lit-element todo app with SSR?


Fix login issues with meshery provider requires reload

Signed-off-by: Abhishek-kumar09<EMAIL_ADDRESS>Description
So whenenever we logout and login again, it requires a reload for the authentication to be done. After experimenting with lots of things came out that it was redirection issue coming from server, So the good old issue is no more😜😋
https://user-images.githubusercontent.com/48255244/145686002-9861f363-d407-4994-8bcd-fcd1e8728c11.mp4
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

Codecov Report

Merging #4762 (e7fcf21) into master (5915d34) will decrease coverage by 6.95%.
The diff coverage is n/a.

:exclamation: Current head e7fcf21 differs from pull request most recent head a26991e. Consider uploading reports for the commit a26991e to get more accurate results

@@            Coverage Diff             @@
##           master    #4762      +/-   ##
==========================================
- Coverage   32.32%   25.37%   -6.96%
==========================================
  Files          61       61
  Lines        5116     5116
==========================================
- Hits         1654     1298     -356
- Misses       3034     3500     +466
+ Partials      428      318     -110

Flag
Coverage Δ

gointegrationtests
?

unittests
25.37% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

mesheryctl/internal/cli/root/system/start.go
1.11% <0.00%> (-58.52%)
:arrow_down:

mesheryctl/internal/cli/root/system/restart.go
2.50% <0.00%> (-47.50%)
:arrow_down:

mesheryctl/internal/cli/root/system/status.go
0.00% <0.00%> (-47.44%)
:arrow_down:

mesheryctl/internal/cli/root/system/update.go
1.05% <0.00%> (-34.74%)
:arrow_down:

mesheryctl/internal/cli/root/system/stop.go
1.78% <0.00%> (-23.22%)
:arrow_down:

mesheryctl/internal/cli/root/system/check.go
1.39% <0.00%> (-22.38%)
:arrow_down:

mesheryctl/internal/cli/root/system/logs.go
3.29% <0.00%> (-20.88%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55c3409...a26991e. Read the comment docs.

Yes, provider tokens are flushed out of cookies.


Add constant in traits test

Add tests for  #139
I don't know if it's enough. Let me know
Checklist:

[ ] All tests pass in CI
[ ] There are sufficient tests for the new fix/feature
[ ] Grammar rules have not been renamed unless absolutely necessary (x rules renamed)
[ ] The conflicts section hasn't grown too much (x new conflicts)
[ ] The parser size hasn't grown too much (master: STATE_COUNT, PR: STATE_COUNT) (check the value of STATE_COUNT in src/parser.c)

can someone relaunch the CI please?


Daily returns

Changes to calculations to Risk Metrics.
A few months ago, we made changes to how we calculated the cumulative risk metrics for our backtester (https://www.quantopian.com/posts/corrections-to-cumulative-risk-metrics). Frank Grossman recently pointed out that he found a discrepancy between the 12 month Zipline metrics and the overall risk metrics that the backtester was showing. We found that the difference came from how we were calculating average daily returns. Before we were using cumulative returns divided by the total number of trading days. We found that this served well for short periods of time (e.g. 12 months) but often overstated the returns when it came to backtest periods of greater than a year. Now we've switched to using a method widely incorporated throughout the industry and that's to use the arithmetic mean to calculate daily returns, so simply put, we are taking the average of daily returns to give us back the mean daily return used to calculate our risk metrics. The actual calculations of the risk metrics (Sharpe, Sortino, Information) remain the same as it was before in this post (https://www.quantopian.com/posts/corrections-to-cumulative-risk-metrics), only the annualized average returns used for the calculations have changed.
With the new way of calculating daily mean return() (daily_returns.mean()), every metric associated will have differing values:
Alpha
Sharpe
Sortino
Information
Beta remains the same

@dunster @seongsean did anything ever come of this?  It's been sitting unmerged for over a year...

I believe that this has been superseded by using empyrical for risk metrics.


Blazor InputFile component example should be extended.

The Blazor InputFile component example should be extended to include:

Creating a proper Http request using PostAsync
Binding a controller action to receive the request using IFormFile

Please see the example linked below for ideas. https://github.com/EdCharbeneau/BlazorPowerHour

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5ea474d5-234a-f393-8609-892a2deb5d2e
Version Independent ID: c11d981c-05af-c19d-a333-feedd5978639
Content: ASP.NET Core Blazor file uploads
Content Source: aspnetcore/blazor/file-uploads.md
Product: aspnet-core
Technology: aspnetcore-blazor
GitHub Login: @guardrex
Microsoft Alias: riande

@guardrex Let's get this work prioritized. I witnessed much confusion this morning while Ed was covering this topic on his Twitch stream. A handful of customers in the audience were looking for the same info in our docs.

@guardrex Let's get this work prioritized. I witnessed much confusion this morning while Ed was covering this topic on his Twitch stream. A handful of customers in the audience were looking for the same info in our docs.

Thanks @scottaddie and @EdCharbeneau ... Let's call this a dup of https://github.com/dotnet/AspNetCore.Docs/issues/20323.

Thanks @scottaddie and @EdCharbeneau ... Let's call this a dup of https://github.com/dotnet/AspNetCore.Docs/issues/20323.


Suggested updates to the FlaskClient driver

Hi - first off thank you for this wonderful library. I'm using it in conjunction with pytest-bdd and Flask and for the most part it's been plain sailing, however I am currently using a custom FlaskClient that overrides the _do_method to resolve 2 issues/traits I came across.
302 and 303 behaviour
By default most browser (correctly or incorrectly) will use the GET method when redirecting based on a 302 response, and 303s (I believe) specifically address this confusion by declaring that the 303 response from a POST request is to a GET resource.
At the moment using the FlaskClient all 30X responses retain the current method, so if a 303 response is received from a POST request then the _do_method will make another request using the post not the intended get method.
args and form data
Flask provides access to data transmitted via POST or PUT through request.form, alternatively data transmitted in the URL (e.g via GET) is available through request.args (there's also request.values which allows you to access data in either but that's not important here).
Using the FlaskClient when you submit a form with a method attribute of GET (e.g <form method="GET" ...>) the request will be made as using the GET method but the data will be sent as if POSTed and will end up in the request.form attribute. As you can imagine this then causes  issues because the view being called is looking for the submitted data in request.args.
The cause of the issue is that data is always sent in the _do_method using the data keyword argument when calling the func_method, but for a GET request (I believe) the data keyword should not sent as None and the data should be set against the URL as a query.
Fixing these issues
As I mentioned at the start of the issue I'm currently using a patched driver based on the existing FlaskClient to resolve these issues/traits. I've included the code below in case others come across the same issues in the short-term and as a suggestion as to how they might be fixed. If you'd like me to contribute the changes as a pull request I'm more than happy to have a look at doing so over the weekend.

Note: My patch is using Python3.4+, hence from urllib import parse so would need to be made compatible with 2.7+ though I think that would be a relatively easy task.

from urllib import parse

from splinter.browser import _DRIVERS
from splinter.driver import flaskclient

__all__ = ['FlaskClient']

class FlaskClient(flaskclient.FlaskClient):
    """
    A patched `FlaskClient` driver that implements more standard `302`/`303`
    behaviour and that sets data for `GET` requests against the URL.
    """

    driver_name = 'flask'

    def _do_method(self, method, url, data=None):

        # Set the initial URL and client/HTTP method
        self._url = url
        func_method = getattr(self._browser, method.lower())

        # Continue to make requests until a non 30X response is recieved
        while True:
            self._last_urls.append(url)

            # If we're making a GET request set the data against the URL as a
            # query.
            if method.lower() == 'get':

                # Parse the existing URL and it's query
                url_parts = parse.urlparse(url)
                url_params = parse.parse_qs(url_parts.query)

                # Update any existing query dictionary with the `data` argument
                url_params.update(data or {})
                url_parts = url_parts._replace(
                        query=parse.urlencode(url_params, doseq=True))

                # Rebuild the URL
                url = parse.urlunparse(url_parts)

                # As the `data` argument will be passed as a keyword argument to
                # the `func_method` we set it `None` to prevent it populating
                # `flask.request.form` on `GET` requests.
                data = None

            # Call the flask client
            self._response = func_method(
                url,
                headers=self._custom_headers,
                data=data,
                follow_redirects=False
                )

            # Implement more standard `302`/`303` behaviour
            if self._response.status_code in (302, 303):
                func_method = getattr(self._browser, 'get')

            # If the response was not in the `30X` range we're done
            if self._response.status_code not in (301, 302, 303, 305, 307):
                break

            # If the response was in the `30X` range get next URL to request
            url = self._response.headers['Location']

        self._url = self._last_urls[-1]
        self._post_load()

# Patch the default `FlaskClient` driver
_DRIVERS['flask'] = FlaskClient

Hi, nice catch!
I'm +1 for these improvements.

@andrewsmedina - Happy to, will give it a shot this weekend :)

@andrewsmedina OK I've pushed a pull request for this #517.

@anthonyjb thank you for contribute. can you add some tests for this pr?

Hi @andrewsmedina,
Yes sorry I merely checked it passed existing flask tests, should have added tests for the identified issues. However, before I do I found another issues yesterday were checkbox input values can't be sent through, they need to be pre converted to a list before being passed in data, e.g:
# Ensure all data has been converted to a supported type
if data:
    for k, v in data.items():
        if isinstance(v, CheckboxValues):
            v = list(v)
        data[k] = v

If you don't convert this then with the FlaskClient at least you get the __repr__ string for the CheckboxValues object through instead of the values themselves.
I'll post a fix for this in the next couple of days but I want to check;

should the issue be solved in the FlaskClient or should it be fixed in the submit method of LxmlDriver.
Are there other input types that have the same issue, e.g a select field with multiple options checked.

I'll review and post an update as soon as I know a long with a PR and some tests checking these issues are resolved.

Just wanted to chime in and say that the patch mentioned up top worked great. Thanks @anthonyjb!

Hi, I've just started using splinter with Flask and have run into the redirect issue; rather than having to patch things ourselves it would be really nice to see the fix merged. Would you be okay @anthonyjb @andrewsmedina if I opened a new pull request with @anthonyjb's fix (#517) plus some tests?

@lfdebrux I'm ok with that ;)

Fixed by #721


Including FB messenger under encrypted backups

Facebook Messenger also relies on encrypted backups and uses the same system that WhatsApp uses. See: https://engineering.fb.com/wp-content/uploads/2023/12/TheLabyrinthEncryptedMessageStorageProtocol_12-6-2023.pdf , page 34 under the "User-Chosen PINs" section for a reference.
Also, here is a better permalink for the WhatsApp whitepaper: https://www.whatsapp.com/security/WhatsApp_Security_Encrypted_Backups_Whitepaper.pdf

Thanks!

Doesn't Labyrinth do quite a bit more than just encrypted backup?


error using SSL certs

Konga throws below error when you configure cert and key in .env file and try to start Konga
npm start

<EMAIL_ADDRESS>start /opt/konga
node --harmony app.js

Using postgres DB Adapter.
Database exists. Continue...
error: A hook (http) failed to load!
error: Error: error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
at Object.createSecureContext (_tls_common.js:156:17)
at Server (_tls_wrap.js:873:27)
at new Server (https.js:62:14)
at createServer (https.js:85:10)
at _afterLoadingSessionHookIfApplicable (/opt/konga/node_modules/sails/lib/hooks/http/initialize.js:87:28)
at _waitForSessionHookIfApplicable (/opt/konga/node_modules/sails/lib/hooks/http/initialize.js:40:16)
at Hook.initialize (/opt/konga/node_modules/sails/lib/hooks/http/initialize.js:42:7)
at Hook.wrapper [as initialize] (/opt/konga/node_modules/@sailshq/lodash/lib/index.js:3250:19)
at /opt/konga/node_modules/sails/lib/hooks/index.js:88:16
at /opt/konga/node_modules/async/lib/async.js:52:16
at /opt/konga/node_modules/async/lib/async.js:548:17
at /opt/konga/node_modules/async/lib/async.js:542:17
at _arrayEach (/opt/konga/node_modules/async/lib/async.js:85:13)
at Immediate.taskComplete (/opt/konga/node_modules/async/lib/async.js:541:13)
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
at process.topLevelDomainCallback (domain.js:126:23)
error: A hook (orm) failed to load!

@DewangS , how exactly have you configured them? Are they in PEM format?


Check that we handle co-authorship

See https://help.github.com/en/github/committing-changes-to-your-project/creating-a-commit-with-multiple-authors.

[x] Check what that looks like in the webhooks and if we handle it
[ ] Doc it on the settings page
[ ] Maybe change our pair commit tools to use this

In the webhooks, it just appears in the commit message as written. No other field is set.
Same via the commit API.
So to support this, we'd probably need to parse the email from the commit message and use the API to map it to a username, and/or some cache mapping emails to usernames.
Not worth it since we've got something that works already. Closing.

We now have some support for this: https://github.com/barsoom/ex-remit/commit/235971684b919c02e8fd65e6589d16f68ed2d815


Fixed a typo with the account properties

Description
Fixes a typo on a couple of internal properties of the AccountProperties class. There is no breaking change since they are internal.
Type of change
Please delete options that are not relevant.

[x] Typo fix (non-breaking change which fixes an issue)

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: Elfocrash sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

/AzurePipelines run


Non-ascii character in jquery.contextMenu.js causes compatibility issues in Rails

Hi there,
Just a quick fix to remove a non-ascii character from one of the comments in jquery.contextMenu.js.  (It caused the dreaded "Encoding::CompatibilityError - incompatible character encodings: UTF-8 and ASCII-8BIT" within my Rails project.)
Thanks for the great work!

There's another one on line 1001, an elipses (...).

merged and released with version 1.5.7


Remove non-existent download strategies.

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew tests with your changes locally?

These were removed and are also no longer in compat/.

Nice catch 👍


Is it going to be possible to search for variants by genomic location using cytogenetic nomenclature?

As it is possible in the web interface.

@sprintell e.g.https://www.ebi.ac.uk/gwas/regions/2q37.1