Create a STAC catalog for the REMA COG data

Once the data has been converted to COG (see #1), it needs to be uploaded to the cloud and cataloged. The  uploading itself is easy. The cataloging is the hard part/
Probably the best way to do the catalog is with the spatiotemporal asset catalog specification: https://github.com/radiantearth/stac-spec/
This basically means creating a bunch of .json files which describe the data collection. There are various automated tools and scripts to generate these: https://github.com/radiantearth/stac-spec/blob/master/implementations.md#ecosystem

Some background on cloud storage:

https://cloud.google.com/storage/docs/
https://cloud.google.com/storage/docs/gsutil (this is the command line tool you use to upload)

Our bucket is
https://console.cloud.google.com/storage/browser/pangeo-pgc?project=pangeo-181919&folder&organizationId=819335046878

Project is pangeo-181919

Command to upload (recusive)
gsutil -m cp -r /path/to/data gs://pangeo-pgc/

Check this out! It works!
http://www.cogeo.org/map/#/url/https%3A%2F%2Fstorage.googleapis.com%2Fpangeo-pgc%2F38_48_8m_dem_COG_RAW.tif

@jjspergel the code you sent me for producing the json files is running. It works with only one copy of the satstac.py file sat in the '8m' directory if i use the following
forfiles /s /m *_COG_LZW.tif /c "cmd /c python ../../satstac.py @path @file"
One thing, though is the file structure may not be exactly as I put them up in the bucket.
Looking at this example:
09_39_8m_dem_COG_LZW.txt
It looks like it is https://storage.googleapis.com/pangeo-pgc/09_39_8m_dem_COG_LZW.json
when it should be https://storage.googleapis.com/pangeo-pgc/8m/09_39_8m_dem_COG_LZW.json
https://console.cloud.google.com/storage/browser/pangeo-pgc?project=pangeo-181919&folder&organizationId=819335046878
i will try to edit the satstac.py file and rerun i think.

Sorry @jkingslake , but yes I think that should be an easy fix - just edit the "link_pref" to include the subfolder "8m". I also would recommend renaming the py file, my compiler got confused later because both the module and the code I wrote are called satstac.

Done.
Where should the json files need to sit?
I suspect the answer is that it doesnt matter, but just to check before I upload them.

I'm working right now on how we want to combine them into a json catalog. I
don't think it really matters, just as long as they can be easily accessed
by the code that will combine/index them all.
On Wed, Apr 3, 2019 at 3:37 PM jkingslake<EMAIL_ADDRESS>wrote:

Done.
Where should the json files need to sit?
I suspect the answer is that it doesnt matter, but just to check before I
upload them.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rabernat/pangeo-rema/issues/2#issuecomment-479629563,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Ao-5EZFtUJezbkyzdL8QioFMaRMn9FBDks5vdQL1gaJpZM4b8NLw
.

--
Julian J. Spergel
Lamont Doherty Earth Observatory
University of Chicago '16
Geophysical Sciences

I would hold off on uploading the .json files. I'm having some issues with
combining them. They might need a little bit of tweaking to get the correct
syntax and format to work with the code. Sorry for the inconvenience.
On Wed, Apr 3, 2019 at 3:41 PM Julian Jacob Spergel<EMAIL_ADDRESS>wrote:

I'm working right now on how we want to combine them into a json catalog.
I don't think it really matters, just as long as they can be easily
accessed by the code that will combine/index them all.
On Wed, Apr 3, 2019 at 3:37 PM jkingslake<EMAIL_ADDRESS>wrote:

Done.
Where should the json files need to sit?
I suspect the answer is that it doesnt matter, but just to check before I
upload them.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rabernat/pangeo-rema/issues/2#issuecomment-479629563,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Ao-5EZFtUJezbkyzdL8QioFMaRMn9FBDks5vdQL1gaJpZM4b8NLw
.

--
Julian J. Spergel
Lamont Doherty Earth Observatory
University of Chicago '16
Geophysical Sciences

--
Julian J. Spergel
Lamont Doherty Earth Observatory
University of Chicago '16
Geophysical Sciences

Glad to see stuff happening without me!
Just want to remind you that you can use this git repo to share the scripts you are using to generate the data, produce stac, etc (rather than emailing python files around).

It seems like the satstac program can only catalog these json files if there's no "link" section of the .json file already, so we'll need to remove that from the json code and remake all the .json files. I'm still working on getting them to catalog/combine properly.
The issue right now is that the automatically generated links look like:
'https://storage.googleapis.com/pangeo-pgc/8m\\stac-catalog\\34_39_8m_dem_COG_LZW.json'
which I think will create problems when uploaded. I'll keep you updated.

Hi Jonny,
I forget how long it took to make .json files last time, but if it's a long
time, would it be possible to run the new code overnight tonight? The only
change is the removal of the link-forming section. As before use:
forfiles /s /m *_COG_LZW.tif /c "cmd /c python ../../satstac_jsoncode.py
@path https://github.com/path @file https://github.com/file
Let me know if you can run this tonight or not. No real rush, I just want
to schedule work this week.  Thanks!
On Wed, Apr 3, 2019 at 3:37 PM jkingslake<EMAIL_ADDRESS>wrote:

Done.
Where should the json files need to sit?
I suspect the answer is that it doesnt matter, but just to check before I
upload them.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rabernat/pangeo-rema/issues/2#issuecomment-479629563,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Ao-5EZFtUJezbkyzdL8QioFMaRMn9FBDks5vdQL1gaJpZM4b8NLw
.

--
Julian J. Spergel
Lamont Doherty Earth Observatory
University of Chicago '16
Geophysical Sciences
def satstac(rasterPath,rasterName):
from satstac import Catalog, Collection, Item
import xarray as xr
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
import rasterio
import rasterio.features
import rasterio.warp
from shapely import geometry
import json
import cartopy

class REMAStacItem:

    def __init__(self,rasterPath, rasterName):

        self.rasterPath = rasterPath
        self.id      = rasterName
        self.stac_item   = {"id": rasterName[:-4],
                            "rownum": str(rasterName)[0:2],
                            "colnum": str(rasterName)[3:5],
                            "type": "Feature",
                            "bbox": self.calcBBox(),
                            "geometry": self.calcGeometry(),
                            "units": 'meters',
                            "resolution": 8
                           }

        self.stac_item['properties']=self.createProperties_meta()
        self.stac_item['links'] = self.create_links()

    def calcBBox(self):

        with rasterio.open(self.rasterPath) as dataset:
            bbox = rasterio.warp.transform_bounds(dataset.crs, 'EPSG:4326',dataset.bounds.left,dataset.bounds.bottom,
                                          dataset.bounds.right,dataset.bounds.top)

        #self.bbox=json.loads(json.dumps(geometry.mapping(bbox)))

        return json.loads(json.dumps(bbox))

    def calcGeometry(self):

        with rasterio.open(self.rasterPath) as dataset:
            profile = dataset.profile
                # Read the dataset's valid data mask as a ndarray.
            geom = geometry.box(*rasterio.warp.transform_bounds(dataset.crs, 'EPSG:4326',dataset.bounds.left,dataset.bounds.bottom,
                                          dataset.bounds.right,dataset.bounds.top))

        self.geometry=json.loads(json.dumps(geometry.mapping(geom)))

        return json.loads(json.dumps(geometry.mapping(geom)))

    #def createAssetList(self):
     #   pass

    def createProperties_meta(self):
        meta_prop_dict = {}
        with rasterio.open(self.id) as dataset:
                meta_prop_dict = dataset.meta
                crs = meta_prop_dict['crs']
                crs = 'EPSG:'+str(crs.to_epsg())
                meta_prop_dict['crs'] = crs
                profile = dataset.profile
        return meta_prop_dict

     #   if imdPath:
      #      o = urlparse(imdPath)
       #     if o.scheme == 's3':
        #        s3 = boto3.resource("s3")
         #       bucket = o.netloc
          #      key = o.path.lstrip('/')
           #     obj = s3.Object(bucket, key)
            #    body = obj.get()['Body'].read()
             #   root = ET.fromstring(body)

            #else:

             #   tree = ET.parse(imdPath)
              #  root = tree.getroot()

            #self.metaDataStruct = iterate_OverXML(root, recursionTagList=['IMD', 'IMAGE'])
        #elif vrtPath:
         #   with rasterio.open(vrtPath) as src:
          #      self.metaDataStruct = process_nitf_tags(src.tags())

       # else:
        #    print("no Data Specified")
        #self.eoDict = self.processMetaData_To_Properties(self.metaDataStruct, self.provider, self.license)
        #return json.loads(json.dumps(meta_prop_dict))

    def write_toJSON(self, filename):
        with open(filename, 'w') as fp:
            json.dump(self.stac_item, fp, indent=2)

myStacItem = REMAStacItem(rasterPath,rasterName)
filename = myStacItem.stac_item['id']+'.json'
myStacItem.write_toJSON(filename)

Julian,
Yes, I can do this. Although you will have to remind me the details.
Jonny
On Mon, Jun 10, 2019 at 3:56 PM jjspergel<EMAIL_ADDRESS>wrote:

Hi Jonny,
I forget how long it took to make .json files last time, but if it's a long
time, would it be possible to run the new code overnight tonight? The only
change is the removal of the link-forming section. As before use:
forfiles /s /m *_COG_LZW.tif /c "cmd /c python ../../satstac_jsoncode.py
@path https://github.com/path @file https://github.com/file
Let me know if you can run this tonight or not. No real rush, I just want
to schedule work this week. Thanks!
On Wed, Apr 3, 2019 at 3:37 PM jkingslake<EMAIL_ADDRESS>wrote:

Done.
Where should the json files need to sit?
I suspect the answer is that it doesnt matter, but just to check before I
upload them.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/rabernat/pangeo-rema/issues/2#issuecomment-479629563
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/Ao-5EZFtUJezbkyzdL8QioFMaRMn9FBDks5vdQL1gaJpZM4b8NLw
.

--
Julian J. Spergel
Lamont Doherty Earth Observatory
University of Chicago '16
Geophysical Sciences
def satstac(rasterPath,rasterName):
from satstac import Catalog, Collection, Item
import xarray as xr
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
import rasterio
import rasterio.features
import rasterio.warp
from shapely import geometry
import json
import cartopy
class REMAStacItem:
def init(self,rasterPath, rasterName):
self.rasterPath = rasterPath
self.id = rasterName
self.stac_item = {"id": rasterName[:-4],
"rownum": str(rasterName)[0:2],
"colnum": str(rasterName)[3:5],
"type": "Feature",
"bbox": self.calcBBox(),
"geometry": self.calcGeometry(),
"units": 'meters',
"resolution": 8
}
self.stac_item['properties']=self.createProperties_meta()
self.stac_item['links'] = self.create_links()
def calcBBox(self):
with rasterio.open(self.rasterPath) as dataset:
bbox = rasterio.warp.transform_bounds(dataset.crs,
'EPSG:4326',dataset.bounds.left,dataset.bounds.bottom,
dataset.bounds.right,dataset.bounds.top)
#self.bbox=json.loads(json.dumps(geometry.mapping(bbox)))
return json.loads(json.dumps(bbox))
def calcGeometry(self):
with rasterio.open(self.rasterPath) as dataset:
profile = dataset.profile
Read the dataset's valid data mask as a ndarray.
geom = geometry.box(*rasterio.warp.transform_bounds(dataset.crs,
'EPSG:4326',dataset.bounds.left,dataset.bounds.bottom,
dataset.bounds.right,dataset.bounds.top))
self.geometry=json.loads(json.dumps(geometry.mapping(geom)))
return json.loads(json.dumps(geometry.mapping(geom)))
#def createAssetList(self):
pass
def createProperties_meta(self):
meta_prop_dict = {}
with rasterio.open(self.id) as dataset:
meta_prop_dict = dataset.meta
crs = meta_prop_dict['crs']
crs = 'EPSG:'+str(crs.to_epsg())
meta_prop_dict['crs'] = crs
profile = dataset.profile
return meta_prop_dict
if imdPath:
o = urlparse(imdPath)
if o.scheme == 's3':
s3 = boto3.resource("s3")
bucket = o.netloc
key = o.path.lstrip('/')
obj = s3.Object(bucket, key)
body = obj.get()['Body'].read()
root = ET.fromstring(body)
#else:
tree = ET.parse(imdPath)
root = tree.getroot()
#self.metaDataStruct = iterate_OverXML(root, recursionTagList=['IMD',
'IMAGE'])
#elif vrtPath:
with rasterio.open(vrtPath) as src:
self.metaDataStruct = process_nitf_tags(src.tags())
else:
print("no Data Specified")
#self.eoDict = self.processMetaData_To_Properties(self.metaDataStruct,
self.provider, self.license)
#return json.loads(json.dumps(meta_prop_dict))
def write_toJSON(self, filename):
with open(filename, 'w') as fp:
json.dump(self.stac_item, fp, indent=2)
myStacItem = REMAStacItem(rasterPath,rasterName)
filename = myStacItem.stac_item['id']+'.json'
myStacItem.write_toJSON(filename)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rabernat/pangeo-rema/issues/2?email_source=notifications&email_token=ALTXJ3L6SAINUDNEUULRPXTPZ2WVLA5CNFSM4G7Q2LYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXLBUHA#issuecomment-500570652,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALTXJ3IQ7R64NPJNBEGBGDLPZ2WVLANCNFSM4G7Q2LYA
.

--
Jonathan Kingslake
Assistant Professor
Dept. Earth and Environmental Sciences,
Lamont-Doherty Earth Observatory of Columbia University
61 Rt. 9W, Palisades, NY 10964-8000
jkingslake.com
@jkingslake

It's been a little while for me too. I believe you just need to move the
python code I sent to the folder with the REMA tiles in it, run a terminal
with the directory in that same folder with the REMA files, and then run
the line of code I sent. If it doesn't work, maybe we can try it tomorrow
together.
Julian
On Mon, Jun 10, 2019 at 4:16 PM jkingslake<EMAIL_ADDRESS>wrote:

Julian,
Yes, I can do this. Although you will have to remind me the details.
Jonny
On Mon, Jun 10, 2019 at 3:56 PM jjspergel<EMAIL_ADDRESS>wrote:

Hi Jonny,
I forget how long it took to make .json files last time, but if it's a
long
time, would it be possible to run the new code overnight tonight? The
only
change is the removal of the link-forming section. As before use:
forfiles /s /m *_COG_LZW.tif /c "cmd /c python ../../satstac_jsoncode.py
@path https://github.com/path @file https://github.com/file
Let me know if you can run this tonight or not. No real rush, I just want
to schedule work this week. Thanks!
On Wed, Apr 3, 2019 at 3:37 PM jkingslake<EMAIL_ADDRESS>wrote:

Done.
Where should the json files need to sit?
I suspect the answer is that it doesnt matter, but just to check
before I
upload them.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/rabernat/pangeo-rema/issues/2#issuecomment-479629563
,
or mute the thread
<

https://github.com/notifications/unsubscribe-auth/Ao-5EZFtUJezbkyzdL8QioFMaRMn9FBDks5vdQL1gaJpZM4b8NLw

.

--
Julian J. Spergel
Lamont Doherty Earth Observatory
University of Chicago '16
Geophysical Sciences
def satstac(rasterPath,rasterName):
from satstac import Catalog, Collection, Item
import xarray as xr
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
import rasterio
import rasterio.features
import rasterio.warp
from shapely import geometry
import json
import cartopy
class REMAStacItem:
def init(self,rasterPath, rasterName):
self.rasterPath = rasterPath
self.id = rasterName
self.stac_item = {"id": rasterName[:-4],
"rownum": str(rasterName)[0:2],
"colnum": str(rasterName)[3:5],
"type": "Feature",
"bbox": self.calcBBox(),
"geometry": self.calcGeometry(),
"units": 'meters',
"resolution": 8
}
self.stac_item['properties']=self.createProperties_meta()
self.stac_item['links'] = self.create_links()
def calcBBox(self):
with rasterio.open(self.rasterPath) as dataset:
bbox = rasterio.warp.transform_bounds(dataset.crs,
'EPSG:4326',dataset.bounds.left,dataset.bounds.bottom,
dataset.bounds.right,dataset.bounds.top)
#self.bbox=json.loads(json.dumps(geometry.mapping(bbox)))
return json.loads(json.dumps(bbox))
def calcGeometry(self):
with rasterio.open(self.rasterPath) as dataset:
profile = dataset.profile
Read the dataset's valid data mask as a ndarray.
geom = geometry.box(*rasterio.warp.transform_bounds(dataset.crs,
'EPSG:4326',dataset.bounds.left,dataset.bounds.bottom,
dataset.bounds.right,dataset.bounds.top))
self.geometry=json.loads(json.dumps(geometry.mapping(geom)))
return json.loads(json.dumps(geometry.mapping(geom)))
#def createAssetList(self):
pass
def createProperties_meta(self):
meta_prop_dict = {}
with rasterio.open(self.id) as dataset:
meta_prop_dict = dataset.meta
crs = meta_prop_dict['crs']
crs = 'EPSG:'+str(crs.to_epsg())
meta_prop_dict['crs'] = crs
profile = dataset.profile
return meta_prop_dict
if imdPath:
o = urlparse(imdPath)
if o.scheme == 's3':
s3 = boto3.resource("s3")
bucket = o.netloc
key = o.path.lstrip('/')
obj = s3.Object(bucket, key)
body = obj.get()['Body'].read()
root = ET.fromstring(body)
#else:
tree = ET.parse(imdPath)
root = tree.getroot()
#self.metaDataStruct = iterate_OverXML(root, recursionTagList=['IMD',
'IMAGE'])
#elif vrtPath:
with rasterio.open(vrtPath) as src:
self.metaDataStruct = process_nitf_tags(src.tags())
else:
print("no Data Specified")
#self.eoDict = self.processMetaData_To_Properties(self.metaDataStruct,
self.provider, self.license)
#return json.loads(json.dumps(meta_prop_dict))
def write_toJSON(self, filename):
with open(filename, 'w') as fp:
json.dump(self.stac_item, fp, indent=2)
myStacItem = REMAStacItem(rasterPath,rasterName)
filename = myStacItem.stac_item['id']+'.json'
myStacItem.write_toJSON(filename)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/rabernat/pangeo-rema/issues/2?email_source=notifications&email_token=ALTXJ3L6SAINUDNEUULRPXTPZ2WVLA5CNFSM4G7Q2LYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXLBUHA#issuecomment-500570652
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/ALTXJ3IQ7R64NPJNBEGBGDLPZ2WVLANCNFSM4G7Q2LYA
.

--
Jonathan Kingslake
Assistant Professor
Dept. Earth and Environmental Sciences,
Lamont-Doherty Earth Observatory of Columbia University
61 Rt. 9W, Palisades, NY 10964-8000
jkingslake.com
@jkingslake
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rabernat/pangeo-rema/issues/2?email_source=notifications&email_token=AKH3SENWAB5ZU7PMGL3E4VTPZ2Y7VA5CNFSM4G7Q2LYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXLDJOI#issuecomment-500577465,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKH3SENV5BYDTCRAQMJUUK3PZ2Y7VANCNFSM4G7Q2LYA
.

--
Julian J. Spergel
Lamont Doherty Earth Observatory
University of Chicago '16
Geophysical Sciences

The latest code for creating json files is here

I have code to collect the json files that works. Let's meet before implementing it so I can make sure that we have a file structure and metadata that we agree on. then we can upload it all to the Google Bucket!
``
satstac_collection.txt


Missing Transfers in usb analysis

When analyzing an usb trace I noticed that some of the larger bulk transfers are not present in the ViewSB output.
As a comparison I looked at same trace in Wireshark and noticed that the packets are present in the trace but missing:
One example (multiple exists in my trace....)

the IN Transaction from packet #9307-9309 is shown in both Wireshark and ViewSB;
the following packets #9311-9223 are missing in ViewSB (marked in Wireshark)
from packet #9324 onwards (the PING/NAK sequence ViewSB stats showing the data)

Capture device is OpenVizsla, capturing the hs traffic to pcap file (USB link layer packets);  SOF packets filtered out on both ViewSB and Wireshark.
The tui frontend is showing same issue (but more difficult to interpret/compare against the wireshark output):

This sounds like it's related to #14 - those packets have probably been consumed by the decoder but, since the transfer hasn't finished, they haven't been emitted. We need a way to flush the partial transfer when the capture ends.

Indeed likely related to #14, especially considering the comment of xairy who noted similar missing (interrupt) packets.
I doubt that all these missing transfers are partial/incomplete; looking at the packet capture in wireshark I noticed that multiple transfers with different end-points seem be interleaved. My gut feel is that this confuses the ViewSB analyser/grouping logic.

I've extract a fragment from my initial capture in case someone is interested to reproduce this issue:
missing-transfers.pcap.gz

Some further progress.... found that the USBPacketSpecializer is not correctly classifying the USB-2 High-Speed PING packets as Tokens (NAK limiting ping flow control); see diagram from the USB2 spec:

Expect to send a PR later today :-)

@miek, while digging through this issue I start to become aware that you are probably right on the money that this issue is related to the decoder not 'detecting' the end of the transfer.
Thinking about this, in my example I'm looking at an audio device that is effectively streaming data using a couple of bulk endpoints. So from looking at the low-level packets on the USB bus these transfers effectively will never 'complete'. On a higher protocol layer the data is likely transferred in packets that consist of 1 (or more) USB transfers, depending on the configured buffer sizes (probably configured during setup in the device initialization, but potentially even statically defined in the firmware).
When I disable the "USBTransferGrouper" decoder in decoders/grouping.py (setting INCLUDE_IN_ALL = False) I can nicely see the low-level packets that got missing so-far; but this disables most of the great decoding in ViewSB.
Since silently missing packets/transfers defeats the purpose of low-level usb decoding, I start to wonder if the grouping in transfers is always desirable. It almost feels like these transfers would need to be grouped per device/endpoint and shown as individual traces (like on a scope or logic analyzer) -- but this is a completely difference feature and would require discussion with the ViewSB product owner(s).
For now I'll continue to see if I can improve the USBTransferGrouper to group the various (NAK etc.) transactions related into the transfer, but add a way to terminate the Transfer so can actually see what is going on on the bus.
Any feedback/comments appreciated.

Hi @martinling; in my view both the Transfers at usb protocol level as well as the collation at higher protocol level are needed, and useful depending on your specific need.
Lumping both the low-level USB protocol and the higher level application layer together does not make much sense as your effective data packets will be surrounded by the (in my case) many IN/PING packets that get NAK'ed.
I like your example, and this is much in line with what I have in mind as a future improvement: move the higher-level transfers into their own group, and have the low-level usb transfers within. To get this right for pending/ongoing transfers (like in case the stream does not see an end) this will also require quite some GUI work to display these 'pending/in-progress' transfers -- which would solve long-standing solve #10
Another improvement idea I have is to add an aggregator for the consecutive NAK'ed packets (when there are a handful this is still manageable, but I have seen cases where there are many (1000 or so) PINGs before the receiver is ready.

... device streams up to 40MB/s of data on a single bulk endpoint, so about 80,000 512-byte packets per second. Every now and again there is a control transfer. What I'm interested in is the relationship between the control transfers and the data stream.

This raises an interesting challenge; currently the packets are shown as a linear list that has been turned into a tree structure.  When you are interested in analysing events between different end-points, it will be key to be able to see these in a common time-domain -- something the linear list is not really helpful.
What you likely would like to see is an alternate view that allows to show multiple transfers on a common timeline; a bit like a logic analyser or oscilloscope would visualise the time relationship. This view would then show the time-relationship between the events on the bus; and the (existing) detailed view allows to zoom in to the packet details.  Selecting something in the packet list would highlight the corresponding event in the time domain view, and vice versa...  Hope below crude diagram helps illustrate my thought:

Yes, I agree that we want both levels of grouping - the low-level 512B USB transfers, and some higher level grouping of the overall data stream they form. The existing grouping seems to have been a shortcut that wasn't ideal for either purpose. At the protocol level it obscured the low level 512B transfers, and at the higher level view, it caused a continuous data stream to be broken into rather arbitrary chunks.
I think the long standing issue you refer to is #14 rather than #10.
The mismatch between the tree structure presented by the UI and the interleaving of endpoint traffic on the wire is indeed an issue. I think eventually there does need to be a logic analyzer / oscilloscope style view of things as well as you say. For LUNA we also have the possibility to capture other logic signals alongside the USB capture, so this would integrate with that ability.
I think there might be some things that could be done within the existing tree view though, to illustrate the overlapping timings of different operations. For instance, using connecting lines to connect different groups, and using timestamp ranges rather than simply a single timestamp per item. Perhaps it might look something like this:
0.000 - 0.001    [+] Vendor request #1
0.002 - 1.234 ┌──[+] EP 1 IN: 19.6MB (40295 transfers of 512B)
0.003 - 1.123 │┌─[+] EP 2 OUT: 3.6MB (7345 transfers of 512B)
1.125 - 1.126 ││ [+] Vendor request #2
1.127 - 2.345 ├┼─[+] EP 1 IN: 6.1MB (12456 transfers of 512B)
1.128 - 2.234 │├─[+] EP 2 OUT: 19.6MB (40295 transfers of 512B)
2.400 - 2.401 ││ [+] Vendor request #3
2.402 - 3.345 └┼─[+] EP 1 IN: 6.1MB (12456 transfers of 512B)
3.500 - 3.501  │ [+] Vendor request #4
3.678 - now    └─[+] EP2 OUT: 16.3MB (33295 transfers of 512B)

One thing to note is that a "transfer" means something specific in the USB spec, it is made up of one or more data packets and is completed by a short packet - so labelling each individual data transaction as a "transfer" is misleading.

One thing to note is that a "transfer" means something specific in the USB spec, it is made up of one or more data packets and is completed by a short packet - so labelling each individual data transaction as a "transfer" is misleading.

Would you mind sharing a link/reference to the USB spec where this is defined? I'm primarily relying on the USB Spec revision 2.0 (April 27, 2000), and definitely want to stay true to the specification and avoid confusion or even being misleading.
Could it be you're referring to the I/O Request Packets (IRPs), since these use a 'short package' to signal completion in variable-size data amounts?  Anyway, I'm more than happy to learn and up-skill my understanding of the USB protocol.
My interpretation is that if we start from the data captured on the bus (that is, for capture devices that tap into the physical layer (USB Wire), with devices like OpenVizsla, Rhododendron, Luna etc.), and work our way up; we we'll find for our data transfers USB packets, that make up Transactions (USB Framed Data), that make up Transfers (USB Framed Data).  (ie. the flows on left hand corner on figure 5-9)

Would you mind sharing a link/reference to the USB spec where this is defined? I'm primarily relying on the USB Spec revision 2.0 (April 27, 2000), and definitely want to stay true to the specification and avoid confusion or even being misleading.

It's described for each transfer type under 5.5.3, 5.6.3, 5.7.3, and 5.8.

Could it be you're referring to the I/O Request Packets (IRPs), since these use a 'short package' to signal completion in variable-size data amounts? Anyway, I'm more than happy to learn and up-skill my understanding of the USB protocol.

In a way, because they're closely related. An IRP is only a term to describe a request for a data transfer from the host software, it doesn't actually describe anything happening on the bus and any details of it are defined by the operating system (it might correspond to calling libusb_submit_transfer for example). So, when a transfer is completed by a short packet, the request for that transfer completes too.

Thanks for the feedback. I’ve updated my code to keep the transfers grouped as-is by default; and added an optional command-line argument to fragment the transfers when a specified number of bytes has been transferred.
This offers a workable workaround until we have a proper solution to visualize the ‘in progress transfers’ (issue #14) for the scenarios where transfers are huge or endless streaming.

Closed in preparation for archiving the repository. We have replaced ViewSB with Packetry.


WIP - Fix regressions for LXD runners

Applicable spec: n/a
Overview

Rationale
LXD scheduled integration tests are currently failing.
Module Changes

Library/Dependency Changes

Checklist

[ ] The contributing guide was applied
[ ] The documentation is generated using src-docs
[ ] The PR is tagged with appropriate label (urgent, trivial, complex)
[ ] The library version in pyproject.toml is incremented

The .env file used by lxd is in github-runner-operator.


Web pack failed to compile

ERROR in Could not resolve module......
Command

ng build
ng serve

Description

I cloned working angular 4 project. Then upgraded the package.json file to the latest version of Angular 7 .
But when I run ng build or ng serve, It displays series of error. I tried to use relative path for layout-routing.module.ts but the error persists
Kindly help me on this.
Thanks in advance.
Minimal Reproduction

Simple steps to reproduce this bug.
ng build
or
ng serve --host <IP_ADDRESS> --port 8080 --disableHostCheck
Error

ERROR in Could not resolve module app/optometrists-pool/optometrists-pool.module relative to app/layout/layout-routing.module.ts
ERROR in ./src/assets/css/style.css (./node_modules/@angular-devkit/build-angular/src/angular-cli-files/plugins/raw-css-loader.js!./node_modules/postcss-loader/src??embedded!./src/assets/css/style.css)
Module Error (from ./node_modules/postcss-loader/src/index.js):
(Emitted value instead of an instance of Error) CssSyntaxError: /home/ubuntu/wicaredev/wicare2/wicare-practice/src/assets/css/style.css:27:13: Can't resolve './../fonts/Apple%20Chancery.eot' in '/home/ubuntu/wicaredev/wicare2/wicare-practice/src/assets/css'

  25 | @font-face {
  26 |   font-family: 'Apple Chancery';
> 27 |   src: url('./../fonts/Apple Chancery.eot');
     |             ^
  28 |   src: url('./../fonts/Apple Chancery.woff2') format('woff2'),
  29 |        url('./../fonts/Apple Chancery.eot?#iefix') format('embedded-opentype');

ERROR in ./src/assets/css/style.css (./node_modules/@angular-devkit/build-angular/src/angular-cli-files/plugins/raw-css-loader.js!./node_modules/postcss-loader/src??embedded!./src/assets/css/style.css)
Module Error (from ./node_modules/postcss-loader/src/index.js):
(Emitted value instead of an instance of Error) CssSyntaxError: /home/ubuntu/wicaredev/wicare2/wicare-practice/src/assets/css/style.css:28:13: Can't resolve './../fonts/Apple%20Chancery.woff2' in '/home/ubuntu/wicaredev/wicare2/wicare-practice/src/assets/css'

  26 |   font-family: 'Apple Chancery';
  27 |   src: url('./../fonts/Apple Chancery.eot');
> 28 |   src: url('./../fonts/Apple Chancery.woff2') format('woff2'),
     |             ^
  29 |        url('./../fonts/Apple Chancery.eot?#iefix') format('embedded-opentype');
  30 |   font-weight: normal;

ERROR in ./src/assets/css/style.css (./node_modules/@angular-devkit/build-angular/src/angular-cli-files/plugins/raw-css-loader.js!./node_modules/postcss-loader/src??embedded!./src/assets/css/style.css)
Module Error (from ./node_modules/postcss-loader/src/index.js):
(Emitted value instead of an instance of Error) CssSyntaxError: /home/ubuntu/wicaredev/wicare2/wicare-practice/src/assets/css/style.css:29:13: Can't resolve './../fonts/Apple%20Chancery.eot' in '/home/ubuntu/wicaredev/wicare2/wicare-practice/src/assets/css'

  27 |   src: url('./../fonts/Apple Chancery.eot');
  28 |   src: url('./../fonts/Apple Chancery.woff2') format('woff2'),
> 29 |        url('./../fonts/Apple Chancery.eot?#iefix') format('embedded-opentype');
     |             ^
  30 |   font-weight: normal;
  31 |   font-style: normal;

ERROR in multi ./assets/css/bootstrap.min.css ./node_modules/font-awesome/css/font-awesome.min.css ./node_modules/primeng/resources/primeng.min.css ./node_modules/primeng/resources/themes/omega/theme.css ./src/assets/css/animate.css ./src/assets/css/style.css ./src/assets/css/dev.css
Module not found: Error: Can't resolve '/home/ubuntu/wicaredev/wicare2/wicare-practice/assets/css/bootstrap.min.css' in '/home/ubuntu/wicaredev/wicare2/wicare-practice'
ℹ ｢wdm｣: Failed to compile.

Desired Behaviour

Compiled successfully
ng v

Angular CLI: 7.3.4
Node: 10.15.1
OS: linux x64
Angular: 7.2.7
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.13.4
@angular-devkit/build-angular     0.11.4
@angular-devkit/build-optimizer   0.11.4
@angular-devkit/build-webpack     0.11.4
@angular-devkit/core              7.3.4
@angular-devkit/schematics        7.3.4
@angular/cli                      7.3.4
@ngtools/webpack                  7.1.4
@schematics/angular               7.3.4
@schematics/update                0.13.4
rxjs                              6.4.0
typescript                        3.2.4
webpack                           4.29.6

layout-routing.module.ts

   {
                path: 'optometry-pool',
                loadChildren: 'app/optometrists-pool/optometrists-pool.module#OptometristsPoolModule',
            }

tsconfig.app.json

{
  "extends": "../tsconfig.json",
  "compilerOptions": {
    "outDir": "../out-tsc/app",
    "baseUrl": "./",
    "module": "commonjs",
    "types": []
  },
  "exclude": [
    "test.ts",
    "**/*.spec.ts"
  ]
}

Development IDE

AWS Cloud9 IDE

Hi, lazy routes to relative source files must start with ./
See: https://angular.io/guide/lazy-loading-ngmodules
If the problem persists please file a new issue with a minimal repro.
You can read here why this is needed. A good way to make a minimal repro is to create a new app via ng new repro-app and adding the minimum possible code to show the problem. Then you can push this repository to github and link it here.
This might be related to your directory structure so its really important to get an accurate repro to diagnose this.
Thanks

Thank you for your response @alan-agius4. I already tried changing the absolute path to relative path  before submitting the query.
But am gonna give the second suggestion a trial.

Thank you @alan-agius4 as you pointed out that issue might be related to the directory. Got it resolved.
The webpack compilation is almost successful, now displaying this:

Please does that mean I need to downgrade the Angular CLI?
Below is my angular cli  version:


Update threat-analytics.md

Added Required permissions and roles to access threat analytics

issue#https://github.com/MicrosoftDocs/microsoft-365-docs/issues/7094

@JohanFreelancer9  Please copy review

@schmurky Please merge this PR. Thanks

Hi @DulceMontemayor can you check, this is regarding TA permissions.

@DulceMontemayor Please

@yogkumgit i need to check this with the PM as the MDE and M365D experience might be a bit different. I'll get back to you as soon as i get his feedback

@DulceMontemayor Awaiting your kind response. Thanks

@yogkumgit i've approved it but pls update the branch because it's out of date with the base branch. Thanks!

@DulceMontemayor i have updated the branch. Thank you

@tiburd Please merge this PR. Thanks


NTR: non-membrane-bounded organelle formation

Please provide as much information as you can:

Suggested term label:
non-membrane-bounded organelle formation

Definition (free text)

LD
'cellular component assembly'
and (results_in_assembly_of some non-membrane-bounded organelle)

Reference, in format PMID:#######
PMID: 28225081

Gene product name and ID to be annotated to this term

Parent term(s)
GO:0070925 organelle assembly

Children terms (if applicable) Should any existing terms that should be moved underneath this new proposed term?
(inferred):
'bacterial-type flagellum assembly'
'cellulosome assembly'
'centriole assembly'
'kinetochore assembly'
'lipid droplet formation'
'myofibril assembly'
'P granule assembly'
'P-body assembly'
'podosome assembly'
'proteasome storage granule assembly'
'ribosome assembly'
'septin ring assembly'
'spindle assembly'
'stress granule assembly'
'terminal web assembly'

Synonyms (please specify, EXACT, BROAD, NARROW or RELATED)
+synonym: "non-membrane-bounded organelle assembly" EXACT []
+synonym: "non-membrane-bounded organelle formation" EXACT []
+synonym: "non-membrane-enclosed organelle assembly" EXACT []
+synonym: "non-membrane-enclosed organelle formation" EXACT []

Cross-references

For enzymes, please provide RHEA and/or EC numbers.

Can also provide MetaCyc, KEGG, Wikipedia, and other links.

Any other information

For IDPO alignment - requested by DisProt @federicaquaglia @esalladini

+[Term]
+id: GO:0140694
+name: non-membrane-bounded organelle assembly
+def: "The aggregation, arrangement and bonding together of a set of components to form a non-membrane-bounded organelle." [PMID:28225081]
+synonym: "non-membrane-bounded organelle assembly" EXACT []
+synonym: "non-membrane-bounded organelle formation" EXACT []
+synonym: "non-membrane-enclosed organelle assembly" EXACT []
+synonym: "non-membrane-enclosed organelle formation" EXACT []
+intersection_of: GO:0022607 ! cellular component assembly
+intersection_of: results_in_assembly_of GO:0043228 ! non-membrane-bounded organelle
+property_value: term_tracker_item https://github.com/geneontology/go-ontology/issues/21939 xsd:anyURI
+created_by: pg
+creation_date: 2021-08-09T15:30:06Z

Children terms (if applicable) Should any existing terms that should be moved underneath this new proposed term?
(inferred):

No
'bacterial-type flagellum assembly'
'centriole assembly'
'kinetochore assembly'
'myofibril assembly'
'ribosome assembly'
'septin ring assembly'
'spindle assembly'
Yes
'P granule assembly'
'P-body assembly'
'proteasome storage granule assembly'
'stress granule assembly'
Don't know
'cellulosome assembly'
'lipid droplet formation'
'podosome assembly'
'terminal web assembly'
As an aside, I worry a little about these "formation terms" being used for  phenotypes for any member of a condensate/stress granule etc which if not present the granule does not form correctly. This is a bit like annotating a mutated subunit of the proteasome to proteasome assembly.
IMHO to use such terms something should be either a) a chaperone (not part of the final complex/condensate) or be shown to be a substrate in some regulation (i.e. modification dependent).

As an aside, I worry a little about these "formation terms" being used for phenotypes for any member of a condensate/stress granule etc which if not present the granule does not form correctly.

Right, we discussed this- this is a general problem for all assembly terms. In this case there is a MF, molecular condensate scaffold activity, that serves to assemble to condensate, at least in some cases. The review that @esalladini provided explains this quite nicely PMID:28225081

WRT the terms you say should not be children:
No
'bacterial-type flagellum assembly'
'centriole assembly'
'kinetochore assembly'
'myofibril assembly'
'ribosome assembly'
'septin ring assembly'
'spindle assembly'
they classify automatically because they themselves are non-membrane-bounded organelles. Why do you think they are not relevant there?

I don't think they are "condensates" at least flagellum,  ribosome, and spindle have a defined stoichiometry, or defined structure.  I'm less sure about kinetochore but I don't think it is a condensate- but perhaps it is-  I will ask.
We  already  said "condensate" is narrower than "are non-membrane-bounded organelles", so maybe we really do need to define and add the term.

The flagellum, kinetochore,  ribosome, spindle have a very defined, regulated, step-wise assembly pathway.
I thought that  "condensate" formation  was more of an intrinsic property of the proteins. Although I guess this is "fuzzy".
I still think we need to be wary of calling everything  that is not membrane-bound a condensate.

Right, we discussed this- this is a general problem for all assembly terms. In this case there is a MF, molecular condensate scaffold activity, that serves to assemble to condensate, at least in some cases. The review that @esalladini provided explains this quite nicely PMID:28225081

Right. But I would not annotate a scaffold  that was part of a complex to a BP "protein complex assembly" if the complex is self-organizing. If activity needs to happen (i.e.  a modification) to make  the scaffold function as a scaffold then I think that it is part of an 'active process of assembly'. Otherwise it is only a scaffold  involved in a signalling pathway (or some other pathway).  Also we requested scaffold  terms before but these were prevented. We had to use molecular adaptor (even though I think they are slightly  different things, and some  proteins like MAP kianse scaffolds are both scaffolds and adaptors). I'll add these comments to the scaffold ticket.
By

Maybe I am over-thinking this. It's just that most of the annotation I look at for these "assembly" processes the evidence is poor. In some cases the expression level drives condensate formation. However in others there is some active step-wise process.  That seems to be what the review says too. It's important only to annotate the active processes (and that it aligns with what we mean by a GO  BP). I guess in most cases I don't see a difference in the practice  that has been discussed for protein complex assembly.
For example
we don't annotate
MAP-kinase scaffold activity
The binding activity of a molecule that functions as a physical support for the assembly of a multiprotein mitogen-activated protein kinase (MAPK) complex. Binds multiple kinases of the MAPKKK cascade, and also upstream signaling proteins, permitting those molecules to function in a coordinated way. Bringing together multiple enzymes and their substrates enables the signal to be transduced quickly and efficiently.
to a term "MAPK complex assembly" (there is no such term), but we annotate it to the signalling process.
This was decided back at the signalling meeting in 2019, but maybe we want to do it some other way? Whatever, we should be consistent -I find it very confusing that  assembly of condensates and complexes would be treated differently.  I'm also a bit disturbed that we pull in so many inferred annotations to "assembly of non membrane-bound complexes" to gene products for which there seems to be no evidence  for  a direct role in this process.
(Note that these concerns predated any of these recent  new term request in this area, I just didn't get around to reporting them yet -these changes actually make things clearer, and I think the scaffold MF term will be useful, it's the process I have a problem with in some cases).
I guess it does no harm to let this roll along for a bit but this seems to be one of those areas where everything which interacts with something within a complex or condensate  during its formation would become annotated to its "assembly".

In this specific case all I did was to create a grouping term for already existing 'non-membrane bounded organelle assembly' terms.
But I agree evidence is often lacking, and certainly knockouts should not be used as evidence for this process.

In this specific case all I did was to create a grouping term for already existing 'non-membrane bounded organelle assembly' terms.

I know, I'll leave this for now....

Based on this paper, I would include kinetochore in the condensate list
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7341897/


Enable openjdk-systemtest on all Linux platforms in Adopt

Spread sheet with the matrix of platforms: https://docs.google.com/spreadsheets/d/1X4CCfvMoCgEavRbvejHrTvPnqj37MB-_C6LB6b8Akkc/edit#gid=1690526288

All Linux builds have been created, but the ones for which we have no build or platform have been kept disabled.


Is there a way to access object's fields using a variable (string) as key ?

Hey there ! First, great job with ReScript ! 👍
I'm trying to read the dependencies field of a package.json file and to iterate on its keys. As I do not known which keys will be in my dependencies object, I'm typing it 'a. But I can't figure out how to access the value corresponding to each key while iterating:
@bs.val external __dirname: string = "__dirname"

@bs.module("fs")
external readFileSync: (string, string) => string = "readFileSync"

@bs.scope("JSON") @bs.val
external parseJSON: string => 'a = "parse"

let package = readFileSync(`${__dirname}/package.json`, "utf8")
let parsed = parseJSON(package)
let dependencies = parsed["dependencies"]

Js.Obj.keys(dependencies)->Js.Array2.forEach(name => Js.Log(dependencies[name])) // <-- the error happens here

I get the following error on dependencies[name]:
This has type: string
Somewhere wanted: int

You can convert string to int with Belt.Int.fromString.

Is there any way to do such a thing ?

Hi @mdubourg001 you might like to post questions like these on the ReScript forum where more people will see it. This issue tracker is for issues with the rescript-lang.org site.
Regarding your question, a quick version may be something like this:
type package = {dependencies: Js.Dict.t<string>}

@bs.module external package: package = "../package.json"

Js.Dict.keys(package.dependencies)->Js.Array2.forEach(name => {
  Js.log(name)
})

But you may get better solutions from others on the forum.

Oh I'm sorry, I did not know there was a forum, I'll post my question there.
Anyway, thanks for your fast answer. 👌


Look at setting up the Cloudtrail to Parquet so we can search using athena.

https://github.com/alsmola/cloudtrail-parquet-glue

Do we still want to do this now that we have cloudtrail going to sentinel?


virtual slots should be marked on the left side of the tree

Currently virtual slots for non-power-of-two arenas are marked on the right side of the tree. This makes find_free prefer the right side for allocations (until full). Nothing wrong with that as far as regular usage goes but if there are actual allocations on the right it makes downsizing the arena impossible. If the virtual slots were put on the left side of the tree the bias will be on the left which will allow for downsizing on the right (provided that it is free of course).

This will unfortunately not work as on a left-biased tree there can be slots that will get dis-joined if the tree is expanded upwards.


[Draft] Sailthru Consent Connector

Closes https://github.com/ethyca/fidesplus/issues/518
❗ ROUGH DRAFT - UNTESTED
Code Changes

[ ] list your code changes here

Steps to Confirm

[ ] list any manual steps for reviewers to confirm the changes

Pre-Merge Checklist

[ ] All CI Pipelines Succeeded
Documentation:

[ ] documentation complete, PR opened in fidesdocs
[ ] documentation issue created in fidesdocs

[ ] Issue Requirements are Met
[ ] Relevant Follow-Up Issues Created
[ ] Update CHANGELOG.md

Description Of Changes
First commit - add an initial rough draft of the sailthru consent connector - untested and subject to change. (Credentials needed)
Adds a sailthru authentication override to support https://getstarted.sailthru.com/developers/api-basics/technical/#Authentication_Requests

Based on the docs, I thought this is what we could do for Sailthru.
On Opt-out

Optout_email : basic
Sms_marketing_status : opt-out
Sms_transactional_status : opt-out
Cookies : set to empty

On Opt-in
Optout_email : none
Sms_marketing_status : opt-in
Sms_transactional_status : opt-in
Cookies : no action

Hey @saiatethyca thanks for the feedback here.  sms_marketing_status and optout_email line up. I'll look into adding cookies.
Are you sure about sms_transactional_status?  transactional communications seem different than marketing communcations.  https://getstarted.sailthru.com/messaging/triggered-messages/transactional-overview/ Just double-checking here.

Thanks, Dawn for the response!
I agree transaction seems aggressive. I'll see if I can get the client's response to this question.

In the API docs it looks like if you're adding cookies, it's something like this, but I can't find anywhere that an empty dict {} removes the cookies?
{
<EMAIL_ADDRESS>    "cookies":
        {
            "sailthru_content":"U9JEVqLefidFAAAA"
        }
}

Closing - this PR is pretty far out of date.  If we get creds for sailthru we can cherry-pick the relevant commit from this branch and continue from there.


Starlark function generator

I like the way Starlark functions are really easy to write and very short (< 10 lines of code).
I want to be able to publish functions (to the kpt-function-catalog) written in Starlark with minimal effort.
I also need to be able to provide input variables to Starlark functions, so they can be used as validators & mutators.
Since almost all the code and Dockerfile would be boilerplate, it seems like a generator functions could be created, like:
kpt fn generate <name> --lang=starlark

This would generate the Dockerfile and a placeholder file for the starlark code, along with a script or makefile for building and publishing the image.

@karlkfi Why do you need a Dockerfile here? It sounds like you want to have the starlark function with a baked-in starlark script and then publish it to the catalog?
The idea of starlark function is that we provide the image and user provide the functionConfig.
Publishing the functionConfig of starlark function is equivalent to publishing a function, since that's the only thing that changes.
We can potentially have a separate index for starlark functionConfig.
@droot @karlkfi WDYT?

Why do you need a Dockerfile here? It sounds like you want to have the starlark function with a baked-in starlark script and then publish it to the catalog?

Yes

Publishing the functionConfig of starlark function is equivalent to publishing a function

No.
Publishing a function as a yaml file in a git repo is NOT equivalent to publishing it as a docker image. With a docker image you can include the usage of the function in any kptfile pipeline.
If you can make kptfile reference a yaml file in a git repo without needing to publish it as an image, then maybe they would be more equivalent. That would certainly be easier. Otherwise they need to be published as container images, right?

I agree, having a light weight skylark sdk that scaffolds the Dockerfile and function script and some make helpers to dev and publish the starlark script as function makes sense. It will be sort of a parallel to our typescript and Go SDKs for writing functions.

Another option would be to be able to reference a function by URL, instead of just container image. Then I could "publish" a starlark function just by hosting it in a github repo.
The downside is that it would be effectively as secure as sudo curl|bash (not that a container published by someone else is much better).


Support yield keyword

Fix #4309

Those files are auto-generated by xtask. To support yield we have to add a YieldExpr rule to the ungrammar file and the Expr rule. Then regenerate these files here and adjust the parser.

Those files are auto-generated by xtask. To support yield we have to add a YieldExpr rule to the ungrammar file and the Expr rule. Then regenerate these files here and adjust the parser.

ungrammar 1.7 is out.

ungrammar 1.7 is out.

As a heads up since you haven't touched that part here before either, to make the parser support the yield syntax you will have to add the parsing for it at the correct places in the https://github.com/rust-analyzer/rust-analyzer/blob/master/crates/parser/src/grammar/expressions/atom.rs module.

As a heads up since you haven't touched that part here before either, to make the parser support the yield syntax you will have to add the parsing for it at the correct places in the https://github.com/rust-analyzer/rust-analyzer/blob/master/crates/parser/src/grammar/expressions/atom.rs module.

@Veykril  Thanks for guiding!

@Veykril  Thanks for guiding!

lgtm 👍
bors r+

lgtm 👍
bors r+


[staging-next] llvmPackages_19.libclc: fix cross compilation

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ofborg build pkgsCross.aarch64-multiplatform.llvmPackages_19.libclc pkgsCross.aarch64-multiplatform.llvmPackages_18.libclc

mesa build still fails, requires:
diff --git a/pkgs/development/libraries/mesa/default.nix b/pkgs/development/libraries/mesa/default.nix
index 5e2651b764b3..310007311848 100644
--- a/pkgs/development/libraries/mesa/default.nix
+++ b/pkgs/development/libraries/mesa/default.nix
@@ -138,6 +138,7 @@ in stdenv.mkDerivation {

   patches = [
     ./opencl.patch
+    ./mesa-clc-cross.patch
   ];

   postPatch = ''
diff --git a/pkgs/development/libraries/mesa/mesa-clc-cross.patch b/pkgs/development/libraries/mesa/mesa-clc-cross.patch
new file mode 100644
index<PHONE_NUMBER>00..e94c136ce385
--- /dev/null
+++ b/pkgs/development/libraries/mesa/mesa-clc-cross.patch
@@ -0,0 +1,13 @@
+diff --git a/src/compiler/clc/meson.build b/src/compiler/clc/meson.build
+index 74767d08d..1194e0e0f 100644
+--- a/src/compiler/clc/meson.build
++++ b/src/compiler/clc/meson.build
+@@ -124,8 +124,4 @@ prog_mesa_clc = executable(
+   c_args : [pre_args, no_override_init_args],
+   link_args : [ld_args_build_id],
+   dependencies : [idep_mesaclc, dep_llvm, dep_spirv_tools, idep_getopt],
+-  # If we can run host binaries directly, just build mesa_clc for the host.
+-  # Most commonly this happens when doing a cross compile from an x86_64 build
+-  # machine to an x86 host
+-  native : not meson.can_run_host_binaries(),
+ )

That didn't build for me:
[2903/3039] Generating builtins.opt.clspv--.bc
/nix/store/ypkf4zwhgja2b0sd9mvhwzycn4kx0ldq-llvm-19.1.5/bin/opt: WARNING: failed to create target machine for 'spir-unknown-unknown': unable to get target for 'spir-unknown-unknown', see --version and --triple.
[2904/3039] Generating clspv--.bc
FAILED: clspv--.bc /build/libclc-src-19.1.5/libclc/build/clspv--.bc
cd /build/libclc-src-19.1.5/libclc/build && prepare_builtins -o clspv--.bc builtins.opt.clspv--.bc
/bin/sh: prepare_builtins: not found
[2905/3039] Generating builtins.opt.clspv64--.bc
/nix/store/ypkf4zwhgja2b0sd9mvhwzycn4kx0ldq-llvm-19.1.5/bin/opt: WARNING: failed to create target machine for 'spir64-unknown-unknown': unable to get target for 'spir64-unknown-unknown', see --version and --triple.
[...]
ninja: build stopped: subcommand failed.

The warnings are unrelated

/bin/sh: prepare_builtins: not found

Ought to be fixed by this. Wrong commit?

That's quite possible.


pin go-control-plane so contour continues to build

A transient dependency has a breaking change in a pre-release version - this pins that dependency so contour continues to build
failure here: https://github.com/knative-sandbox/net-contour/pull/628

/lgtm


Umount failed by kubelet in K8S 1.18.14 and ubuntu

What happened:
kubet display errors to umount volume fuse and volume has not point of mount (if is located folder :
/var/lib/kubelet/pods/c5936d69-59ff-46e9-9054-50056b4b2d3d/volumes/azure~blobfuse/fuse
The folder has data , is the blobcontainer with its folder listed ok, but there is not any point of mount related with that... because never a pount of mount is created but if is local folder is connected with blobfuse data successfull, so the proccess of mount is not complete, so the umount always fails as well.
How to find errors? --> from agent node performs this tail:
 tail -2000 /var/log/syslog |grep fuse
Feb 10 16:55:07 aks-memoptimized-28286210-vmss00000C kubelet[16774]: I0210 16:55:07.829276   16774 reflector.go:181] Stopping reflector *v1.Secret (0s) from object-"logstreaming"/"blobfusecreds"
Feb 10 16:55:07 aks-memoptimized-28286210-vmss00000C kubelet[16774]: I0210 16:55:07.874833   16774 reconciler.go:196] operationExecutor.UnmountVolume started for volume "fuse" (UniqueName: "flexvolume-azure/blobfuse/c5936d69-59ff-46e9-9054-50056b4b2d3d-fuse") pod "c5936d69-59ff-46e9-9054-50056b4b2d3d" (UID: "c5936d69-59ff-46e9-9054-50056b4b2d3d")
Feb 10 16:55:07 aks-memoptimized-28286210-vmss00000C kubelet[16774]: E0210 16:55:07.883581   16774 nestedpendingoperations.go:301] Operation for "{volumeName:flexvolume-azure/blobfuse/c5936d69-59ff-46e9-9054-50056b4b2d3d-fuse podName:c5936d69-59ff-46e9-9054-50056b4b2d3d nodeName:}" failed. No retries permitted until 2021-02-10 16:55:08.383547099 +0000 UTC m=+469.396197557 (durationBeforeRetry 500ms). Error: "UnmountVolume.TearDown failed for volume \"fuse\" (UniqueName: \"flexvolume-azure/blobfuse/c5936d69-59ff-46e9-9054-50056b4b2d3d-fuse\") pod \"c5936d69-59ff-46e9-9054-50056b4b2d3d\" (UID: \"c5936d69-59ff-46e9-9054-50056b4b2d3d\") : remove /var/lib/kubelet/pods/c5936d69-59ff-46e9-9054-50056b4b2d3d/volumes/azure~blobfuse/fuse: directory not empty"
Feb 10 16:55:08 aks-memoptimized-28286210-vmss00000C kubelet[16774]: I0210 16:55:08.476582   16774 reconciler.go:196] operationExecutor.UnmountVolume started for volume "fuse" (UniqueName: "flexvolume-azure/blobfuse/c5936d69-59ff-46e9-9054-50056b4b2d3d-fuse") pod "c5936d69-59ff-46e9-9054-50056b4b2d3d" (UID: "c5936d69-59ff-46e9-9054-50056b4b2d3d")

What you expected to happen:
mount and mount sucessfull like in K8s version 1.17.x (is ok in that version)
How to reproduce it:
create a pod with a node in k8s version (aks) 1.18.14 and 18.04.1-Ubuntu SMP  x86_64 x86_64 x86_64 GNU/Linux
        volumeMounts:
        - mountPath: /var/lib/folderNmounted
          name: fuse
          subPath: folderInBlob```
        [..]
      volumes:
      - flexVolume:
          driver: azure/blobfuse
          options:
            container: name-of-container
            mountoptions: --file-cache-timeout-in-seconds=120
            tmppath: /tmp/blobfuse
          secretRef:
            name: blobfusecreds

**Anything else we need to know?**:

**Environment**:
- Kubernetes version (use `kubectl version`):
- Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.14", GitCommit:"5de7fd1f9555368a86eb0f8f664dc58055c17269", GitTreeState:"clean", BuildDate:"2021-01-18T09:31:01Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

- OS (e.g. from /etc/os-release): aks nodes 18.04.1-Ubuntu SMP x86_64 GNU/Linux

- Kernel (e.g. `uname -a`): 5.4.0-1036-azure #38~18.04.1-Ubuntu SMP Wed Jan 6 18:26:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

- Install tools:
- Others:

are you using latest 1.0.17 version? We did a workaround fix for this issue a few months ago:
https://github.com/Azure/kubernetes-volume-drivers/pull/93

are you using latest 1.0.17 version? We did a workaround fix for this issue a few months ago:
#93

Yes, I installed latest: mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.17
But same error is reproduced again like is reported in #93

I have local files connected yet with blob, but is not any point of mount:
ls -la /var/lib/kubelet/pods/0d290342-e496-47b7-b04f-53943eba8d39/volumes/azure~blobfuse/fuse
total 12
drwxr-x--- 3 root root 4096 Feb 10 02:20 .
drwxr-x--- 3 root root 4096 Feb 10 02:20 ..
drwxr-x--- 3 root root 4096 Feb 10 02:20 data

So your latest fix,  the IF could not be working waiting != 0 ...
        if [ "$?" != "0" ]; then
                echo "`date` EXEC: rm -r --one-file-system ${MNTPATH}" >> $LOG
                rm -r --one-file-system "${MNTPATH}" >> $LOG 2>&1
        fi

Because my file: cat /var/log/blobfuse-driver.log |grep -v Success
Only reports this kind of messages, nothing about " EXEC:rm ":
/blobfuse-influx -o allow_other  --file-cache-timeout-in-seconds=120
Wed Feb 10 13:47:55 UTC 2021 EXEC: mkdir -p /var/lib/kubelet/pods/579c2832-6e31-4dbf-bc65-85242cb42c04/volumes/azure~blobfuse/fuse
Wed Feb 10 13:47:55 UTC 2021 INF: AZURE_STORAGE_ACCESS_KEY is set
Wed Feb 10 13:47:55 UTC 2021 INF: export storage account - export AZURE_STORAGE_ACCOUNT=************
Wed Feb 10 13:47:55 UTC 2021 EXEC: blobfuse /var/lib/kubelet/pods/579c2832-6e31-4dbf-bc65-85242cb42c04/volumes/azure~blobfuse/fuse --container-name=logstreaming-pro-blob-metrics --tmp-path=/tmp/blobfuse-influx -o allow_other  --file-cache-timeout-in-seconds=120

what's the result of findmnt -n ${MNTPATH} 2>/dev/null | cut -d' ' -f1 ?
https://github.com/Azure/kubernetes-volume-drivers/blob/5b19277fd15011908c562cd31eee9113575a1623/flexvolume/blobfuse/deployment/blobfuse-flexvol-installer/blobfuse#L28

The result is empty :
***@aks-*********-28286210-vmss000000:/#
***@aks-*********-28286210-vmss000000:/# findmnt --verbose /var/lib/kubelet/pods/579c2832-6e31-4dbf-bc65-85242cb42c04/volumes/azure~blobfuse/fuse
***@aks-*********-28286210-vmss000000:/#
***@aks-*********-28286210-vmss000000:/#

But if you list the folder, is displayed remote blob files:
# ls -la /var/lib/kubelet/pods/579c2832-6e31-4dbf-bc65-85242cb42c04/volumes/azure~blobfuse/fuse
total 12
drwxr-x--- 3 root root 4096 Feb 10 13:47 .
drwxr-x--- 3 root root 4096 Feb 10 13:47 ..
drwxr-x--- 2 root root 4096 Feb 10 13:47 influxdb

The result is empty :
***@aks-*********-28286210-vmss000000:/#
***@aks-*********-28286210-vmss000000:/# findmnt --verbose /var/lib/kubelet/pods/579c2832-6e31-4dbf-bc65-85242cb42c04/volumes/azure~blobfuse/fuse
***@aks-*********-28286210-vmss000000:/#
***@aks-*********-28286210-vmss000000:/#

But if you list the folder, is displayed remote blob files:
# ls -la /var/lib/kubelet/pods/579c2832-6e31-4dbf-bc65-85242cb42c04/volumes/azure~blobfuse/fuse
total 12
drwxr-x--- 3 root root 4096 Feb 10 13:47 .
drwxr-x--- 3 root root 4096 Feb 10 13:47 ..
drwxr-x--- 2 root root 4096 Feb 10 13:47 data

So I update the continuous error message displayed in syslog for more information of this case:
Feb 15 08:52:23 aks-*********-28286210-vmss000000 kubelet[4301]: I0215 08:52:23.885457    4301 reconciler.go:196] operationExecutor.UnmountVolume started for volume "fuse" (UniqueName: "flexvolume-azure/blobfuse/0d290342-e496-47b7-b04f-53943eba8d39-fuse") pod "0d290342-e496-47b7-b04f-53943eba8d39" (UID: "0d290342-e496-47b7-b04f-53943eba8d39")
Feb 15 08:52:23 aks-*********-28286210-vmss000000 kubelet[4301]: E0215 08:52:23.892043    4301 nestedpendingoperations.go:301] Operation for "{volumeName:flexvolume-azure/blobfuse/0d290342-e496-47b7-b04f-53943eba8d39-fuse podName:0d290342-e496-47b7-b04f-53943eba8d39 nodeName:}" failed. No retries permitted until 2021-02-15 08:54:25.892004845 +0000 UTC m=+520881.230263167 (durationBeforeRetry 2m2s). Error: "UnmountVolume.TearDown failed for volume \"fuse\" (UniqueName: \"flexvolume-azure/blobfuse/0d290342-e496-47b7-b04f-53943eba8d39-fuse\") pod \"0d290342-e496-47b7-b04f-53943eba8d39\" (UID: \"0d290342-e496-47b7-b04f-53943eba8d39\") : remove /var/lib/kubelet/pods/0d290342-e496-47b7-b04f-53943eba8d39/volumes/azure~blobfuse/fuse: directory not empty"

could you run mount | grep blobfuse to check again? it seems that azure~blobfuse/fuse/data is not a blobfuse mount

mount | grep blobfuse

It's that, the folder is a blobfuse because I browse for all files like Azure portal with explorer, but if you run : mount | grep blobfuse , the results is empty againg, so the pod is not existing now, not mounted as a partition but folder local folder is connected yet to blobfuse. Could be a inode problems? I think is a low level problem here , right? Because the pod is not already exists in node:
/# mount |grep -i fuse
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)

mount | grep blobfuse

It's that, the folder is a blobfuse because I browse for all files like Azure portal with explorer, but if you run : mount | grep blobfuse , the results is empty againg, so the pod is not existing now, not mounted as a partition but local folder is connected yet to blobfuse. Could be a inode problems? I think is a low level problem here , right? Because the pod is not already exists in node:
/# mount |grep -i fuse
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)

For that problem, I did tests with yaml deployment, and when I remove "--file-cache-timeout-in-seconds=120" , the blobfuse is mounted , is displayed with (mount | grep blobfuse) ... and pod has the folder mounted, but the folder is empty (not connected really to blob or not display successfully any files), so here there is another problem to investigate and I decided type again de original config to resolve.

Were you able to fix it?

not sure how to fix it, in that case, you can delete that pod with unmount failure directly.

I suspect it's due to subPath: folderInBlob, would you not use subPath and try again?

I suspect it's due to subPath: folderInBlob, would you not use subPath and try again?

When I remove subPath... same the volume is not mounted. I paste the node messages when I remove the pod:
Feb 22 13:16:44 aks-**********-28286210-vmss00000C kubelet[21384]: I0222 13:16:44.949941   21384 kubelet.go:1926] SyncLoop (DELETE, "api"): "test-blobfuse-4q68n_logstreaming(d58cc83d-f65b-4d76-9416-266659a92a9a)"
Feb 22 13:17:15 aks-***********-28286210-vmss00000C kubelet[21384]: I0222 13:17:15.892151   21384 kubelet.go:1955] SyncLoop (PLEG): "test-blobfuse-4q68n_logstreaming(d58cc83d-f65b-4d76-9416-266659a92a9a)", event: &pleg.PodLifecycleEvent{ID:"d58cc83d-f65b-4d76-9416-266659a92a9a", Type:"ContainerDied", Data:"4cd9064ec1d91b43208b3c9933ed0f4ba466c21b7f3d054a6f98ff3c0c181641"}
Feb 22 13:17:15 aks-************-28286210-vmss00000C kubelet[21384]: I0222 13:17:15.892593   21384 kubelet.go:1955] SyncLoop (PLEG): "test-blobfuse-4q68n_logstreaming(d58cc83d-f65b-4d76-9416-266659a92a9a)", event: &pleg.PodLifecycleEvent{ID:"d58cc83d-f65b-4d76-9416-266659a92a9a", Type:"ContainerDied", Data:"ae9efad5ea987e5aa26e646e8e5f679857a37884b5f2e62e26a78bd6364cbcc7"}
Feb 22 13:17:15 aks-***********-28286210-vmss00000C kubelet[21384]: I0222 13:17:15.908516   21384 reflector.go:181] Stopping reflector *v1.Secret (0s) from object-"logstreaming"/"blobfusecreds"
Feb 22 13:17:15 aks-***********-28286210-vmss00000C kubelet[21384]: I0222 13:17:15.971625   21384 reconciler.go:196] operationExecutor.UnmountVolume started for volume "fuse" (UniqueName: "flexvolume-azure/blobfuse/d58cc83d-f65b-4d76-9416-266659a92a9a-fuse") pod "d58cc83d-f65b-4d76-9416-266659a92a9a" (UID: "d58cc83d-f65b-4d76-9416-266659a92a9a")
Feb 22 13:17:15 aks-*************-28286210-vmss00000C kubelet[21384]: I0222 13:17:15.977750   21384 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "flexvolume-azure/blobfuse/d58cc83d-f65b-4d76-9416-266659a92a9a-fuse" (OuterVolumeSpecName: "fuse") pod "d58cc83d-f65b-4d76-9416-266659a92a9a" (UID: "d58cc83d-f65b-4d76-9416-266659a92a9a"). InnerVolumeSpecName "fuse". PluginName "flexvolume-azure/blobfuse", VolumeGidValue ""
Feb 22 13:17:16 aks-***********-28286210-vmss00000C kubelet[21384]: I0222 13:17:16.071967   21384 reconciler.go:319] Volume detached for volume "fuse" (UniqueName: "flexvolume-azure/blobfuse/d58cc83d-f65b-4d76-9416-266659a92a9a-fuse") on node "aks-***********-28286210-vmss00000c" DevicePath ""
Feb 22 13:17:16 aks-***********-28286210-vmss00000C kubelet[21384]: I0222 13:17:16.917291   21384 kubelet.go:1926] SyncLoop (DELETE, "api"): "test-blobfuse-4q68n_logstreaming(d58cc83d-f65b-4d76-9416-266659a92a9a)"
Feb 22 13:17:16 aks-***********-28286210-vmss00000C kubelet[21384]: I0222 13:17:16.921308   21384 kubelet.go:1920] SyncLoop (REMOVE, "api"): "test-blobfuse-4q68n_logstreaming(d58cc83d-f65b-4d76-9416-266659a92a9a)"
Feb 22 13:17:16 aks-************-28286210-vmss00000C kubelet[21384]: I0222 13:17:16.921342   21384 kubelet.go:2118] Failed to delete pod "test-blobfuse-4q68n_logstreaming(d58cc83d-f65b-4d76-9416-266659a92a9a)", err: pod not found

Coud be related this other bug with fuse driver in ubuntu? azure-storage-fuse#428

Coud be related this other bug with fuse driver in ubuntu? azure-storage-fuse#428

could be, would you set --log-level=LOG_DEBUG in mountOptions and try get debugging info again? thanks.

Coud be related this other bug with fuse driver in ubuntu? azure-storage-fuse#428

could be, would you set --log-level=LOG_DEBUG in mountOptions and try get debugging info again? thanks.

Sorry I haven't had a time, I will try to update with results from this point.
On the other hand, the problem could be related with new containerd runtime for kubernetes 1.19-1.2x ? Here explains there is a  different operation because Docker is not present, now is Containerd:
https://docs.microsoft.com/en-us/azure/aks/cluster-configuration#container-runtime-configuration

Hi, I face the same issue. But when I run with -d option I see this error.
Apr 21 15:41:01 aks-agentpool-81843983-vmss000002 blobfuse[28182]: Function azs_destroy, in file /home/amnguye/Desktop/azure-storage-fuse/blobfuse/utilities.cpp, line 523: azs_destroy called.
Apr 21 15:41:01 aks-agentpool-81843983-vmss000002 kubelet[5112]: E0421 15:41:01.882741    5112 driver-call.go:266] Failed to unmarshal output for command: mount, output: "Running scope as unit: run-r8641be8b9a02473ca466dd988a59a218.scope
Running scope as unit: run-r89cf977564c444fc985d39bada8adca5.scope
FUSE library version: 2.9.7
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.31
flags=0x03fffffb
max_readahead=0x00020000
   INIT: 7.19
   flags=0x00000031
   max_readahead=0x00400000
   max_write=0x00400000
   max_background=128
   congestion_threshold=96
   unique: 2, success, outsize: 40
fuse: reading device: Invalid argument
{ \"status\": \"Failure\", \"message\": \"Failed to mount device /dev/ at /var/lib/kubelet/pods/e1a0e429-df65-40d1-bd22-883388f8ef8c/volumes/azure~blobfuse/blobstorage, accountname:*******, error log:Wed Apr 21 15:41:01 UTC 2021 EXEC: /usr/bin/systemd-run --scope -- blobfuse /var/lib/kubelet/pods/e1a0e429-df65-40d1-bd22-883388f8ef8c/volumes/azure~blobfuse/blobstorage --container-name=something --tmp-path=/tmp/something -o allow_other -d --file-cache-timeout-in-seconds=120 -o allow_other --log-level=LOG_DEBUG\" }
", error: invalid character 'R' looking for beginning of value
Apr 21 15:41:01 aks-agentpool-81843983-vmss000002 kubelet[5112]: W0421 15:41:01.882782    5112 driver-call.go:149] FlexVolume: driver call failed: executable: /etc/kubernetes/volumeplugins/azure~blobfuse/blobfuse, args: [mount /var/lib/kubelet/pods/e1a0e429-df65-40d1-bd22-883388f8ef8c/volumes/azure~blobfuse/blobstorage {"container":"something","kubernetes.io/fsType":"","kubernetes.io/pod.name":"******-5fb769cdfc-rh5c7","kubernetes.io/pod.namespace":"bc-v1","kubernetes.io/pod.uid":"e1a0e429-df65-40d1-bd22-883388f8ef8c","kubernetes.io/pvOrVolumeName":"blobstorage","kubernetes.io/readwrite":"rw","kubernetes.io/secret/accountconnectionstring":"RGVmYXVsdEVuZHBvaW50c1Byb3RvY29sPWh0dHBzO0FjY291bnROYW1lPWJjaW5zaWdodGJsb2I7QWNjb3VudEtleT1QdHA0Vk4wWGU2REYwZ1BabDNqZVBaOHhNdEZJUzlpMFM5YVFRMlpSTkg0RThSRUQ3SUh4SDNiMjlvWGttNFgrcFN6enVqcjlNTTc0ZnRRTmpMME9qZz09O0VuZHBvaW50U3VmZml4PWNvcmUud2luZG93cy5uZXQ=","kubernetes.io/secret/accountkey":"UHRwNFZOMFhlNkRGMGdQWmwzamVQWjh4TXRGSVM5aTBTOWFRUTJaUk5INEU4UkVEN0lIeEgzYjI5b1hrbTRYK3BTenp1anI5TU03NGZ0UU5qTDBPamc9PQ==","kubernetes.io/secret/accountname":"YmNpbnNpZ2h0YmxvYg==","kubernetes.io/serviceAccount.name":"default","mountoptions":"-d --file-cache-timeout-in-seconds=120 -o allow_other --log-level=LOG_DEBUG","tmppath":"/tmp/something"}], error: exit status 1, output: "Running scope as unit: run-r8641be8b9a02473ca466dd988a59a218.scope
Running scope as unit: run-r89cf977564c444fc985d39bada8adca5.scope
FUSE library version: 2.9.7
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.31
flags=0x03fffffb
max_readahead=0x00020000
   INIT: 7.19
   flags=0x00000031
   max_readahead=0x00400000
   max_write=0x00400000
   max_background=128
   congestion_threshold=96
   unique: 2, success, outsize: 40
fuse: reading device: Invalid argument
{ \"status\": \"Failure\", \"message\": \"Failed to mount device /dev/ at /var/lib/kubelet/pods/e1a0e429-df65-40d1-bd22-883388f8ef8c/volumes/azure~blobfuse/blobstorage, accountname:********, error log:Wed Apr 21 15:41:01 UTC 2021 EXEC: /usr/bin/systemd-run --scope -- blobfuse /var/lib/kubelet/pods/e1a0e429-df65-40d1-bd22-883388f8ef8c/volumes/azure~blobfuse/blobstorage --container-name=something --tmp-path=/tmp/something -o allow_other -d --file-cache-timeout-in-seconds=120 -o allow_other --log-level=LOG_DEBUG\" }
"
Apr 21 15:41:01 aks-agentpool-81843983-vmss000002 kubelet[5112]: E0421 15:41:01.882916    5112 nestedpendingoperations.go:301] Operation for "{volumeName:azure/blobfuse/e1a0e429-df65-40d1-bd22-883388f8ef8c-blobstorage podName:e1a0e429-df65-40d1-bd22-883388f8ef8c nodeName:}" failed. No retries permitted until 2021-04-21 15:41:33.882893614 +0000 UTC m=+112885.696156115 (durationBeforeRetry 32s). Error: "MountVolume.SetUp failed for volume \"blobstorage\" (UniqueName: \"azure/blobfuse/e1a0e429-df65-40d1-bd22-883388f8ef8c-blobstorage\") pod \"******-5fb769cdfc-rh5c7\" (UID: \"e1a0e429-df65-40d1-bd22-883388f8ef8c\") : invalid character 'R' looking for beginning of value"
Apr 21 15:41:08 aks-agentpool-81843983-vmss000002 kernel: [112947.713732] IPv4: martian source <IP_ADDRESS> from <IP_ADDRESS>, on dev cbr0

is -d mount options valid in blobfuse? @snachiap

Hi, With the mount Option, I added "-d" and I was able to see better logs. It would be helpful to go through this thread to see the compatibility problem between blobfuse and libfuse in details.
https://github.com/Azure/azure-storage-fuse/issues/395

Solved for me! you need to upgrade AKS cluster nodepools or recreate new nodepool to obtain a node with new OS Ubuntu 18.04.5 instead of 18.04.1 which I believe this was the original problem... for example this problem is solved in:

Kernel Version:             5.4.0-1048-azure
OS Image:                   Ubuntu 18.04.5 LTS
Operating System:           linux
Architecture:               amd64
Container Runtime Version:  docker://19.3.14
Kubelet Version:            v1.18.14
Kube-Proxy Version:         v1.18.14

$ kubectl describe pod blobfuse-flexvol-installer-265db -n kube-system |grep Image
    Image:          mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.18

Pending to check in AKS upgraded nodepools with K8s v 1.20.x ...


Not working on iOS 15?

I see the state changes to 2 but I don't see a preview on my device and its not streaming to the twitch rtmp url.
Any tips?

oh you need to request permissions first... I figured the lib did that


[Vercel] site doesn't load after deployment

Description
When deployed, for unknown reason build command on vercel runs twice causing multiple JS files/assets to be created ( attached deployment logs from vercel).
This results in the site crashing due to loading assets from the incorrect build.
build.log
I also can reproduce it by running vercel build && vercel deploy --prebuilt locally.
vite.config.ts
import { sentryVitePlugin } from '@sentry/vite-plugin';
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import vike from 'vike/plugin';
import { fileURLToPath, URL } from 'url';

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [
    react(),
    vike(),
    sentryVitePlugin({
      org: 'greetai-inc',
      project: 'app',
    }),
  ],

  resolve: {
    alias: [
      {
        find: '@',
        replacement: fileURLToPath(new URL('./src', import.meta.url)),
      },
      {
        find: '@greet-button',
        replacement: fileURLToPath(new URL('../greet-button', import.meta.url)),
      },
    ],
  },

  build: {
    commonjsOptions: {
      esmExternals: true,
    },

    sourcemap: true,
  },
});

package.json
  "scripts": {
    "dev": "vite --port 3000",
    "build": "echo 'I will never be called because the `vercel-build` script supersedes me'",
    "vercel-build": "vite build"
  },

Website which crashes: https://greetai-platform-e55y8xyvi-greetai.vercel.app/
Note: some deployments with new commits work, redeploying doesn't work.
Note: project is inside monorepo packages/app

Can you provide a minimal reproduction starting from https://github.com/brillout/vike-with-vercel? I'll re-open then; in the meantime I'm inclined to think it's a user land issue, not a Vike issue.

@brillout Thanks for quick response.
Message from Vercel support in case it helps
Hi Nika,

We really appreciate your patience, I understand your case has been open for some time. Our engineers have been comparing working deployments to the one with the 404 error.

Both of these appear to have "entry-client-routing" files within `assets/entries/`. We initially believed there could be a misconfiguration of caching however it doesn't seem like this project has a caching setup

When reviewing the error on https://greetai-platform-8pxyxwo8s-greetai.vercel.app/ we found that this file isn't present in the deployment. However it appeared in the build logs and has been generated
<script src="/assets/entries/entry-client-routing.2f2f0d6a.js" type="module" async></script>

There is a different "entry-client-routing" file mentioned in the build logs which is not referenced when loading the page
https://greetai-platform-8pxyxwo8s-greetai.vercel.app/assets/entries/entry-client-routing.132fa3c5.js

We believe that there are two "entry-client-routing" files generated on build and sometimes the wrong one is referenced. We can confirm that there are no Vercel platform issues causing this and we believe this is an issue related to the custom build configuration.

Please let us know if you have any other questions.

@brillout  Reproduced it with the repo you linked. Looks like build command executes twice again, which Vercel says is root cause of the site crashing.
Website: https://vike-with-vercel-tau.vercel.app/
Here is the deployment logs
[02:55:44.576] Running build in Washington, D.C., USA (East) – iad1
[02:55:44.797] Cloning github.com/nshelia/vike-with-vercel (Branch: main, Commit: 8f5b65e)
[02:55:45.058] Cloning completed: 269.385ms
[02:55:45.155] Previous build cache not available
[02:55:45.415] Running "vercel build"
[02:55:45.918] Vercel CLI 33.5.5
[02:55:46.727] Detected `pnpm-lock.yaml` version 6 generated by pnpm 8
[02:55:46.735] Installing dependencies...
[02:55:47.450] Lockfile is up to date, resolution step is skipped
[02:55:47.486] Progress: resolved 1, reused 0, downloaded 0, added 0
[02:55:47.536] Packages: +189
[02:55:47.536] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[02:55:48.489] Progress: resolved 189, reused 0, downloaded 137, added 133
[02:55:49.490] Progress: resolved 189, reused 0, downloaded 188, added 188
[02:55:49.543] Progress: resolved 189, reused 0, downloaded 189, added 189, done
[02:55:49.673]<EMAIL_ADDRESS>postinstall$ node install.js
[02:55:49.685]<EMAIL_ADDRESS>postinstall$ node install.js
[02:55:49.686]<EMAIL_ADDRESS>postinstall$ node install.js
[02:55:49.770]<EMAIL_ADDRESS>postinstall: Done
[02:55:49.792]<EMAIL_ADDRESS>postinstall: Done
[02:55:49.814]<EMAIL_ADDRESS>postinstall: Done
[02:55:50.473]
[02:55:50.474] dependencies:
[02:55:50.474] + @vercel/node 3.0.6
[02:55:50.474] + @vitejs/plugin-vue 4.3.4
[02:55:50.474] + @vue/compiler-sfc 3.3.4
[02:55:50.474] + @vue/server-renderer 3.3.4
[02:55:50.474] + typescript 5.2.2
[02:55:50.474] + vike 0.4.142
[02:55:50.474] + vite 4.4.9
[02:55:50.474] + vue 3.3.4
[02:55:50.474]
[02:55:50.482] Done in 3.5s
[02:55:50.519] Running "pnpm run vercel-build"
[02:55:51.034]
[02:55:51.034] > @ vercel-build /vercel/path0
[02:55:51.034] > vite build
[02:55:51.034]
[02:55:51.521] vite v4.4.9 building for production...
[02:55:51.797] transforming...
[02:55:52.942] ✓ 146 modules transformed.
[02:55:53.028] rendering chunks...
[02:55:53.071] computing gzip size...
[02:55:53.080] dist/client/vike.json                                                               0.24 kB │ gzip:  0.18 kB
[02:55:53.080] dist/client/_temp_manifest.json                                                     2.86 kB │ gzip:  0.54 kB
[02:55:53.080] dist/client/assets/static/logo.bebe2e90.svg                                         5.79 kB │ gzip:  1.97 kB
[02:55:53.080] dist/client/assets/static/default.page.server.aa8aa952.css                          0.47 kB │ gzip:  0.28 kB
[02:55:53.081] dist/client/assets/entries/renderer_default.page.server.extractAssets.94b6ea98.js   0.00 kB │ gzip:  0.02 kB
[02:55:53.081] dist/client/assets/entries/pages_about_index.page.3dc13ff6.js                       0.30 kB │ gzip:  0.24 kB
[02:55:53.081] dist/client/assets/entries/pages_index_index.page.c81700fb.js                       0.63 kB │ gzip:  0.40 kB
[02:55:53.081] dist/client/assets/entries/entry-server-routing.421f720b.js                         2.70 kB │ gzip:  1.18 kB
[02:55:53.081] dist/client/assets/entries/renderer_default.page.client.642fb2e6.js                 5.23 kB │ gzip:  2.48 kB
[02:55:53.081] dist/client/assets/chunks/chunk-08c5c351.js                                        28.34 kB │ gzip: 10.17 kB
[02:55:53.081] dist/client/assets/entries/entry-client-routing.c7a5b77b.js                        39.02 kB │ gzip: 12.63 kB
[02:55:53.082] dist/client/assets/chunks/chunk-d680a90b.js                                        48.91 kB │ gzip: 19.94 kB
[02:55:53.097] vite v4.4.9 building SSR bundle for production...
[02:55:53.110] transforming...
[02:55:53.161] ✓ 13 modules transformed.
[02:55:53.173] rendering chunks...
[02:55:53.180] dist/server/package.json                              0.02 kB
[02:55:53.180] dist/server/importBuild.cjs                           0.46 kB
[02:55:53.180] dist/server/chunks/chunk-cc2b3d55.js                  0.19 kB
[02:55:53.180] dist/server/importBuild.mjs                           0.20 kB
[02:55:53.180] dist/server/entries/pages_about_index-page.mjs        0.69 kB
[02:55:53.180] dist/server/entries/pages_index_index-page.mjs        1.79 kB
[02:55:53.180] dist/server/pageFiles.mjs                             1.79 kB
[02:55:53.181] dist/server/entries/renderer_default-page-server.mjs  2.72 kB
[02:55:53.181] ✓ built in 82ms
[02:55:53.181] ✓ built in 1.66s
[02:55:53.226] Detected `pnpm-lock.yaml` version 6 generated by pnpm 8
[02:55:53.226] Running "pnpm run vercel-build"
[02:55:53.742]
[02:55:53.742] > @ vercel-build /vercel/path0
[02:55:53.742] > vite build
[02:55:53.742]
[02:55:54.237] vite v4.4.9 building for production...
[02:55:54.514] transforming...
[02:55:55.702] ✓ 146 modules transformed.
[02:55:55.793] rendering chunks...
[02:55:55.832] computing gzip size...
[02:55:55.838] dist/client/vike.json                                                               0.24 kB │ gzip:  0.18 kB
[02:55:55.839] dist/client/_temp_manifest.json                                                     2.86 kB │ gzip:  0.54 kB
[02:55:55.839] dist/client/assets/static/logo.bebe2e90.svg                                         5.79 kB │ gzip:  1.97 kB
[02:55:55.839] dist/client/assets/static/default.page.server.aa8aa952.css                          0.47 kB │ gzip:  0.28 kB
[02:55:55.839] dist/client/assets/entries/renderer_default.page.server.extractAssets.94b6ea98.js   0.00 kB │ gzip:  0.02 kB
[02:55:55.839] dist/client/assets/entries/pages_about_index.page.3dc13ff6.js                       0.30 kB │ gzip:  0.24 kB
[02:55:55.839] dist/client/assets/entries/pages_index_index.page.c81700fb.js                       0.63 kB │ gzip:  0.40 kB
[02:55:55.839] dist/client/assets/entries/entry-server-routing.421f720b.js                         2.70 kB │ gzip:  1.18 kB
[02:55:55.840] dist/client/assets/entries/renderer_default.page.client.642fb2e6.js                 5.23 kB │ gzip:  2.48 kB
[02:55:55.840] dist/client/assets/chunks/chunk-08c5c351.js                                        28.34 kB │ gzip: 10.17 kB
[02:55:55.840] dist/client/assets/entries/entry-client-routing.c7a5b77b.js                        39.02 kB │ gzip: 12.63 kB
[02:55:55.840] dist/client/assets/chunks/chunk-d680a90b.js                                        48.91 kB │ gzip: 19.94 kB
[02:55:55.856] vite v4.4.9 building SSR bundle for production...
[02:55:55.870] transforming...
[02:55:55.923] ✓ 13 modules transformed.
[02:55:55.936] rendering chunks...
[02:55:55.943] dist/server/package.json                              0.02 kB
[02:55:55.943] dist/server/importBuild.cjs                           0.46 kB
[02:55:55.943] dist/server/chunks/chunk-cc2b3d55.js                  0.19 kB
[02:55:55.943] dist/server/importBuild.mjs                           0.20 kB
[02:55:55.943] dist/server/entries/pages_about_index-page.mjs        0.69 kB
[02:55:55.943] dist/server/entries/pages_index_index-page.mjs        1.79 kB
[02:55:55.943] dist/server/pageFiles.mjs                             1.79 kB
[02:55:55.943] dist/server/entries/renderer_default-page-server.mjs  2.72 kB
[02:55:55.943] ✓ built in 86ms
[02:55:55.943] ✓ built in 1.71s
[02:55:57.543] Build Completed in /vercel/output [11s]
[02:55:57.681] Deploying outputs...
[02:56:05.930]
[02:56:06.178] Deployment completed
[02:56:10.536] Uploading build cache [22.02 MB]...
[02:56:11.834] Build cache uploaded: 1.298s

I cannot reproduce any issue: it successfully deploys.
I'm also seeing that the vercel-build script is run twice, but that's a Vercel question which doesn't seem related to Vike. Maybe you can ask Vercel support why Vercel runs it twice. Let me know what they reply, I'm curious.

From conversation with support

We can see "vite build" appearing twice in build logs as described. When you said it "happens randomly" did you mean that it randomly triggers twice or randomly crashes? The reason we're asking is because every build we have checked so far seems to include "vite build" twice.
Do you know if this happens when building locally? For example:
vercel build && vercel deploy --prebuilt
Further details of the command above are available in our Vercel CLI documentation.

We believe that there are two "entry-client-routing" files generated on build and sometimes the wrong one is referenced. We can confirm that there are no Vercel platform issues causing this and we believe this is an issue related to the custom build configuration.

So, yea, the question very much seems to be why is package.json#vercel-build being trigged twice by Vercel?
Maybe there is an alternative other than using package.json#vercel-build. Last time I tried to use package.json#build it didn't work, but maybe there is (nowadays) a way to make it work.
Keep us updated, it would be nice to update the Vike on Vercel example.

Reverting back to standard build command worked! Tested with 2 projects. Thanks for help.

https://github.com/brillout/vike-with-vercel/pull/2 did the trick. The build is now only executed once.

Thanks for circling back on this. Btw. in case that's an option for your company, we're looking for sponsors.


Roll forward "Add example using Slurm static compute nodes"

Original PR #2393 caused test failures.
Original PR was reverted in GoogleCloudPlatform/hpc-toolkit#2404.
This PR is the roll-forward and contains the fix to the broken tests in a new commit. It makes the new task lists optional by providing a default of an empty list.

I have run one test that exercises each base-integration-test and slurm-integration-test tests to spot check.


[squeue] fix NONE time cli-fallback bug

slurm 22.x can emmit NONE null vals on cmd line. Although we don't guarantee backward compatibility, this doesn't cost us anything to support. Bugs is red-green tested off-course. We might experience this same problem for NAbleFloats, but I'll wait for someone to file a bug against that before implementing it.
resolves #26

@jmcastelo I'll merge this once you confirm that this branch solves your problem. It took no effort to support. If you have a different idea in mind, feel free to open a separate PR


KMS: revoke grant raises NotFound

Revoke grant raises NotFoundException when the grant does not exist.
This aligns moto behavior with boto3/aws.
Also removed test_revoke_grant_by_token as it was just a copy of test_retire_grant_by_token.
There is no revoke_grant_by_token functionality in boto3's kms client.
It must have have been a copy-paste leftover.
Issue 5411

Codecov Report

Merging #5410 (b629327) into master (87683a7) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #5410   +/-   ##
=======================================
  Coverage   96.06%   96.06%
=======================================
  Files         708      708
  Lines       70968    70969    +1
=======================================
+ Hits        68178    68179    +1
  Misses       2790     2790

Flag
Coverage Δ

servertests
35.82% <0.00%> (-0.01%)
:arrow_down:

unittests
96.00% <100.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

moto/kms/models.py
94.09% <100.00%> (+0.01%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

LGTM - thanks for fixing this @xmooner, and welcome to Moto!

Thank you @bblommers !


ddl(ticdc): support batch create table ddl

What problem does this PR solve?

Issue Number: close https://github.com/pingcap/tiflow/issues/11009
What is changed and how it works?
Check List
Tests

Integration test
Manual test (add detailed scripts or steps below)

create changefeed.
set global tidb_enable_fast_create_table=on; for the upstream tidb'
concurrently create 1000 tables in upstream, and insert data
check the tables and data in the downstream tidb

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
`None`.

/retest

/retest


Nginx ingress controller fails to start with custom template

Hi,
I'm trying to create ingress controller with custom template like in the basic example. When I use the same nginx.tmpl file i get the following output from nginx controller log:
I1118 15:13:31.787977       1 main.go:94] Using build: https://github.com/bprashanth/contrib.git - git-92b2bac I1118 15:13:31.829894       1 main.go:123] Validated default/default-http-backend as the default backend W1118 15:13:31.830209       1 ssl.go:132] no file dhparam.pem found in secrets F1118 15:13:31.832432       1 main.go:71] invalid NGINX template: template: nginx.tmpl:210: function "buildAuthLocation" not defined

@vglazkov you need to use customize the template shipped with the docker image (use docker cp to extract the template)
Please keep in mind that the template is tied to go code. That's the reason of the failure.
(the template in the repo uses functions not present in the docker image)

@aledbf thanks for the quick response! I was able to fix the issue with template that you mentioned. I think, that instruction should be fixed, because it directly points to the wrong nginx template file, that i've tried before:
https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/examples/custom-template/README.md

I think, that instruction should be fixed, because it directly points to the wrong nginx template file, that i've tried before:

Noted. Apologies for the error in the example.

I ran into the same problem. Is this the Dockerfile you are referring to? https://github.com/kubernetes/ingress/blob/master/controllers/nginx/rootfs/Dockerfile


http-outbound for Soap call is not emitting "statusCode"

http-outbound for SOAP Webservice outbound call is not emitting "statusCode".
Please could you check on this

'http-outbound'/'https-outbound' events return undefined response statusCode if client uses request, request-promise or any other http libraries that do not use callback when making http.request.   They mention this in the code comment (https://github.com/RuntimeTools/appmetrics/blob/master/probes/http-outbound-probe.js#L91), but there is no explanation on work-around for popular http libraries such as 'request', 'request-promise', 'node-fetch'... (node, 'axios' works)


About ecm package

@TomTranter @rtimms
"ecm" package is depend on the python in between 2.5 to 2.9.
and remaining packages will depend on python above 3.0
How can I use this?
because of the "ecm" package, I am not able to do any simulation.
please help me.

Can you please help to simulate jelly roll thermal model?
Help me with installing ecm package

Oh I see, that's an existing package. yeah it's also what I have called this one. In the process of renaming it JellyBaMM or something. I can help but not immediately. Like I said, it's still being reworked

See here for how help on installation https://github.com/TomTranter/pybamm_pnm/blob/main/.github/workflows/test_on_push.yml

Can you please share some reference papers or books for this modeling?

From: Tom Tranter @.>
Sent: Thursday, January 19, 2023 10:51:17 PM
To: TomTranter/pybamm_pnm @.>
Cc: Sourabh Jadhav @.>; Author @.>
Subject: Re: [TomTranter/pybamm_pnm] About ecm package (Issue #31)
See here for how help on installation https://github.com/TomTranter/pybamm_pnm/blob/main/.github/workflows/test_on_push.yml
—
Reply to this email directly, view it on GitHubhttps://github.com/TomTranter/pybamm_pnm/issues/31#issuecomment-1397338393, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU7LVHUB3VI57O7XYZREXQDWTFZY3ANCNFSM6AAAAAAUAMNJ74.
You are receiving this because you authored the thread.Message ID: @.***>

https://iopscience.iop.org/article/10.1149/1945-7111/aba44b/meta
https://iopscience.iop.org/article/10.1149/1945-7111/abd44f/meta

Sir,
I read the papers and I want to appreciate you.
You are doing a great job.

From: Tom Tranter @.>
Sent: Friday, January 20, 2023 1:43:09 PM
To: TomTranter/pybamm_pnm @.>
Cc: Sourabh Jadhav @.>; Author @.>
Subject: Re: [TomTranter/pybamm_pnm] About ecm package (Issue #31)
https://iopscience.iop.org/article/10.1149/1945-7111/aba44b/meta
https://iopscience.iop.org/article/10.1149/1945-7111/abd44f/meta
—
Reply to this email directly, view it on GitHubhttps://github.com/TomTranter/pybamm_pnm/issues/31#issuecomment-1398047028, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU7LVHXHQUGNAE3ECTOJEMTWTJCJLANCNFSM6AAAAAAUAMNJ74.
You are receiving this because you authored the thread.Message ID: @.***>