Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Follow up for: Use the umbrella crate for the parachain template #5993 #7464

Open
wants to merge 55 commits into
base: master
Choose a base branch
from

Conversation

seemantaggarwal
Copy link
Contributor

@seemantaggarwal seemantaggarwal commented Feb 5, 2025

Resolving sc-tracing not being resolved when imported through the polkadot sdk

@seemantaggarwal seemantaggarwal changed the title removing unused imports Follow up for: Use the umbrella crate for the parachain template #5993 Feb 5, 2025
@serban300 serban300 added the R0-silent Changes should not be mentioned in any release notes label Feb 6, 2025
serban300
serban300 previously approved these changes Feb 6, 2025
@serban300 serban300 dismissed their stale review February 6, 2025 11:31

Will take another look when the PR is ready for review

@iulianbarbu
Copy link
Contributor

iulianbarbu commented Feb 12, 2025

The node crate still fails to build because an error is raised here during the compilation, when the macro is expanded: https://github.com/paritytech/polkadot-sdk/blob/master/substrate/client/tracing/proc-macro/src/lib.rs#L123. crate_name function searches through the node's Cargo.toml for the sc-tracing dependency, or for the renamed version of the same crate (https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml), but it can not find it, since we rely on polkadot-sdk with its node feature, which pulls the sc-tracing crate and re-exports it, and the compilation fails as a result.

I would attempt a fix in sc-tracing-proc-macro crate, maybe based on cargo-metadata (https://docs.rs/cargo_metadata/latest/cargo_metadata/), where we parse through the dependency graph. Not sure how easy it is, but I would check for our current crate dependencies, search for polkadot-sdk, and then iterate through its dependencies (and here check if sc-tracing exists ). I am not sure if sc-tracing will show up no matter what polkadot-sdk features we enable, because if this is the case, then we might report sc-tracing can be used in the crate that uses polkadot-sdk, but in practice this would fail, because no enabled features pulls it in. However, this is already better since for the cases where sc-tracing is pulled in because of an enabled feature that pulls it in, the compilation wouldn't fail (assuming use polkadot-sdk::* is used in the file where sc_tracing is used).

@seemantaggarwal
Copy link
Contributor Author

Maybe I can try it out instead, we can discuss of what and how, that way, we try it out, and I learn something, 2 birds one stone ;)

@iulianbarbu
Copy link
Contributor

iulianbarbu commented Feb 13, 2025

Just posting the code change and the file. It successfully builds now.

	let crate_name = match crate_name("sc-tracing") {
		Ok(FoundCrate::Itself) => Ident::new("sc_tracing", Span::call_site()),
		Ok(FoundCrate::Name(crate_name)) => Ident::new(&crate_name, Span::call_site()),
		Err(_) => match crate_name("polkadot-sdk") {
			Ok(FoundCrate::Itself) | Ok(FoundCrate::Name(_)) =>
				Ident::new("sc_tracing", Span::call_site()),
			Err(_) => return Error::new(Span::call_site(), "Neither one of `sc-tracing` or `polkadot-sdk` hasn't been found in your dependencies, or no `polkadot-sdk`'s enabled feature pulls in `sc-tracing`.").to_compile_error().into(),
		},
	};

Ok(FoundCrate::Name(crate_name)) => Ident::new(&crate_name, Span::call_site()),
Err(e) => return Error::new(Span::call_site(), e).to_compile_error().into(),
let crate_name = match crate_name("polkadot-sdk") {
Ok(FoundCrate::Name(sdk_name)) => Ident::new(sdk_name.as_str(), Span::call_site()),
Copy link
Member

@ggwpez ggwpez Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check how this is used; path sc-tracing::logging makes sense, but polkadot-sdk::logging does not.
It needs to be changed in this case to polkadot-sdk::sc-tracing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used generate_access_from_frame_or_crate

@serban300 serban300 force-pushed the seemant-parachain-umbrella-crate-mig branch from de994cf to b498920 Compare February 24, 2025 15:03
@serban300 serban300 force-pushed the seemant-parachain-umbrella-crate-mig branch from b498920 to f0128f5 Compare February 24, 2025 15:09
Copy link
Contributor Author

@seemantaggarwal seemantaggarwal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@@ -15,11 +15,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.

use frame_support_procedural_tools::generate_access_from_frame_or_crate;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we can document this somewhere? what this is?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a comment above the function definition:

/// Generate the crate access for the crate using 2018 syntax.
///
/// If `frame` is in scope, it will use `polkadot_sdk_frame::deps::<def_crate>`. Else, it will try
/// and find `<def_crate>` directly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This functionality seems to be shared between frame and substrate client. I would not pull in frame namespaced crates as dependencies for substrate/client crates (FYI, a quick fuzzy search for frame in substrate/client is not referencing any frame dependency). I wonder if we should add this (and maybe other related logic) to an existing shared place or a new one - e.g. substrate/utils (although, there isn't a crate for something like it, and we'd need to create one).

Also, the function docs miss to mention that it looks for polkadot-sdk as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, also thought of that, but I don't know. I didn't completely check, but I guess frame-support-procedural-tools should contain pretty generic methods. If it does, I think we could just use it. Maybe we should rename it or move it under primitives. I have to check.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure.

@ggwpez @kianenigma what do you think ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we shouldn't bring in frame_support_procedural_tools but rather create a helper function, somewhere accessible to the tracing proc macro, that similarly resolves dependencies.

it will check if polkadot-sdk is in scope, use that, else look for the crate name directly.

It should not care about polkadot-sdk-frame, because this is all client side code, which is why I am saying we should not use frame_support_procedural_tools (other than it being a weird dependency).

@seemantaggarwal
Copy link
Contributor Author

/cmd fmt

Copy link
Contributor

@iulianbarbu iulianbarbu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parachain template workspace still fails to build and the error originates in the prefix_logs_with macro.

@seemantaggarwal
Copy link
Contributor Author

The parachain template workspace still fails to build and the error originates in the prefix_logs_with macro.

yeah, I will take a look, I am not entirely sure why and what other options i can try, clearly tried quite a few things. i am still confused as to why cargo build works but cargo nexttest run fails

Signed-off-by: Iulian Barbu <[email protected]>
Copy link
Contributor

@serban300 serban300 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just left a nit. Otherwise looks good

/// Resolve the correct path for sc_tracing:
/// - If `polkadot-sdk` is in scope, returns a Path corresponding to `polkadot_sdk::sc_tracing`
/// - Otherwise, falls back to `sc_tracing`
pub fn resolve_sc_tracing() -> Result<Path> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to put this logic in a separate function and file if we don't need to reuse it.

@paritytech-workflow-stopper
Copy link

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/13875873988
Failed job name: fmt

@seemantaggarwal
Copy link
Contributor Author

/cmd fmt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
R0-silent Changes should not be mentioned in any release notes T17-Templates This PR/Issue is related to templates
Projects
Status: Milestone 1
Development

Successfully merging this pull request may close these issues.

5 participants