Skip to content

[server] parse as stream #133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
trett wants to merge 1 commit into from
+22 −18
Closed

[server] parse as stream #133

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import org.scalajs.linker.interface.ModuleSplitStyle

import scala.sys.process.*

lazy val projectVersion = "2.3.2"
lazy val projectVersion = "2.3.3"
lazy val organizationName = "ru.trett"
lazy val scala3Version = "3.7.4"
lazy val circeVersion = "0.14.15"
Expand Down
2 changes: 1 addition & 1 deletion scripts/local-docker/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ services:
- host.docker.internal:host-gateway

server:
image: server:2.3.2
image: server:2.3.3
container_name: rss_server
restart: always
depends_on:
Expand Down
36 changes: 20 additions & 16 deletions server/src/main/scala/ru/trett/rss/server/services/ChannelService.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import ru.trett.rss.server.repositories.ChannelRepository
import java.time.OffsetDateTime
import scala.concurrent.duration.DurationInt
import scala.jdk.CollectionConverters.*
import org.http4s.Status

class ChannelService(channelRepository: ChannelRepository, client: Client[IO])(using
loggerFactory: LoggerFactory[IO]
Expand Down Expand Up @@ -70,23 +71,26 @@ class ChannelService(channelRepository: ChannelRepository, client: Client[IO])(u
}
channel <-
client
.get(url) { response =>
response.body.compile.to(Array).flatMap { bytes =>
Resource
.fromAutoCloseable(
IO(new XmlReader(new java.io.ByteArrayInputStream(bytes)))
)
.use { reader =>
parse(reader, link).handleErrorWith { error =>
logger.error(error)(
s"Failed to parse the feed: $link"
) *> IO.none
}
}
.get[Option[Channel]](url) {
case Status.Successful(r) => {
Copy link

Copilot AI Dec 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary braces around the case clause. In Scala 3, braces are optional for case clauses and the opening brace here is inconsistent with the following case clause at line 87 which doesn't use braces.

Copilot uses AI. Check for mistakes.
Comment on lines 73 to +75
Copy link

Copilot AI Dec 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client.get[Option[Channel]](url) call uses url derived directly from the HTTP request body parameter link (see ChannelController), allowing any authenticated user to make the server perform HTTP requests to arbitrary hosts (including internal services), which is a classic SSRF risk. An attacker could add a channel URL such as http://127.0.0.1:... or a cloud metadata endpoint and cause the server to fetch internal resources and process their responses. To mitigate this, enforce an allowlist of schemes/hosts (e.g., only http/https and public domains), and optionally block link-local, loopback, and RFC1918 address ranges before invoking client.get.

Copilot uses AI. Check for mistakes.
r.body
.through(fs2.io.toInputStream)
.evalMap(is => {
Resource
.fromAutoCloseable(IO.blocking(new XmlReader(is)))
.use { reader => parse(reader, link) }
})
Comment on lines +78 to +82
Copy link

Copilot AI Dec 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error handling for the parse operation has been removed. The old code had .handleErrorWith that logged parsing errors and returned IO.none. Without this error handling, if the parse operation fails (e.g., malformed XML), the error will propagate up and potentially crash the stream. Consider adding error handling using .handleErrorWith or .attempt to gracefully handle parsing failures.

Copilot uses AI. Check for mistakes.
.compile
.last
.map(_.flatten)
}
}
.handleErrorWith { error =>
logger.error(error)(s"Failed to get channel: $link") *> IO.none
case r =>
r.as[String]
.map(b =>
logger.error(
s"Request failed with status ${r.status.code} and body $b"
)
) *> IO.pure(None)
Comment on lines +89 to +93
Copy link

Copilot AI Dec 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger.error call returns Unit, not IO[Unit], which makes this expression incorrect. The .map(b => logger.error(...)) will execute the logger call eagerly during the map, not as part of the IO effect. This should be flatMap or flatTap with the logger call wrapped in IO, such as: r.as[String].flatTap(b => logger.error(...))

Suggested change
.map(b =>
logger.error(
s"Request failed with status ${r.status.code} and body $b"
)
) *> IO.pure(None)
.flatTap(b =>
logger.error(
s"Request failed with status ${r.status.code} and body $b"
)
)
.as(None)

Copilot uses AI. Check for mistakes.
}
} yield channel

Expand Down