Add Podcast RSS feeds (#5487)
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Add correct feed image to RSS channel
* Prefer HLS videos for podcast RSS
Remove video/stream titles, add optional height attribute to podcast RSS
* Prefix podcast RSS images with root server URL
* Add optional video query support to include captions
* Add transcripts & person images to podcast RSS feed
* Prefer webseed/webtorrent files over HLS fragmented mp4s
* Experimentally adding podcast fields to basic config page
* Add validation for new basic config fields
* Don't include "content" in podcast feed, use full description for "description"
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Add correct feed image to RSS channel
* Prefer HLS videos for podcast RSS
Remove video/stream titles, add optional height attribute to podcast RSS
* Prefix podcast RSS images with root server URL
* Add optional video query support to include captions
* Add transcripts & person images to podcast RSS feed
* Prefer webseed/webtorrent files over HLS fragmented mp4s
* Experimentally adding podcast fields to basic config page
* Add validation for new basic config fields
* Don't include "content" in podcast feed, use full description for "description"
* Add medium/socialInteract to podcast RSS feeds. Use HTML for description
* Change base production image to bullseye, install prosody in image
* Add liveItem and trackers to Podcast RSS feeds
Remove height from alternateEnclosure, replaced with title.
* Clear Podcast RSS feed cache when live streams start/end
* Upgrade to Node 16
* Refactor clearCacheRoute to use ApiCache
* Remove unnecessary type hint
* Update dockerfile to node 16, install python-is-python2
* Use new file paths for captions/playlists
* Fix legacy videos in RSS after migration to object storage
* Improve method of identifying non-fragmented mp4s in podcast RSS feeds
* Don't include fragmented MP4s in podcast RSS feeds
* Add experimental support for podcast:categories on the podcast RSS item
* Fix undefined category when no videos exist
Allows for empty feeds to exist (important for feeds that might only go live)
* Add support for podcast:locked -- user has to opt in to show their email
* Use comma for podcast:categories delimiter
* Make cache clearing async
* Fix merge, temporarily test with pfeed-podcast
* Syntax changes
* Add EXT_MIMETYPE constants for captions
* Update & fix tests, fix enclosure mimetypes, remove admin email
* Add test for podacst:socialInteract
* Add filters hooks for podcast customTags
* Remove showdown, updated to pfeed-podcast 6.1.2
* Add 'action:api.live-video.state.updated' hook
* Avoid assigning undefined category to podcast feeds
* Remove nvmrc
* Remove comment
* Remove unused podcast config
* Remove more unused podcast config
* Fix MChannelAccountDefault type hint missed in merge
* Remove extra line
* Re-add newline in config
* Fix lint errors for isEmailPublic
* Fix thumbnails in podcast feeds
* Requested changes based on review
* Provide podcast rss 2.0 only on video channels
* Misc cleanup for a less messy PR
* Lint fixes
* Remove pfeed-podcast
* Add peertube version to new hooks
* Don't use query include, remove TODO
* Remove film medium hack
* Clear podcast rss cache before video/channel update hooks
* Clear podcast rss cache before video uploaded/deleted hooks
* Refactor podcast feed cache clearing
* Set correct person name from video channel
* Styling
* Fix tests
---------
Co-authored-by: Chocobozzz <me@florianbigard.com>
2023-05-22 10:00:05 -04:00
|
|
|
import express from 'express'
|
|
|
|
import { extname } from 'path'
|
|
|
|
import { Feed } from '@peertube/feed'
|
|
|
|
import { cacheRouteFactory } from '@server/middlewares'
|
|
|
|
import { VideoModel } from '@server/models/video/video'
|
|
|
|
import { VideoInclude } from '@shared/models'
|
|
|
|
import { buildNSFWFilter } from '../../helpers/express-utils'
|
|
|
|
import { MIMETYPES, ROUTE_CACHE_LIFETIME, WEBSERVER } from '../../initializers/constants'
|
|
|
|
import {
|
|
|
|
asyncMiddleware,
|
|
|
|
commonVideosFiltersValidator,
|
|
|
|
feedsFormatValidator,
|
|
|
|
setDefaultVideosSort,
|
|
|
|
setFeedFormatContentType,
|
|
|
|
videoFeedsValidator,
|
|
|
|
videosSortValidator,
|
|
|
|
videoSubscriptionFeedsValidator
|
|
|
|
} from '../../middlewares'
|
|
|
|
import { buildFeedMetadata, getCommonVideoFeedAttributes, getVideosForFeeds, initFeed, sendFeed } from './shared'
|
|
|
|
|
|
|
|
const videoFeedsRouter = express.Router()
|
|
|
|
|
|
|
|
const { middleware: cacheRouteMiddleware } = cacheRouteFactory({
|
|
|
|
headerBlacklist: [ 'Content-Type' ]
|
|
|
|
})
|
|
|
|
|
|
|
|
// ---------------------------------------------------------------------------
|
|
|
|
|
2023-07-25 09:18:10 -04:00
|
|
|
videoFeedsRouter.get('/videos.:format',
|
Add Podcast RSS feeds (#5487)
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Add correct feed image to RSS channel
* Prefer HLS videos for podcast RSS
Remove video/stream titles, add optional height attribute to podcast RSS
* Prefix podcast RSS images with root server URL
* Add optional video query support to include captions
* Add transcripts & person images to podcast RSS feed
* Prefer webseed/webtorrent files over HLS fragmented mp4s
* Experimentally adding podcast fields to basic config page
* Add validation for new basic config fields
* Don't include "content" in podcast feed, use full description for "description"
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Add correct feed image to RSS channel
* Prefer HLS videos for podcast RSS
Remove video/stream titles, add optional height attribute to podcast RSS
* Prefix podcast RSS images with root server URL
* Add optional video query support to include captions
* Add transcripts & person images to podcast RSS feed
* Prefer webseed/webtorrent files over HLS fragmented mp4s
* Experimentally adding podcast fields to basic config page
* Add validation for new basic config fields
* Don't include "content" in podcast feed, use full description for "description"
* Add medium/socialInteract to podcast RSS feeds. Use HTML for description
* Change base production image to bullseye, install prosody in image
* Add liveItem and trackers to Podcast RSS feeds
Remove height from alternateEnclosure, replaced with title.
* Clear Podcast RSS feed cache when live streams start/end
* Upgrade to Node 16
* Refactor clearCacheRoute to use ApiCache
* Remove unnecessary type hint
* Update dockerfile to node 16, install python-is-python2
* Use new file paths for captions/playlists
* Fix legacy videos in RSS after migration to object storage
* Improve method of identifying non-fragmented mp4s in podcast RSS feeds
* Don't include fragmented MP4s in podcast RSS feeds
* Add experimental support for podcast:categories on the podcast RSS item
* Fix undefined category when no videos exist
Allows for empty feeds to exist (important for feeds that might only go live)
* Add support for podcast:locked -- user has to opt in to show their email
* Use comma for podcast:categories delimiter
* Make cache clearing async
* Fix merge, temporarily test with pfeed-podcast
* Syntax changes
* Add EXT_MIMETYPE constants for captions
* Update & fix tests, fix enclosure mimetypes, remove admin email
* Add test for podacst:socialInteract
* Add filters hooks for podcast customTags
* Remove showdown, updated to pfeed-podcast 6.1.2
* Add 'action:api.live-video.state.updated' hook
* Avoid assigning undefined category to podcast feeds
* Remove nvmrc
* Remove comment
* Remove unused podcast config
* Remove more unused podcast config
* Fix MChannelAccountDefault type hint missed in merge
* Remove extra line
* Re-add newline in config
* Fix lint errors for isEmailPublic
* Fix thumbnails in podcast feeds
* Requested changes based on review
* Provide podcast rss 2.0 only on video channels
* Misc cleanup for a less messy PR
* Lint fixes
* Remove pfeed-podcast
* Add peertube version to new hooks
* Don't use query include, remove TODO
* Remove film medium hack
* Clear podcast rss cache before video/channel update hooks
* Clear podcast rss cache before video uploaded/deleted hooks
* Refactor podcast feed cache clearing
* Set correct person name from video channel
* Styling
* Fix tests
---------
Co-authored-by: Chocobozzz <me@florianbigard.com>
2023-05-22 10:00:05 -04:00
|
|
|
videosSortValidator,
|
|
|
|
setDefaultVideosSort,
|
|
|
|
feedsFormatValidator,
|
|
|
|
setFeedFormatContentType,
|
|
|
|
cacheRouteMiddleware(ROUTE_CACHE_LIFETIME.FEEDS),
|
|
|
|
commonVideosFiltersValidator,
|
|
|
|
asyncMiddleware(videoFeedsValidator),
|
|
|
|
asyncMiddleware(generateVideoFeed)
|
|
|
|
)
|
|
|
|
|
2023-07-25 09:18:10 -04:00
|
|
|
videoFeedsRouter.get('/subscriptions.:format',
|
Add Podcast RSS feeds (#5487)
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Add correct feed image to RSS channel
* Prefer HLS videos for podcast RSS
Remove video/stream titles, add optional height attribute to podcast RSS
* Prefix podcast RSS images with root server URL
* Add optional video query support to include captions
* Add transcripts & person images to podcast RSS feed
* Prefer webseed/webtorrent files over HLS fragmented mp4s
* Experimentally adding podcast fields to basic config page
* Add validation for new basic config fields
* Don't include "content" in podcast feed, use full description for "description"
* Initial test implementation of Podcast RSS
This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.
I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.
* Update to pfeed-podcast 1.2.2
* Add correct feed image to RSS channel
* Prefer HLS videos for podcast RSS
Remove video/stream titles, add optional height attribute to podcast RSS
* Prefix podcast RSS images with root server URL
* Add optional video query support to include captions
* Add transcripts & person images to podcast RSS feed
* Prefer webseed/webtorrent files over HLS fragmented mp4s
* Experimentally adding podcast fields to basic config page
* Add validation for new basic config fields
* Don't include "content" in podcast feed, use full description for "description"
* Add medium/socialInteract to podcast RSS feeds. Use HTML for description
* Change base production image to bullseye, install prosody in image
* Add liveItem and trackers to Podcast RSS feeds
Remove height from alternateEnclosure, replaced with title.
* Clear Podcast RSS feed cache when live streams start/end
* Upgrade to Node 16
* Refactor clearCacheRoute to use ApiCache
* Remove unnecessary type hint
* Update dockerfile to node 16, install python-is-python2
* Use new file paths for captions/playlists
* Fix legacy videos in RSS after migration to object storage
* Improve method of identifying non-fragmented mp4s in podcast RSS feeds
* Don't include fragmented MP4s in podcast RSS feeds
* Add experimental support for podcast:categories on the podcast RSS item
* Fix undefined category when no videos exist
Allows for empty feeds to exist (important for feeds that might only go live)
* Add support for podcast:locked -- user has to opt in to show their email
* Use comma for podcast:categories delimiter
* Make cache clearing async
* Fix merge, temporarily test with pfeed-podcast
* Syntax changes
* Add EXT_MIMETYPE constants for captions
* Update & fix tests, fix enclosure mimetypes, remove admin email
* Add test for podacst:socialInteract
* Add filters hooks for podcast customTags
* Remove showdown, updated to pfeed-podcast 6.1.2
* Add 'action:api.live-video.state.updated' hook
* Avoid assigning undefined category to podcast feeds
* Remove nvmrc
* Remove comment
* Remove unused podcast config
* Remove more unused podcast config
* Fix MChannelAccountDefault type hint missed in merge
* Remove extra line
* Re-add newline in config
* Fix lint errors for isEmailPublic
* Fix thumbnails in podcast feeds
* Requested changes based on review
* Provide podcast rss 2.0 only on video channels
* Misc cleanup for a less messy PR
* Lint fixes
* Remove pfeed-podcast
* Add peertube version to new hooks
* Don't use query include, remove TODO
* Remove film medium hack
* Clear podcast rss cache before video/channel update hooks
* Clear podcast rss cache before video uploaded/deleted hooks
* Refactor podcast feed cache clearing
* Set correct person name from video channel
* Styling
* Fix tests
---------
Co-authored-by: Chocobozzz <me@florianbigard.com>
2023-05-22 10:00:05 -04:00
|
|
|
videosSortValidator,
|
|
|
|
setDefaultVideosSort,
|
|
|
|
feedsFormatValidator,
|
|
|
|
setFeedFormatContentType,
|
|
|
|
cacheRouteMiddleware(ROUTE_CACHE_LIFETIME.FEEDS),
|
|
|
|
commonVideosFiltersValidator,
|
|
|
|
asyncMiddleware(videoSubscriptionFeedsValidator),
|
|
|
|
asyncMiddleware(generateVideoFeedForSubscriptions)
|
|
|
|
)
|
|
|
|
|
|
|
|
// ---------------------------------------------------------------------------
|
|
|
|
|
|
|
|
export {
|
|
|
|
videoFeedsRouter
|
|
|
|
}
|
|
|
|
|
|
|
|
// ---------------------------------------------------------------------------
|
|
|
|
|
|
|
|
async function generateVideoFeed (req: express.Request, res: express.Response) {
|
|
|
|
const account = res.locals.account
|
|
|
|
const videoChannel = res.locals.videoChannel
|
|
|
|
|
|
|
|
const { name, description, imageUrl, accountImageUrl, link, accountLink } = await buildFeedMetadata({ videoChannel, account })
|
|
|
|
|
|
|
|
const feed = initFeed({
|
|
|
|
name,
|
|
|
|
description,
|
|
|
|
link,
|
|
|
|
isPodcast: false,
|
|
|
|
imageUrl,
|
|
|
|
author: { name, link: accountLink, imageUrl: accountImageUrl },
|
|
|
|
resourceType: 'videos',
|
|
|
|
queryString: new URL(WEBSERVER.URL + req.url).search
|
|
|
|
})
|
|
|
|
|
|
|
|
const data = await getVideosForFeeds({
|
|
|
|
sort: req.query.sort,
|
|
|
|
nsfw: buildNSFWFilter(res, req.query.nsfw),
|
|
|
|
isLocal: req.query.isLocal,
|
|
|
|
include: req.query.include | VideoInclude.FILES,
|
|
|
|
accountId: account?.id,
|
|
|
|
videoChannelId: videoChannel?.id
|
|
|
|
})
|
|
|
|
|
|
|
|
addVideosToFeed(feed, data)
|
|
|
|
|
|
|
|
// Now the feed generation is done, let's send it!
|
|
|
|
return sendFeed(feed, req, res)
|
|
|
|
}
|
|
|
|
|
|
|
|
async function generateVideoFeedForSubscriptions (req: express.Request, res: express.Response) {
|
|
|
|
const account = res.locals.account
|
|
|
|
const { name, description, imageUrl, link } = await buildFeedMetadata({ account })
|
|
|
|
|
|
|
|
const feed = initFeed({
|
|
|
|
name,
|
|
|
|
description,
|
|
|
|
link,
|
|
|
|
isPodcast: false,
|
|
|
|
imageUrl,
|
|
|
|
resourceType: 'videos',
|
|
|
|
queryString: new URL(WEBSERVER.URL + req.url).search
|
|
|
|
})
|
|
|
|
|
|
|
|
const data = await getVideosForFeeds({
|
|
|
|
sort: req.query.sort,
|
|
|
|
nsfw: buildNSFWFilter(res, req.query.nsfw),
|
|
|
|
isLocal: req.query.isLocal,
|
|
|
|
include: req.query.include | VideoInclude.FILES,
|
|
|
|
displayOnlyForFollower: {
|
|
|
|
actorId: res.locals.user.Account.Actor.id,
|
|
|
|
orLocalVideos: false
|
|
|
|
},
|
|
|
|
user: res.locals.user
|
|
|
|
})
|
|
|
|
|
|
|
|
addVideosToFeed(feed, data)
|
|
|
|
|
|
|
|
// Now the feed generation is done, let's send it!
|
|
|
|
return sendFeed(feed, req, res)
|
|
|
|
}
|
|
|
|
|
|
|
|
// ---------------------------------------------------------------------------
|
|
|
|
|
|
|
|
function addVideosToFeed (feed: Feed, videos: VideoModel[]) {
|
|
|
|
/**
|
|
|
|
* Adding video items to the feed object, one at a time
|
|
|
|
*/
|
|
|
|
for (const video of videos) {
|
|
|
|
const formattedVideoFiles = video.getFormattedAllVideoFilesJSON(false)
|
|
|
|
|
|
|
|
const torrents = formattedVideoFiles.map(videoFile => ({
|
|
|
|
title: video.name,
|
|
|
|
url: videoFile.torrentUrl,
|
|
|
|
size_in_bytes: videoFile.size
|
|
|
|
}))
|
|
|
|
|
|
|
|
const videoFiles = formattedVideoFiles.map(videoFile => {
|
|
|
|
return {
|
|
|
|
type: MIMETYPES.VIDEO.EXT_MIMETYPE[extname(videoFile.fileUrl)],
|
|
|
|
medium: 'video',
|
|
|
|
height: videoFile.resolution.id,
|
|
|
|
fileSize: videoFile.size,
|
|
|
|
url: videoFile.fileUrl,
|
|
|
|
framerate: videoFile.fps,
|
|
|
|
duration: video.duration,
|
|
|
|
lang: video.language
|
|
|
|
}
|
|
|
|
})
|
|
|
|
|
|
|
|
feed.addItem({
|
|
|
|
...getCommonVideoFeedAttributes(video),
|
|
|
|
|
|
|
|
id: WEBSERVER.URL + video.getWatchStaticPath(),
|
|
|
|
author: [
|
|
|
|
{
|
|
|
|
name: video.VideoChannel.getDisplayName(),
|
|
|
|
link: video.VideoChannel.getClientUrl()
|
|
|
|
}
|
|
|
|
],
|
|
|
|
torrents,
|
|
|
|
|
|
|
|
// Enclosure
|
|
|
|
video: videoFiles.length !== 0
|
|
|
|
? {
|
|
|
|
url: videoFiles[0].url,
|
|
|
|
length: videoFiles[0].fileSize,
|
|
|
|
type: videoFiles[0].type
|
|
|
|
}
|
|
|
|
: undefined,
|
|
|
|
|
|
|
|
// Media RSS
|
|
|
|
videos: videoFiles,
|
|
|
|
|
|
|
|
embed: {
|
|
|
|
url: WEBSERVER.URL + video.getEmbedStaticPath(),
|
|
|
|
allowFullscreen: true
|
|
|
|
},
|
|
|
|
player: {
|
|
|
|
url: WEBSERVER.URL + video.getWatchStaticPath()
|
|
|
|
},
|
|
|
|
community: {
|
|
|
|
statistics: {
|
|
|
|
views: video.views
|
|
|
|
}
|
|
|
|
}
|
|
|
|
})
|
|
|
|
}
|
|
|
|
}
|